github
html_url | issue_url | id | node_id | user | created_at | updated_at | author_association | body | reactions | issue | performed_via_github_app |
---|---|---|---|---|---|---|---|---|---|---|---|
https://github.com/simonw/datasette/issues/1955#issuecomment-1353701674 | https://api.github.com/repos/simonw/datasette/issues/1955 | 1353701674 | IC_kwDOBm6k_c5Qr9kq | 9599 | 2022-12-15T21:00:51Z | 2022-12-15T21:00:51Z | OWNER | OK, I've broken the test suite here. I'm going to revert these two commits: - https://github.com/simonw/datasette/commit/dc18f62089e5672d03176f217d7840cdafa5c447 - https://github.com/simonw/datasette/commit/51ee8caa4a697fa3f4120e93b1c205b714a6cdc7 Then I'll do a bunch of work making the test suite more robust before I try this again. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1496652622 | |
https://github.com/simonw/datasette/issues/1955#issuecomment-1353694582 | https://api.github.com/repos/simonw/datasette/issues/1955 | 1353694582 | IC_kwDOBm6k_c5Qr712 | 9599 | 2022-12-15T20:52:46Z | 2022-12-15T20:52:46Z | OWNER | Just noticed this: https://github.com/simonw/datasette/actions/runs/3706504228/jobs/6281796135 <img width="746" alt="image" src="https://user-images.githubusercontent.com/9599/207964564-46c31685-db7a-494f-9aab-a611d25e5511.png"> This suggests that the regular tests passed in CI fine, but the non-serial ones failed. I'm going to try running everything using `pytest -n auto` without splitting serial and non-serial tests. Maybe the serial thing isn't needed any more? | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1496652622 | |
https://github.com/simonw/datasette/issues/1955#issuecomment-1353683238 | https://api.github.com/repos/simonw/datasette/issues/1955 | 1353683238 | IC_kwDOBm6k_c5Qr5Em | 9599 | 2022-12-15T20:42:18Z | 2022-12-15T20:42:18Z | OWNER | Possibly related issue: - https://github.com/pytest-dev/pytest-xdist/issues/60 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1496652622 | |
https://github.com/simonw/datasette/issues/1955#issuecomment-1353680261 | https://api.github.com/repos/simonw/datasette/issues/1955 | 1353680261 | IC_kwDOBm6k_c5Qr4WF | 9599 | 2022-12-15T20:39:19Z | 2022-12-15T20:39:19Z | OWNER | When I hit `Ctr+C` here's the traceback I get: ``` ^C^CException ignored in: <module 'threading' from '/Users/simon/.pyenv/versions/3.10.3/lib/python3.10/threading.py'> Traceback (most recent call last): File "/Users/simon/.pyenv/versions/3.10.3/lib/python3.10/threading.py", line 1530, in _shutdown !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! KeyboardInterrupt !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! /Users/simon/.pyenv/versions/3.10.3/lib/python3.10/threading.py:324: KeyboardInterrupt (to show a full traceback on KeyboardInterrupt use --full-trace) Traceback (most recent call last): File "/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/bin/pytest", line 8, in <module> atexit_call() File "/Users/simon/.pyenv/versions/3.10.3/lib/python3.10/concurrent/futures/thread.py", line 31, in _python_exit sys.exit(console_main()) File "/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/_pytest/config/__init__.py", line 187, in console_main t.join() File "/Users/simon/.pyenv/versions/3.10.3/lib/python3.10/threading.py", line 1089, in join self._wait_for_tstate_lock() File "/Users/simon/.pyenv/versions/3.10.3/lib/python3.10/threading.py", line 1109, in _wait_for_tstate_lock if lock.acquire(block, timeout): KeyboardInterrupt: code = main() File "/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/_pytest/config/__init__.py", line 164, in main ret: Union[ExitCode, int] = config.hook.pytest_cmdline_main( File "/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/pluggy/_hooks.py", line 265, in __call__ return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult) File "/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/pluggy/_manager.py", line 80, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) File "/Users/simon/.local/share/virtualenvs/datasette-AW… | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1496652622 | |
https://github.com/simonw/datasette/issues/1955#issuecomment-1353516572 | https://api.github.com/repos/simonw/datasette/issues/1955 | 1353516572 | IC_kwDOBm6k_c5QrQYc | 9599 | 2022-12-15T18:15:28Z | 2022-12-15T18:15:28Z | OWNER | I added `return` to the first line of that test to disable it, then ran again - and now it's hanging at about the same progress point through the tests but in a different test: ![Image](https://user-images.githubusercontent.com/9599/207936587-30ebf780-c0da-4e62-b20b-e274e0adaa19.png) So this time it was hanging at `test_urlsafe_components()`. So it's clearly not the individual tests themselves that are the problem - something about running the entire test suite in one go is incompatible with this change for some reason. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1496652622 | |
https://github.com/simonw/datasette/issues/1955#issuecomment-1353512099 | https://api.github.com/repos/simonw/datasette/issues/1955 | 1353512099 | IC_kwDOBm6k_c5QrPSj | 9599 | 2022-12-15T18:11:27Z | 2022-12-15T18:11:27Z | OWNER | This is surprising! ![Image](https://user-images.githubusercontent.com/9599/207935885-e1f51983-0621-4490-86a6-fafd4c876f41.png) The logs suggest that the test suite hung running this test here: https://github.com/simonw/datasette/blob/dc18f62089e5672d03176f217d7840cdafa5c447/tests/test_utils.py#L55-L58 I find that very hard to believe. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1496652622 | |
https://github.com/simonw/datasette/issues/1955#issuecomment-1353509776 | https://api.github.com/repos/simonw/datasette/issues/1955 | 1353509776 | IC_kwDOBm6k_c5QrOuQ | 9599 | 2022-12-15T18:09:26Z | 2022-12-15T18:09:26Z | OWNER | I added this to `conftest.py`: ```python @pytest.fixture(autouse=True) def log_name_of_test_before_test(request): # To help identify tests that are hanging name = str(request.node) with open("/tmp/test.log", "a") as f: f.write(name + "\n") yield ``` This logs out the name of each test to `/tmp/test.log` before running the test - so I can wait until it hangs and see which test it was that caused that. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1496652622 | |
https://github.com/simonw/datasette/issues/1955#issuecomment-1353473571 | https://api.github.com/repos/simonw/datasette/issues/1955 | 1353473571 | IC_kwDOBm6k_c5QrF4j | 9599 | 2022-12-15T17:43:28Z | 2022-12-15T17:43:48Z | OWNER | Running: pytest -n auto -x -v On may laptop to see if I can replicate. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1496652622 | |
https://github.com/simonw/datasette/issues/1955#issuecomment-1353473086 | https://api.github.com/repos/simonw/datasette/issues/1955 | 1353473086 | IC_kwDOBm6k_c5QrFw- | 9599 | 2022-12-15T17:43:08Z | 2022-12-15T17:43:08Z | OWNER | It looks like that fix _almost_ works... except it seems to push the tests into an infinite loop or similar? They're not finishing their runs from what I can see. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1496652622 | |
https://github.com/simonw/datasette/issues/1955#issuecomment-1353448095 | https://api.github.com/repos/simonw/datasette/issues/1955 | 1353448095 | IC_kwDOBm6k_c5Qq_qf | 9599 | 2022-12-15T17:25:05Z | 2022-12-15T17:25:05Z | OWNER | So actually that `setup_db()` function I wrote back in 2019 has not been executing for most of Datasette's tests. Which seems bad. I'm inclined to ditch `AsgiLifespan` entirely in favour of the mechanism I described above, where `invoke_startup()` is called for every request on the first request processed by the server. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1496652622 | |
https://github.com/simonw/datasette/issues/1955#issuecomment-1353443718 | https://api.github.com/repos/simonw/datasette/issues/1955 | 1353443718 | IC_kwDOBm6k_c5Qq-mG | 9599 | 2022-12-15T17:23:12Z | 2022-12-15T17:23:55Z | OWNER | That may not be the best fix here. It turns out this pattern: ```python async def get(self, path, **kwargs): async with httpx.AsyncClient(app=self.app) as client: return await client.get(self._fix(path), **kwargs) ``` Doesn't trigger that `AsgiLifespan` class. I wrote about that previously in this TIL: https://til.simonwillison.net/asgi/lifespan-test-httpx | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1496652622 | |
https://github.com/simonw/datasette/issues/1955#issuecomment-1353423584 | https://api.github.com/repos/simonw/datasette/issues/1955 | 1353423584 | IC_kwDOBm6k_c5Qq5rg | 9599 | 2022-12-15T17:13:18Z | 2022-12-15T17:22:59Z | OWNER | Wow, just spotted this in the code - it turns out I solved this problem a different (and better) way long before i introduced `invoke_startup()`! https://github.com/simonw/datasette/blob/e054704fb64d1f23154ec43b81b6c9481ff8202f/datasette/app.py#L1416-L1440 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1496652622 | |
https://github.com/simonw/datasette/issues/1955#issuecomment-1352674924 | https://api.github.com/repos/simonw/datasette/issues/1955 | 1352674924 | IC_kwDOBm6k_c5QoC5s | 9599 | 2022-12-15T07:46:36Z | 2022-12-15T07:46:36Z | OWNER | It's possible the fix for this might be for the first incoming HTTP request to trigger `invoke_startup()` if it hasn't been called yet - similar to the hack I put in place for `datasette.client.get()` in tests: https://github.com/simonw/datasette/blob/e054704fb64d1f23154ec43b81b6c9481ff8202f/datasette/app.py#L1728-L1731 This would be a much more elegant fix, I could remove those multiple `invoke_startup()` calls entirely - and remove this tip from the documentation too: https://docs.datasette.io/en/0.63.2/testing_plugins.html#setting-up-a-datasette-test-instance | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1496652622 | |
https://github.com/simonw/datasette/issues/1955#issuecomment-1352643333 | https://api.github.com/repos/simonw/datasette/issues/1955 | 1352643333 | IC_kwDOBm6k_c5Qn7MF | 9599 | 2022-12-15T07:07:29Z | 2022-12-15T07:07:29Z | OWNER | Datasette 0.63 is the release that broke this, thanks to this issue: - https://github.com/simonw/datasette/issues/1809 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1496652622 | |
https://github.com/simonw/datasette/issues/1955#issuecomment-1352643049 | https://api.github.com/repos/simonw/datasette/issues/1955 | 1352643049 | IC_kwDOBm6k_c5Qn7Hp | 9599 | 2022-12-15T07:07:10Z | 2022-12-15T07:07:10Z | OWNER | This is definitely a regression: Datasette is meant to work in those environments, and I didn't think to test them when I added the `invoke_startup()` hook. Coincidentally I actually built a plugin for running Datasette with Gunicorn just a couple of months ago: https://datasette.io/plugins/datasette-gunicorn And I just tested and it has the same bug you describe here! Filed: - https://github.com/simonw/datasette-gunicorn/issues/5 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1496652622 |