{"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1353701674", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1353701674, "node_id": "IC_kwDOBm6k_c5Qr9kq", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T21:00:51Z", "updated_at": "2022-12-15T21:00:51Z", "author_association": "OWNER", "body": "OK, I've broken the test suite here.\r\n\r\nI'm going to revert these two commits:\r\n\r\n- https://github.com/simonw/datasette/commit/dc18f62089e5672d03176f217d7840cdafa5c447\r\n- https://github.com/simonw/datasette/commit/51ee8caa4a697fa3f4120e93b1c205b714a6cdc7\r\n\r\nThen I'll do a bunch of work making the test suite more robust before I try this again.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1353694582", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1353694582, "node_id": "IC_kwDOBm6k_c5Qr712", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T20:52:46Z", "updated_at": "2022-12-15T20:52:46Z", "author_association": "OWNER", "body": "Just noticed this: https://github.com/simonw/datasette/actions/runs/3706504228/jobs/6281796135\r\n\r\n\"image\"\r\n\r\nThis suggests that the regular tests passed in CI fine, but the non-serial ones failed.\r\n\r\nI'm going to try running everything using `pytest -n auto` without splitting serial and non-serial tests. Maybe the serial thing isn't needed any more?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1353683238", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1353683238, "node_id": "IC_kwDOBm6k_c5Qr5Em", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T20:42:18Z", "updated_at": "2022-12-15T20:42:18Z", "author_association": "OWNER", "body": "Possibly related issue:\n- https://github.com/pytest-dev/pytest-xdist/issues/60", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1353680261", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1353680261, "node_id": "IC_kwDOBm6k_c5Qr4WF", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T20:39:19Z", "updated_at": "2022-12-15T20:39:19Z", "author_association": "OWNER", "body": "When I hit `Ctr+C` here's the traceback I get:\n```\n^C^CException ignored in: \nTraceback (most recent call last):\n File \"/Users/simon/.pyenv/versions/3.10.3/lib/python3.10/threading.py\", line 1530, in _shutdown\n!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! KeyboardInterrupt !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!\n/Users/simon/.pyenv/versions/3.10.3/lib/python3.10/threading.py:324: KeyboardInterrupt\n(to show a full traceback on KeyboardInterrupt use --full-trace)\nTraceback (most recent call last):\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/bin/pytest\", line 8, in \n atexit_call()\n File \"/Users/simon/.pyenv/versions/3.10.3/lib/python3.10/concurrent/futures/thread.py\", line 31, in _python_exit\n sys.exit(console_main())\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/_pytest/config/__init__.py\", line 187, in console_main\n t.join()\n File \"/Users/simon/.pyenv/versions/3.10.3/lib/python3.10/threading.py\", line 1089, in join\n self._wait_for_tstate_lock()\n File \"/Users/simon/.pyenv/versions/3.10.3/lib/python3.10/threading.py\", line 1109, in _wait_for_tstate_lock\n if lock.acquire(block, timeout):\nKeyboardInterrupt: \n code = main()\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/_pytest/config/__init__.py\", line 164, in main\n ret: Union[ExitCode, int] = config.hook.pytest_cmdline_main(\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/pluggy/_hooks.py\", line 265, in __call__\n return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/pluggy/_manager.py\", line 80, in _hookexec\n return self._inner_hookexec(hook_name, methods, kwargs, firstresult)\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/pluggy/_callers.py\", line 60, in _multicall\n return outcome.get_result()\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/pluggy/_result.py\", line 60, in get_result\n raise ex[1].with_traceback(ex[2])\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/pluggy/_callers.py\", line 39, in _multicall\n res = hook_impl.function(*args)\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/_pytest/main.py\", line 315, in pytest_cmdline_main\n return wrap_session(config, _main)\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/_pytest/main.py\", line 303, in wrap_session\n config.hook.pytest_sessionfinish(\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/pluggy/_hooks.py\", line 265, in __call__\n return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/pluggy/_manager.py\", line 80, in _hookexec\n return self._inner_hookexec(hook_name, methods, kwargs, firstresult)\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/pluggy/_callers.py\", line 55, in _multicall\n gen.send(outcome)\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/_pytest/terminal.py\", line 798, in pytest_sessionfinish\n outcome.get_result()\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/pluggy/_result.py\", line 60, in get_result\n raise ex[1].with_traceback(ex[2])\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/pluggy/_callers.py\", line 39, in _multicall\n res = hook_impl.function(*args)\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/xdist/dsession.py\", line 88, in pytest_sessionfinish\n nm.teardown_nodes()\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/xdist/workermanage.py\", line 79, in teardown_nodes\n self.group.terminate(self.EXIT_TIMEOUT)\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/execnet/multi.py\", line 215, in terminate\n safe_terminate(\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/execnet/multi.py\", line 311, in safe_terminate\n reply.get()\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/execnet/gateway_base.py\", line 206, in get\n self.waitfinish(timeout)\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/execnet/gateway_base.py\", line 213, in waitfinish\n if not self._result_ready.wait(timeout):\n File \"/Users/simon/.pyenv/versions/3.10.3/lib/python3.10/threading.py\", line 600, in wait\n signaled = self._cond.wait(timeout)\n File \"/Users/simon/.pyenv/versions/3.10.3/lib/python3.10/threading.py\", line 320, in wait\n waiter.acquire()\nKeyboardInterrupt\n```\nIt looks to me like this relates to `pytest-xdist` istelf - it's waiting on some locks but `site-packages/xdist/workermanage.py` shows up in that track.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1353516572", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1353516572, "node_id": "IC_kwDOBm6k_c5QrQYc", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T18:15:28Z", "updated_at": "2022-12-15T18:15:28Z", "author_association": "OWNER", "body": "I added `return` to the first line of that test to disable it, then ran again - and now it's hanging at about the same progress point through the tests but in a different test:\n\n![Image](https://user-images.githubusercontent.com/9599/207936587-30ebf780-c0da-4e62-b20b-e274e0adaa19.png)\n\nSo this time it was hanging at `test_urlsafe_components()`.\n\nSo it's clearly not the individual tests themselves that are the problem - something about running the entire test suite in one go is incompatible with this change for some reason.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1353512099", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1353512099, "node_id": "IC_kwDOBm6k_c5QrPSj", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T18:11:27Z", "updated_at": "2022-12-15T18:11:27Z", "author_association": "OWNER", "body": "This is surprising!\n\n![Image](https://user-images.githubusercontent.com/9599/207935885-e1f51983-0621-4490-86a6-fafd4c876f41.png)\n\nThe logs suggest that the test suite hung running this test here:\n\nhttps://github.com/simonw/datasette/blob/dc18f62089e5672d03176f217d7840cdafa5c447/tests/test_utils.py#L55-L58\n\nI find that very hard to believe.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1353509776", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1353509776, "node_id": "IC_kwDOBm6k_c5QrOuQ", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T18:09:26Z", "updated_at": "2022-12-15T18:09:26Z", "author_association": "OWNER", "body": "I added this to `conftest.py`:\n\n```python\n@pytest.fixture(autouse=True)\ndef log_name_of_test_before_test(request):\n # To help identify tests that are hanging\n name = str(request.node)\n with open(\"/tmp/test.log\", \"a\") as f:\n f.write(name + \"\\n\")\n yield\n```\nThis logs out the name of each test to `/tmp/test.log` before running the test - so I can wait until it hangs and see which test it was that caused that.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1353473571", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1353473571, "node_id": "IC_kwDOBm6k_c5QrF4j", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T17:43:28Z", "updated_at": "2022-12-15T17:43:48Z", "author_association": "OWNER", "body": "Running:\r\n\r\n pytest -n auto -x -v\r\n\r\nOn may laptop to see if I can replicate.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1353473086", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1353473086, "node_id": "IC_kwDOBm6k_c5QrFw-", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T17:43:08Z", "updated_at": "2022-12-15T17:43:08Z", "author_association": "OWNER", "body": "It looks like that fix _almost_ works... except it seems to push the tests into an infinite loop or similar? They're not finishing their runs from what I can see.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1353448095", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1353448095, "node_id": "IC_kwDOBm6k_c5Qq_qf", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T17:25:05Z", "updated_at": "2022-12-15T17:25:05Z", "author_association": "OWNER", "body": "So actually that `setup_db()` function I wrote back in 2019 has not been executing for most of Datasette's tests. Which seems bad.\r\n\r\nI'm inclined to ditch `AsgiLifespan` entirely in favour of the mechanism I described above, where `invoke_startup()` is called for every request on the first request processed by the server.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1353443718", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1353443718, "node_id": "IC_kwDOBm6k_c5Qq-mG", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T17:23:12Z", "updated_at": "2022-12-15T17:23:55Z", "author_association": "OWNER", "body": "That may not be the best fix here. It turns out this pattern:\r\n```python\r\n async def get(self, path, **kwargs):\r\n async with httpx.AsyncClient(app=self.app) as client:\r\n return await client.get(self._fix(path), **kwargs)\r\n```\r\nDoesn't trigger that `AsgiLifespan` class.\r\n\r\nI wrote about that previously in this TIL: https://til.simonwillison.net/asgi/lifespan-test-httpx", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1353423584", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1353423584, "node_id": "IC_kwDOBm6k_c5Qq5rg", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T17:13:18Z", "updated_at": "2022-12-15T17:22:59Z", "author_association": "OWNER", "body": "Wow, just spotted this in the code - it turns out I solved this problem a different (and better) way long before i introduced `invoke_startup()`!\r\n\r\nhttps://github.com/simonw/datasette/blob/e054704fb64d1f23154ec43b81b6c9481ff8202f/datasette/app.py#L1416-L1440", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1352674924", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1352674924, "node_id": "IC_kwDOBm6k_c5QoC5s", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T07:46:36Z", "updated_at": "2022-12-15T07:46:36Z", "author_association": "OWNER", "body": "It's possible the fix for this might be for the first incoming HTTP request to trigger `invoke_startup()` if it hasn't been called yet - similar to the hack I put in place for `datasette.client.get()` in tests:\r\n\r\nhttps://github.com/simonw/datasette/blob/e054704fb64d1f23154ec43b81b6c9481ff8202f/datasette/app.py#L1728-L1731\r\n\r\nThis would be a much more elegant fix, I could remove those multiple `invoke_startup()` calls entirely - and remove this tip from the documentation too: https://docs.datasette.io/en/0.63.2/testing_plugins.html#setting-up-a-datasette-test-instance", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1352643333", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1352643333, "node_id": "IC_kwDOBm6k_c5Qn7MF", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T07:07:29Z", "updated_at": "2022-12-15T07:07:29Z", "author_association": "OWNER", "body": "Datasette 0.63 is the release that broke this, thanks to this issue:\r\n\r\n- https://github.com/simonw/datasette/issues/1809", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1352643049", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1352643049, "node_id": "IC_kwDOBm6k_c5Qn7Hp", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T07:07:10Z", "updated_at": "2022-12-15T07:07:10Z", "author_association": "OWNER", "body": "This is definitely a regression: Datasette is meant to work in those environments, and I didn't think to test them when I added the `invoke_startup()` hook.\r\n\r\nCoincidentally I actually built a plugin for running Datasette with Gunicorn just a couple of months ago:\r\n\r\nhttps://datasette.io/plugins/datasette-gunicorn\r\n\r\nAnd I just tested and it has the same bug you describe here! Filed:\r\n\r\n- https://github.com/simonw/datasette-gunicorn/issues/5\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null}