{"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1356640463", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1356640463, "node_id": "IC_kwDOBm6k_c5Q3LDP", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-18T02:45:18Z", "updated_at": "2022-12-18T02:45:18Z", "author_association": "OWNER", "body": "... and with this change, the following now works correctly:\r\n\r\n```\r\n% datasette install datasette-gunicorn\r\n% datasette gunicorn fixtures.db -p 8855\r\n[2022-12-17 18:44:29 -0800] [7651] [INFO] Starting gunicorn 20.1.0\r\n[2022-12-17 18:44:29 -0800] [7651] [INFO] Listening at: http://127.0.0.1:8855 (7651)\r\n[2022-12-17 18:44:29 -0800] [7651] [INFO] Using worker: uvicorn.workers.UvicornWorker\r\n[2022-12-17 18:44:29 -0800] [7653] [INFO] Booting worker with pid: 7653\r\n[2022-12-17 18:44:29 -0800] [7653] [INFO] Started server process [7653]\r\n[2022-12-17 18:44:29 -0800] [7653] [INFO] Waiting for application startup.\r\n[2022-12-17 18:44:29 -0800] [7653] [INFO] Application startup complete.\r\n```\r\nSo this issue is now fixed!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1356640266", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1356640266, "node_id": "IC_kwDOBm6k_c5Q3LAK", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-18T02:43:00Z", "updated_at": "2022-12-18T02:43:00Z", "author_association": "OWNER", "body": "https://github.com/simonw/datasette/actions/runs/3722908296/jobs/6314093163 shows that new test passing in CI:\r\n\r\n```\r\nGenerated a certificate for 'localhost', '127.0.0.1', '::1'\r\nConfigure your server to use the following files:\r\n cert=/home/runner/work/datasette/datasette/server.pem\r\n key=/home/runner/work/datasette/datasette/server.key\r\nConfigure your client to use the following files:\r\n cert=/home/runner/work/datasette/datasette/client.pem\r\nINFO: Started server process [4036]\r\nINFO: Waiting for application startup.\r\nINFO: Application startup complete.\r\nINFO: Uvicorn running on https://127.0.0.1:8152/ (Press CTRL+C to quit)\r\n % Total % Received % Xferd Average Speed Time Time Time Current\r\n Dload Upload Total Spent Left Speed\r\n\r\nINFO: 127.0.0.1:56726 - \"GET /_memory.json HTTP/1.1\" 200 OK\r\n 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r\n100 213 0 213 0 0 11542 0 --:--:-- --:--:-- --:--:-- 11833\r\nINFO: Shutting down\r\nINFO: Waiting for application shutdown.\r\nINFO: Application shutdown complete.\r\nINFO: Finished server process [4036]\r\n{\"database\": \"_memory\", \"private\": false, \"path\": \"/_memory\", \"size\": 0, \"tables\": [], \"hidden_count\": 0, \"views\": [], \"queries\": [], \"allow_execute_sql\": true, \"table_columns\": {}, \"query_ms\": 1.4545189999921604}0\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1356633937", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1356633937, "node_id": "IC_kwDOBm6k_c5Q3JdR", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-18T02:28:22Z", "updated_at": "2022-12-18T02:28:22Z", "author_association": "OWNER", "body": "This `bash` script does the job:\r\n```bash\r\n#!/bin/bash\r\n\r\n# Generate certificates\r\npython -m trustme\r\n# This creates server.pem, server.key, client.pem\r\n\r\n# Start the server in the background\r\ndatasette --memory \\\r\n --ssl-keyfile=server.key \\\r\n --ssl-certfile=server.pem \\\r\n -p 8152 &\r\n\r\n# Store the background process ID in a variable\r\nserver_pid=$!\r\n\r\n# Wait for the server to start\r\nsleep 2\r\n\r\n# Make a test request using curl\r\ncurl -f --cacert client.pem 'https://localhost:8152/_memory.json'\r\n\r\n# Save curl's exit code (-f option causes it to return one on HTTP errors)\r\ncurl_exit_code=$?\r\n\r\n# Shut down the server\r\nkill $server_pid\r\nsleep 1\r\n\r\n# Clean up the certificates\r\nrm server.pem server.key client.pem\r\n\r\necho $curl_exit_code\r\nexit $curl_exit_code\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1356630092", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1356630092, "node_id": "IC_kwDOBm6k_c5Q3IhM", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-18T02:20:01Z", "updated_at": "2022-12-18T02:20:01Z", "author_association": "OWNER", "body": "Asked ChatGPT:\r\n\r\n> Write a bash script which starts a server in the background using \"datasette -p 8002\", then uses curl to make a test request against it, then shuts the server down again at the end\r\n\r\nIt gave me:\r\n\r\n```bash\r\n#!/bin/bash\r\n\r\n# Start the server in the background\r\ndatasette -p 8002 &\r\n\r\n# Store the background process ID in a variable\r\nserver_pid=$!\r\n\r\n# Make a test request using curl\r\ncurl http://localhost:8002\r\n\r\n# Shut down the server\r\nkill $server_pid\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1356629783", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1356629783, "node_id": "IC_kwDOBm6k_c5Q3IcX", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-18T02:18:43Z", "updated_at": "2022-12-18T02:18:43Z", "author_association": "OWNER", "body": "Various attempts at a fix which didn't work:\r\n\r\n```diff\r\ndiff --git a/tests/conftest.py b/tests/conftest.py\r\nindex 69dee68b..899d36fd 100644\r\n--- a/tests/conftest.py\r\n+++ b/tests/conftest.py\r\n@@ -1,4 +1,3 @@\r\n-import asyncio\r\n import httpx\r\n import os\r\n import pathlib\r\n@@ -6,6 +5,7 @@ import pytest\r\n import pytest_asyncio\r\n import re\r\n import subprocess\r\n+import sys\r\n import tempfile\r\n import time\r\n import trustme\r\n@@ -27,13 +27,23 @@ UNDOCUMENTED_PERMISSIONS = {\r\n _ds_client = None\r\n \r\n \r\n-def wait_until_responds(url, timeout=5.0, client=httpx, **kwargs):\r\n+def wait_until_responds(url, timeout=5.0, client=None, **kwargs):\r\n+ client = client or httpx.Client(**kwargs)\r\n start = time.time()\r\n while time.time() - start < timeout:\r\n try:\r\n- client.get(url, **kwargs)\r\n+ if \"verify\" in kwargs:\r\n+ print(kwargs[\"verify\"])\r\n+ print(\r\n+ \"Contents of verify file: {}\".format(\r\n+ open(kwargs.get(\"verify\")).read()\r\n+ )\r\n+ )\r\n+ print(\"client = {}, kwargs = {}\".format(client, kwargs))\r\n+ client.get(url)\r\n return\r\n- except httpx.ConnectError:\r\n+ except (httpx.ConnectError, httpx.RemoteProtocolError) as ex:\r\n+ print(ex)\r\n time.sleep(0.1)\r\n raise AssertionError(\"Timed out waiting for {} to respond\".format(url))\r\n \r\n@@ -166,7 +176,7 @@ def check_permission_actions_are_documented():\r\n @pytest.fixture(scope=\"session\")\r\n def ds_localhost_http_server():\r\n ds_proc = subprocess.Popen(\r\n- [\"datasette\", \"--memory\", \"-p\", \"8041\"],\r\n+ [sys.executable, \"-m\", \"datasette\", \"--memory\", \"-p\", \"8041\"],\r\n stdout=subprocess.PIPE,\r\n stderr=subprocess.STDOUT,\r\n # Avoid FileNotFoundError: [Errno 2] No such file or directory:\r\n@@ -180,7 +190,7 @@ def ds_localhost_http_server():\r\n ds_proc.terminate()\r\n \r\n \r\n-@pytest.fixture(scope=\"session\")\r\n+@pytest.fixture\r\n def ds_localhost_https_server(tmp_path_factory):\r\n cert_directory = tmp_path_factory.mktemp(\"certs\")\r\n ca = trustme.CA()\r\n@@ -194,6 +204,8 @@ def ds_localhost_https_server(tmp_path_factory):\r\n ca.cert_pem.write_to_path(path=client_cert)\r\n ds_proc = subprocess.Popen(\r\n [\r\n+ sys.executable,\r\n+ \"-m\",\r\n \"datasette\",\r\n \"--memory\",\r\n \"-p\",\r\n@@ -207,7 +219,11 @@ def ds_localhost_https_server(tmp_path_factory):\r\n stderr=subprocess.STDOUT,\r\n cwd=tempfile.gettempdir(),\r\n )\r\n- wait_until_responds(\"http://localhost:8042/\", verify=client_cert)\r\n+ wait_until_responds(\r\n+ \"http://localhost:8042/_memory.json\",\r\n+ verify=client_cert,\r\n+ headers={\"Connection\": \"close\"},\r\n+ )\r\n # Check it started successfully\r\n assert not ds_proc.poll(), ds_proc.stdout.read().decode(\"utf-8\")\r\n yield ds_proc, client_cert\r\ndiff --git a/tests/test_cli_serve_server.py b/tests/test_cli_serve_server.py\r\nindex 1c31e2a3..9320b623 100644\r\n--- a/tests/test_cli_serve_server.py\r\n+++ b/tests/test_cli_serve_server.py\r\n@@ -16,7 +16,11 @@ def test_serve_localhost_http(ds_localhost_http_server):\r\n @pytest.mark.serial\r\n def test_serve_localhost_https(ds_localhost_https_server):\r\n _, client_cert = ds_localhost_https_server\r\n- response = httpx.get(\"https://localhost:8042/_memory.json\", verify=client_cert)\r\n+ response = httpx.get(\r\n+ \"https://localhost:8042/_memory.json\",\r\n+ verify=client_cert,\r\n+ headers={\"Connection\": \"close\"},\r\n+ )\r\n assert {\r\n \"database\": \"_memory\",\r\n \"path\": \"/_memory\",\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1356627931", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1356627931, "node_id": "IC_kwDOBm6k_c5Q3H_b", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-18T02:13:01Z", "updated_at": "2022-12-18T02:13:01Z", "author_association": "OWNER", "body": "Rather than continue to bang my head against this, I'm tempted to rewrite this test to happen outside of Python world - in a bash script run by GitHub Actions, for example.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1356627331", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1356627331, "node_id": "IC_kwDOBm6k_c5Q3H2D", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-18T02:11:17Z", "updated_at": "2022-12-18T02:11:17Z", "author_association": "OWNER", "body": "This issue might be relevant, but I tried the suggested fix in there (`Connection: close` on the incoming requests) and it didn't fix my problem:\r\n- https://github.com/encode/httpx/discussions/2056", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1356626334", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1356626334, "node_id": "IC_kwDOBm6k_c5Q3Hme", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-18T02:04:01Z", "updated_at": "2022-12-18T02:04:07Z", "author_association": "OWNER", "body": "I used the steps to test manually from this comment: https://github.com/simonw/datasette/issues/1221#issuecomment-777901052\r\n\r\nIn one terminal:\r\n```\r\ncd /tmp\r\npython -m trustme\r\ndatasette --memory --ssl-keyfile=/tmp/server.key --ssl-certfile=/tmp/server.pem -p 8003\r\n```\r\nThen in another terminal:\r\n```\r\ncurl --cacert /tmp/client.pem 'https://localhost:8003/_memory.json'\r\n```\r\nThis worked correctly, outputting the expected JSON.\r\n\r\nSo the feature still works, it's just the test that is broken for some reason.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1356625642", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1356625642, "node_id": "IC_kwDOBm6k_c5Q3Hbq", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-18T02:00:57Z", "updated_at": "2022-12-18T02:00:57Z", "author_association": "OWNER", "body": "I added the TLS support here:\r\n- https://github.com/simonw/datasette/issues/1221", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1356625556", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1356625556, "node_id": "IC_kwDOBm6k_c5Q3HaU", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-18T02:00:18Z", "updated_at": "2022-12-18T02:00:18Z", "author_association": "OWNER", "body": "Maybe the reason the ASGI lifespan stuff broke was this line: https://github.com/simonw/datasette/blob/8b73fc6b47dffd8836f5c58aae1e57c1f66a5754/datasette/cli.py#L630-L632", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1356620233", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1356620233, "node_id": "IC_kwDOBm6k_c5Q3GHJ", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-18T01:31:10Z", "updated_at": "2022-12-18T01:31:10Z", "author_association": "OWNER", "body": "During the polling loop it constantly raises:\r\n\r\n`httpx.RemoteProtocolError`: Server disconnected without sending a response", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1356618913", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1356618913, "node_id": "IC_kwDOBm6k_c5Q3Fyh", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-18T01:29:05Z", "updated_at": "2022-12-18T01:29:05Z", "author_association": "OWNER", "body": "Now the only failure is in the `https` test - which fails like this (in CI and on my laptop):\r\n\r\n```\r\n message = str(exc)\r\n> raise mapped_exc(message) from exc\r\nE httpx.RemoteProtocolError: Server disconnected without sending a response.\r\n\r\n/opt/hostedtoolcache/Python/3.11.1/x64/lib/python3.11/site-packages/httpx/_transports/default.py:77: RemoteProtocolError\r\n=========================== short test summary info ============================\r\nERROR tests/test_cli_serve_server.py::test_serve_localhost_https - httpx.RemoteProtocolError: Server disconnected without sending a response.\r\n================= 30 passed, 1264 deselected, 1 error in 6.15s =================\r\n```\r\nThat's this test: https://github.com/simonw/datasette/blob/63fb750f39cac6f49b451387fdff659ecd9edc5c/tests/test_cli_serve_server.py#L16-L24\r\n\r\nAnd this fixture: https://github.com/simonw/datasette/blob/63fb750f39cac6f49b451387fdff659ecd9edc5c/tests/conftest.py#L178-L215\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1356610089", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1356610089, "node_id": "IC_kwDOBm6k_c5Q3Dop", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-18T01:12:39Z", "updated_at": "2022-12-18T01:12:39Z", "author_association": "OWNER", "body": "... and it turns out those tests saved me. Because I forgot to check if `datasette` would actually start a server correctly!\r\n\r\n```\r\n % datasette fixtures.db -p 8852\r\nINFO: Started server process [3538]\r\nINFO: Waiting for application startup.\r\nERROR: Exception in 'lifespan' protocol\r\nTraceback (most recent call last):\r\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/uvicorn/lifespan/on.py\", line 86, in main\r\n await app(scope, self.receive, self.send)\r\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py\", line 78, in __call__\r\n return await self.app(scope, receive, send)\r\n File \"/Users/simon/Dropbox/Development/datasette/datasette/utils/asgi.py\", line 437, in __call__\r\n return await self.asgi(scope, receive, send)\r\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/asgi_csrf.py\", line 39, in app_wrapped_with_csrf\r\n await app(scope, receive, send)\r\n File \"/Users/simon/Dropbox/Development/datasette/datasette/app.py\", line 1457, in __call__\r\n path = scope[\"path\"]\r\nKeyError: 'path'\r\nERROR: Application startup failed. Exiting.\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1356609095", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1356609095, "node_id": "IC_kwDOBm6k_c5Q3DZH", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-18T01:10:43Z", "updated_at": "2022-12-18T01:10:43Z", "author_association": "OWNER", "body": "Improved version of that fixture:\r\n```diff\r\ndiff --git a/tests/conftest.py b/tests/conftest.py\r\nindex 44c44f87..69dee68b 100644\r\n--- a/tests/conftest.py\r\n+++ b/tests/conftest.py\r\n@@ -27,6 +27,17 @@ UNDOCUMENTED_PERMISSIONS = {\r\n _ds_client = None\r\n \r\n \r\n+def wait_until_responds(url, timeout=5.0, client=httpx, **kwargs):\r\n+ start = time.time()\r\n+ while time.time() - start < timeout:\r\n+ try:\r\n+ client.get(url, **kwargs)\r\n+ return\r\n+ except httpx.ConnectError:\r\n+ time.sleep(0.1)\r\n+ raise AssertionError(\"Timed out waiting for {} to respond\".format(url))\r\n+\r\n+\r\n @pytest_asyncio.fixture\r\n async def ds_client():\r\n from datasette.app import Datasette\r\n@@ -161,13 +172,7 @@ def ds_localhost_http_server():\r\n # Avoid FileNotFoundError: [Errno 2] No such file or directory:\r\n cwd=tempfile.gettempdir(),\r\n )\r\n- # Loop until port 8041 serves traffic\r\n- while True:\r\n- try:\r\n- httpx.get(\"http://localhost:8041/\")\r\n- break\r\n- except httpx.ConnectError:\r\n- time.sleep(0.1)\r\n+ wait_until_responds(\"http://localhost:8041/\")\r\n # Check it started successfully\r\n assert not ds_proc.poll(), ds_proc.stdout.read().decode(\"utf-8\")\r\n yield ds_proc\r\n@@ -202,12 +207,7 @@ def ds_localhost_https_server(tmp_path_factory):\r\n stderr=subprocess.STDOUT,\r\n cwd=tempfile.gettempdir(),\r\n )\r\n- while True:\r\n- try:\r\n- httpx.get(\"https://localhost:8042/\", verify=client_cert)\r\n- break\r\n- except httpx.ConnectError:\r\n- time.sleep(0.1)\r\n+ wait_until_responds(\"http://localhost:8042/\", verify=client_cert)\r\n # Check it started successfully\r\n assert not ds_proc.poll(), ds_proc.stdout.read().decode(\"utf-8\")\r\n yield ds_proc, client_cert\r\n@@ -231,12 +231,7 @@ def ds_unix_domain_socket_server(tmp_path_factory):\r\n # Poll until available\r\n transport = httpx.HTTPTransport(uds=uds)\r\n client = httpx.Client(transport=transport)\r\n- while True:\r\n- try:\r\n- client.get(\"http://localhost/_memory.json\")\r\n- break\r\n- except httpx.ConnectError:\r\n- time.sleep(0.1)\r\n+ wait_until_responds(\"http://localhost/_memory.json\", client=client)\r\n # Check it started successfully\r\n assert not ds_proc.poll(), ds_proc.stdout.read().decode(\"utf-8\")\r\n yield ds_proc, uds\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1356600917", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1356600917, "node_id": "IC_kwDOBm6k_c5Q3BZV", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-18T01:02:26Z", "updated_at": "2022-12-18T01:02:26Z", "author_association": "OWNER", "body": "This bit here looks like it could hang!\r\n```python\r\n # Loop until port 8041 serves traffic \r\n while True: \r\n try: \r\n httpx.get(\"http://localhost:8041/\") \r\n break \r\n except httpx.ConnectError: \r\n time.sleep(0.1) \r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1356599930", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1356599930, "node_id": "IC_kwDOBm6k_c5Q3BJ6", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-18T01:01:47Z", "updated_at": "2022-12-18T01:01:47Z", "author_association": "OWNER", "body": "I think that's this test: https://github.com/simonw/datasette/blob/63fb750f39cac6f49b451387fdff659ecd9edc5c/tests/test_cli_serve_server.py#L6-L13\r\n\r\nUsing this fixture: https://github.com/simonw/datasette/blob/63fb750f39cac6f49b451387fdff659ecd9edc5c/tests/conftest.py#L155-L175", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1356596740", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1356596740, "node_id": "IC_kwDOBm6k_c5Q3AYE", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-18T00:59:47Z", "updated_at": "2022-12-18T00:59:47Z", "author_association": "OWNER", "body": "Hitting `Ctrl+C` while using `--full-trace` gave me more clues:\r\n\r\n```\r\n% pytest -m serial tests/test_cli_serve_server.py --full-trace\r\n======================================================= test session starts ========================================================\r\nplatform darwin -- Python 3.10.3, pytest-7.1.3, pluggy-1.0.0\r\nSQLite: 3.39.4\r\nrootdir: /Users/simon/Dropbox/Development/datasette, configfile: pytest.ini\r\nplugins: anyio-3.6.1, xdist-2.5.0, forked-1.4.0, asyncio-0.19.0, timeout-2.1.0, profiling-1.7.0\r\nasyncio: mode=strict\r\ncollected 3 items \r\n\r\ntests/test_cli_serve_server.py ^C^C\r\n\r\n====================================================== no tests ran in 3.49s =======================================================\r\nTraceback (most recent call last):\r\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/httpcore/_exceptions.py\", line 8, in map_exceptions\r\n yield\r\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/httpcore/backends/sync.py\", line 86, in connect_tcp\r\n sock = socket.create_connection(\r\n File \"/Users/simon/.pyenv/versions/3.10.3/lib/python3.10/socket.py\", line 845, in create_connection\r\n raise err\r\n File \"/Users/simon/.pyenv/versions/3.10.3/lib/python3.10/socket.py\", line 833, in create_connection\r\n sock.connect(sa)\r\nConnectionRefusedError: [Errno 61] Connection refused\r\n[...]\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1356595665", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1356595665, "node_id": "IC_kwDOBm6k_c5Q3AHR", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-18T00:58:16Z", "updated_at": "2022-12-18T00:58:16Z", "author_association": "OWNER", "body": "`pytest -m serial` on my Mac laptop also freezes:\r\n\r\n```\r\n(datasette) datasette % pytest -m serial\r\n======================================================= test session starts ========================================================\r\nplatform darwin -- Python 3.10.3, pytest-7.1.3, pluggy-1.0.0\r\nSQLite: 3.39.4\r\nrootdir: /Users/simon/Dropbox/Development/datasette, configfile: pytest.ini\r\nplugins: anyio-3.6.1, xdist-2.5.0, forked-1.4.0, asyncio-0.19.0, timeout-2.1.0, profiling-1.7.0\r\nasyncio: mode=strict\r\ncollected 1295 items / 1264 deselected / 31 selected \r\n\r\ntests/test_package.py . [ 3%]\r\ntests/test_cli_serve_server.py \r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1356489200", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1356489200, "node_id": "IC_kwDOBm6k_c5Q2mHw", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-17T22:29:51Z", "updated_at": "2022-12-17T22:29:51Z", "author_association": "OWNER", "body": "No, it still causes the tests to hang (I let them run for 12 minutes):\r\n\r\n![\"image\"](\"https://user-images.githubusercontent.com/9599/208268223-7fb40e3f-833b-4204-9acb-079adce6452f.png\")
\r\n\r\nInteresting that the regular tests passed an then the `pytest -m serial` ones seem to have failed.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1356487139", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1356487139, "node_id": "IC_kwDOBm6k_c5Q2lnj", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-17T22:16:52Z", "updated_at": "2022-12-17T22:16:52Z", "author_association": "OWNER", "body": "I'm trying this fix again, after a bunch of work on the test suite in:\r\n- #1959", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1353701674", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1353701674, "node_id": "IC_kwDOBm6k_c5Qr9kq", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T21:00:51Z", "updated_at": "2022-12-15T21:00:51Z", "author_association": "OWNER", "body": "OK, I've broken the test suite here.\r\n\r\nI'm going to revert these two commits:\r\n\r\n- https://github.com/simonw/datasette/commit/dc18f62089e5672d03176f217d7840cdafa5c447\r\n- https://github.com/simonw/datasette/commit/51ee8caa4a697fa3f4120e93b1c205b714a6cdc7\r\n\r\nThen I'll do a bunch of work making the test suite more robust before I try this again.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1353694582", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1353694582, "node_id": "IC_kwDOBm6k_c5Qr712", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T20:52:46Z", "updated_at": "2022-12-15T20:52:46Z", "author_association": "OWNER", "body": "Just noticed this: https://github.com/simonw/datasette/actions/runs/3706504228/jobs/6281796135\r\n\r\n![\"image\"](\"https://user-images.githubusercontent.com/9599/207964564-46c31685-db7a-494f-9aab-a611d25e5511.png\")
\r\n\r\nThis suggests that the regular tests passed in CI fine, but the non-serial ones failed.\r\n\r\nI'm going to try running everything using `pytest -n auto` without splitting serial and non-serial tests. Maybe the serial thing isn't needed any more?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1353683238", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1353683238, "node_id": "IC_kwDOBm6k_c5Qr5Em", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T20:42:18Z", "updated_at": "2022-12-15T20:42:18Z", "author_association": "OWNER", "body": "Possibly related issue:\n- https://github.com/pytest-dev/pytest-xdist/issues/60", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1353680261", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1353680261, "node_id": "IC_kwDOBm6k_c5Qr4WF", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T20:39:19Z", "updated_at": "2022-12-15T20:39:19Z", "author_association": "OWNER", "body": "When I hit `Ctr+C` here's the traceback I get:\n```\n^C^CException ignored in: \nTraceback (most recent call last):\n File \"/Users/simon/.pyenv/versions/3.10.3/lib/python3.10/threading.py\", line 1530, in _shutdown\n!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! KeyboardInterrupt !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!\n/Users/simon/.pyenv/versions/3.10.3/lib/python3.10/threading.py:324: KeyboardInterrupt\n(to show a full traceback on KeyboardInterrupt use --full-trace)\nTraceback (most recent call last):\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/bin/pytest\", line 8, in \n atexit_call()\n File \"/Users/simon/.pyenv/versions/3.10.3/lib/python3.10/concurrent/futures/thread.py\", line 31, in _python_exit\n sys.exit(console_main())\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/_pytest/config/__init__.py\", line 187, in console_main\n t.join()\n File \"/Users/simon/.pyenv/versions/3.10.3/lib/python3.10/threading.py\", line 1089, in join\n self._wait_for_tstate_lock()\n File \"/Users/simon/.pyenv/versions/3.10.3/lib/python3.10/threading.py\", line 1109, in _wait_for_tstate_lock\n if lock.acquire(block, timeout):\nKeyboardInterrupt: \n code = main()\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/_pytest/config/__init__.py\", line 164, in main\n ret: Union[ExitCode, int] = config.hook.pytest_cmdline_main(\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/pluggy/_hooks.py\", line 265, in __call__\n return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/pluggy/_manager.py\", line 80, in _hookexec\n return self._inner_hookexec(hook_name, methods, kwargs, firstresult)\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/pluggy/_callers.py\", line 60, in _multicall\n return outcome.get_result()\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/pluggy/_result.py\", line 60, in get_result\n raise ex[1].with_traceback(ex[2])\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/pluggy/_callers.py\", line 39, in _multicall\n res = hook_impl.function(*args)\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/_pytest/main.py\", line 315, in pytest_cmdline_main\n return wrap_session(config, _main)\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/_pytest/main.py\", line 303, in wrap_session\n config.hook.pytest_sessionfinish(\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/pluggy/_hooks.py\", line 265, in __call__\n return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/pluggy/_manager.py\", line 80, in _hookexec\n return self._inner_hookexec(hook_name, methods, kwargs, firstresult)\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/pluggy/_callers.py\", line 55, in _multicall\n gen.send(outcome)\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/_pytest/terminal.py\", line 798, in pytest_sessionfinish\n outcome.get_result()\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/pluggy/_result.py\", line 60, in get_result\n raise ex[1].with_traceback(ex[2])\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/pluggy/_callers.py\", line 39, in _multicall\n res = hook_impl.function(*args)\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/xdist/dsession.py\", line 88, in pytest_sessionfinish\n nm.teardown_nodes()\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/xdist/workermanage.py\", line 79, in teardown_nodes\n self.group.terminate(self.EXIT_TIMEOUT)\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/execnet/multi.py\", line 215, in terminate\n safe_terminate(\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/execnet/multi.py\", line 311, in safe_terminate\n reply.get()\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/execnet/gateway_base.py\", line 206, in get\n self.waitfinish(timeout)\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/execnet/gateway_base.py\", line 213, in waitfinish\n if not self._result_ready.wait(timeout):\n File \"/Users/simon/.pyenv/versions/3.10.3/lib/python3.10/threading.py\", line 600, in wait\n signaled = self._cond.wait(timeout)\n File \"/Users/simon/.pyenv/versions/3.10.3/lib/python3.10/threading.py\", line 320, in wait\n waiter.acquire()\nKeyboardInterrupt\n```\nIt looks to me like this relates to `pytest-xdist` istelf - it's waiting on some locks but `site-packages/xdist/workermanage.py` shows up in that track.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1353516572", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1353516572, "node_id": "IC_kwDOBm6k_c5QrQYc", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T18:15:28Z", "updated_at": "2022-12-15T18:15:28Z", "author_association": "OWNER", "body": "I added `return` to the first line of that test to disable it, then ran again - and now it's hanging at about the same progress point through the tests but in a different test:\n\n\n\nSo this time it was hanging at `test_urlsafe_components()`.\n\nSo it's clearly not the individual tests themselves that are the problem - something about running the entire test suite in one go is incompatible with this change for some reason.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1353512099", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1353512099, "node_id": "IC_kwDOBm6k_c5QrPSj", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T18:11:27Z", "updated_at": "2022-12-15T18:11:27Z", "author_association": "OWNER", "body": "This is surprising!\n\n\n\nThe logs suggest that the test suite hung running this test here:\n\nhttps://github.com/simonw/datasette/blob/dc18f62089e5672d03176f217d7840cdafa5c447/tests/test_utils.py#L55-L58\n\nI find that very hard to believe.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1353509776", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1353509776, "node_id": "IC_kwDOBm6k_c5QrOuQ", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T18:09:26Z", "updated_at": "2022-12-15T18:09:26Z", "author_association": "OWNER", "body": "I added this to `conftest.py`:\n\n```python\n@pytest.fixture(autouse=True)\ndef log_name_of_test_before_test(request):\n # To help identify tests that are hanging\n name = str(request.node)\n with open(\"/tmp/test.log\", \"a\") as f:\n f.write(name + \"\\n\")\n yield\n```\nThis logs out the name of each test to `/tmp/test.log` before running the test - so I can wait until it hangs and see which test it was that caused that.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1353473571", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1353473571, "node_id": "IC_kwDOBm6k_c5QrF4j", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T17:43:28Z", "updated_at": "2022-12-15T17:43:48Z", "author_association": "OWNER", "body": "Running:\r\n\r\n pytest -n auto -x -v\r\n\r\nOn may laptop to see if I can replicate.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1353473086", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1353473086, "node_id": "IC_kwDOBm6k_c5QrFw-", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T17:43:08Z", "updated_at": "2022-12-15T17:43:08Z", "author_association": "OWNER", "body": "It looks like that fix _almost_ works... except it seems to push the tests into an infinite loop or similar? They're not finishing their runs from what I can see.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1353448095", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1353448095, "node_id": "IC_kwDOBm6k_c5Qq_qf", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T17:25:05Z", "updated_at": "2022-12-15T17:25:05Z", "author_association": "OWNER", "body": "So actually that `setup_db()` function I wrote back in 2019 has not been executing for most of Datasette's tests. Which seems bad.\r\n\r\nI'm inclined to ditch `AsgiLifespan` entirely in favour of the mechanism I described above, where `invoke_startup()` is called for every request on the first request processed by the server.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1353443718", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1353443718, "node_id": "IC_kwDOBm6k_c5Qq-mG", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T17:23:12Z", "updated_at": "2022-12-15T17:23:55Z", "author_association": "OWNER", "body": "That may not be the best fix here. It turns out this pattern:\r\n```python\r\n async def get(self, path, **kwargs):\r\n async with httpx.AsyncClient(app=self.app) as client:\r\n return await client.get(self._fix(path), **kwargs)\r\n```\r\nDoesn't trigger that `AsgiLifespan` class.\r\n\r\nI wrote about that previously in this TIL: https://til.simonwillison.net/asgi/lifespan-test-httpx", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1353423584", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1353423584, "node_id": "IC_kwDOBm6k_c5Qq5rg", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T17:13:18Z", "updated_at": "2022-12-15T17:22:59Z", "author_association": "OWNER", "body": "Wow, just spotted this in the code - it turns out I solved this problem a different (and better) way long before i introduced `invoke_startup()`!\r\n\r\nhttps://github.com/simonw/datasette/blob/e054704fb64d1f23154ec43b81b6c9481ff8202f/datasette/app.py#L1416-L1440", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1352674924", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1352674924, "node_id": "IC_kwDOBm6k_c5QoC5s", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T07:46:36Z", "updated_at": "2022-12-15T07:46:36Z", "author_association": "OWNER", "body": "It's possible the fix for this might be for the first incoming HTTP request to trigger `invoke_startup()` if it hasn't been called yet - similar to the hack I put in place for `datasette.client.get()` in tests:\r\n\r\nhttps://github.com/simonw/datasette/blob/e054704fb64d1f23154ec43b81b6c9481ff8202f/datasette/app.py#L1728-L1731\r\n\r\nThis would be a much more elegant fix, I could remove those multiple `invoke_startup()` calls entirely - and remove this tip from the documentation too: https://docs.datasette.io/en/0.63.2/testing_plugins.html#setting-up-a-datasette-test-instance", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1352643333", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1352643333, "node_id": "IC_kwDOBm6k_c5Qn7MF", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T07:07:29Z", "updated_at": "2022-12-15T07:07:29Z", "author_association": "OWNER", "body": "Datasette 0.63 is the release that broke this, thanks to this issue:\r\n\r\n- https://github.com/simonw/datasette/issues/1809", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1352643049", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1352643049, "node_id": "IC_kwDOBm6k_c5Qn7Hp", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T07:07:10Z", "updated_at": "2022-12-15T07:07:10Z", "author_association": "OWNER", "body": "This is definitely a regression: Datasette is meant to work in those environments, and I didn't think to test them when I added the `invoke_startup()` hook.\r\n\r\nCoincidentally I actually built a plugin for running Datasette with Gunicorn just a couple of months ago:\r\n\r\nhttps://datasette.io/plugins/datasette-gunicorn\r\n\r\nAnd I just tested and it has the same bug you describe here! Filed:\r\n\r\n- https://github.com/simonw/datasette-gunicorn/issues/5\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null}