{"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1357084279", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1357084279, "node_id": "IC_kwDOBm6k_c5Q43Z3", "user": {"value": 178162, "label": "andrewdotn"}, "created_at": "2022-12-19T04:34:16Z", "updated_at": "2022-12-19T04:34:16Z", "author_association": "NONE", "body": "You were super-close on the python version of the test here, changing `http` to `https` on 8b73fc6b47dffd8836f5c58aae1e57c1f66a5754 is enough to pass the test:\r\n\r\n```diff\r\ndiff --git a/tests/conftest.py b/tests/conftest.py\r\nindex 69dee68b4a3f..ba07a11d37f6 100644\r\n--- a/tests/conftest.py\r\n+++ b/tests/conftest.py\r\n@@ -207,7 +207,7 @@ def ds_localhost_https_server(tmp_path_factory):\r\n stderr=subprocess.STDOUT,\r\n cwd=tempfile.gettempdir(),\r\n )\r\n- wait_until_responds(\"http://localhost:8042/\", verify=client_cert)\r\n+ wait_until_responds(\"https://localhost:8042/\", verify=client_cert)\r\n # Check it started successfully\r\n assert not ds_proc.poll(), ds_proc.stdout.read().decode(\"utf-8\")\r\n yield ds_proc, client_cert\r\n```\r\n\r\nMy speculation about what was happening here: when `wait_until_responds()` would time out due to SSL connection problems, because `.terminate()` isn\u2019t in a `finally`, the datasette process wouldn\u2019t get killed. That could (1) hang CI and (2) cause all your future local test runs to mysteriously fail because they\u2019d be secretly talking to that old datasette process still hanging around from a past test run with an old temporary server certificate, and that old server cert wouldn\u2019t validate against your newly-created ca cert.\r\n\r\nA `finally` for `.terminate()` would help; a fancier version could be a context manager for running the external `datasette` process that could:\r\n - ensure the process always exited when no longer needed\r\n - if you want to be fancy, call `terminate()`, `wait()` for a short timeout for the process to exit, then try `kill()` and `wait()` again; raise an exception complaining about the seemingly-unkillable process if all that fails\r\n - raise an error if the process exited with a non-zero error code; here it\u2019s likely that some `datasette` executions were secretly failing with `[Errno 48] error while attempting to bind on address ('127.0.0.1', 8042): address already in use`", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1356640463", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1356640463, "node_id": "IC_kwDOBm6k_c5Q3LDP", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-18T02:45:18Z", "updated_at": "2022-12-18T02:45:18Z", "author_association": "OWNER", "body": "... and with this change, the following now works correctly:\r\n\r\n```\r\n% datasette install datasette-gunicorn\r\n% datasette gunicorn fixtures.db -p 8855\r\n[2022-12-17 18:44:29 -0800] [7651] [INFO] Starting gunicorn 20.1.0\r\n[2022-12-17 18:44:29 -0800] [7651] [INFO] Listening at: http://127.0.0.1:8855 (7651)\r\n[2022-12-17 18:44:29 -0800] [7651] [INFO] Using worker: uvicorn.workers.UvicornWorker\r\n[2022-12-17 18:44:29 -0800] [7653] [INFO] Booting worker with pid: 7653\r\n[2022-12-17 18:44:29 -0800] [7653] [INFO] Started server process [7653]\r\n[2022-12-17 18:44:29 -0800] [7653] [INFO] Waiting for application startup.\r\n[2022-12-17 18:44:29 -0800] [7653] [INFO] Application startup complete.\r\n```\r\nSo this issue is now fixed!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1356640266", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1356640266, "node_id": "IC_kwDOBm6k_c5Q3LAK", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-18T02:43:00Z", "updated_at": "2022-12-18T02:43:00Z", "author_association": "OWNER", "body": "https://github.com/simonw/datasette/actions/runs/3722908296/jobs/6314093163 shows that new test passing in CI:\r\n\r\n```\r\nGenerated a certificate for 'localhost', '127.0.0.1', '::1'\r\nConfigure your server to use the following files:\r\n cert=/home/runner/work/datasette/datasette/server.pem\r\n key=/home/runner/work/datasette/datasette/server.key\r\nConfigure your client to use the following files:\r\n cert=/home/runner/work/datasette/datasette/client.pem\r\nINFO: Started server process [4036]\r\nINFO: Waiting for application startup.\r\nINFO: Application startup complete.\r\nINFO: Uvicorn running on https://127.0.0.1:8152/ (Press CTRL+C to quit)\r\n % Total % Received % Xferd Average Speed Time Time Time Current\r\n Dload Upload Total Spent Left Speed\r\n\r\nINFO: 127.0.0.1:56726 - \"GET /_memory.json HTTP/1.1\" 200 OK\r\n 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r\n100 213 0 213 0 0 11542 0 --:--:-- --:--:-- --:--:-- 11833\r\nINFO: Shutting down\r\nINFO: Waiting for application shutdown.\r\nINFO: Application shutdown complete.\r\nINFO: Finished server process [4036]\r\n{\"database\": \"_memory\", \"private\": false, \"path\": \"/_memory\", \"size\": 0, \"tables\": [], \"hidden_count\": 0, \"views\": [], \"queries\": [], \"allow_execute_sql\": true, \"table_columns\": {}, \"query_ms\": 1.4545189999921604}0\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1356633937", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1356633937, "node_id": "IC_kwDOBm6k_c5Q3JdR", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-18T02:28:22Z", "updated_at": "2022-12-18T02:28:22Z", "author_association": "OWNER", "body": "This `bash` script does the job:\r\n```bash\r\n#!/bin/bash\r\n\r\n# Generate certificates\r\npython -m trustme\r\n# This creates server.pem, server.key, client.pem\r\n\r\n# Start the server in the background\r\ndatasette --memory \\\r\n --ssl-keyfile=server.key \\\r\n --ssl-certfile=server.pem \\\r\n -p 8152 &\r\n\r\n# Store the background process ID in a variable\r\nserver_pid=$!\r\n\r\n# Wait for the server to start\r\nsleep 2\r\n\r\n# Make a test request using curl\r\ncurl -f --cacert client.pem 'https://localhost:8152/_memory.json'\r\n\r\n# Save curl's exit code (-f option causes it to return one on HTTP errors)\r\ncurl_exit_code=$?\r\n\r\n# Shut down the server\r\nkill $server_pid\r\nsleep 1\r\n\r\n# Clean up the certificates\r\nrm server.pem server.key client.pem\r\n\r\necho $curl_exit_code\r\nexit $curl_exit_code\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1356630092", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1356630092, "node_id": "IC_kwDOBm6k_c5Q3IhM", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-18T02:20:01Z", "updated_at": "2022-12-18T02:20:01Z", "author_association": "OWNER", "body": "Asked ChatGPT:\r\n\r\n> Write a bash script which starts a server in the background using \"datasette -p 8002\", then uses curl to make a test request against it, then shuts the server down again at the end\r\n\r\nIt gave me:\r\n\r\n```bash\r\n#!/bin/bash\r\n\r\n# Start the server in the background\r\ndatasette -p 8002 &\r\n\r\n# Store the background process ID in a variable\r\nserver_pid=$!\r\n\r\n# Make a test request using curl\r\ncurl http://localhost:8002\r\n\r\n# Shut down the server\r\nkill $server_pid\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1356629783", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1356629783, "node_id": "IC_kwDOBm6k_c5Q3IcX", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-18T02:18:43Z", "updated_at": "2022-12-18T02:18:43Z", "author_association": "OWNER", "body": "Various attempts at a fix which didn't work:\r\n\r\n```diff\r\ndiff --git a/tests/conftest.py b/tests/conftest.py\r\nindex 69dee68b..899d36fd 100644\r\n--- a/tests/conftest.py\r\n+++ b/tests/conftest.py\r\n@@ -1,4 +1,3 @@\r\n-import asyncio\r\n import httpx\r\n import os\r\n import pathlib\r\n@@ -6,6 +5,7 @@ import pytest\r\n import pytest_asyncio\r\n import re\r\n import subprocess\r\n+import sys\r\n import tempfile\r\n import time\r\n import trustme\r\n@@ -27,13 +27,23 @@ UNDOCUMENTED_PERMISSIONS = {\r\n _ds_client = None\r\n \r\n \r\n-def wait_until_responds(url, timeout=5.0, client=httpx, **kwargs):\r\n+def wait_until_responds(url, timeout=5.0, client=None, **kwargs):\r\n+ client = client or httpx.Client(**kwargs)\r\n start = time.time()\r\n while time.time() - start < timeout:\r\n try:\r\n- client.get(url, **kwargs)\r\n+ if \"verify\" in kwargs:\r\n+ print(kwargs[\"verify\"])\r\n+ print(\r\n+ \"Contents of verify file: {}\".format(\r\n+ open(kwargs.get(\"verify\")).read()\r\n+ )\r\n+ )\r\n+ print(\"client = {}, kwargs = {}\".format(client, kwargs))\r\n+ client.get(url)\r\n return\r\n- except httpx.ConnectError:\r\n+ except (httpx.ConnectError, httpx.RemoteProtocolError) as ex:\r\n+ print(ex)\r\n time.sleep(0.1)\r\n raise AssertionError(\"Timed out waiting for {} to respond\".format(url))\r\n \r\n@@ -166,7 +176,7 @@ def check_permission_actions_are_documented():\r\n @pytest.fixture(scope=\"session\")\r\n def ds_localhost_http_server():\r\n ds_proc = subprocess.Popen(\r\n- [\"datasette\", \"--memory\", \"-p\", \"8041\"],\r\n+ [sys.executable, \"-m\", \"datasette\", \"--memory\", \"-p\", \"8041\"],\r\n stdout=subprocess.PIPE,\r\n stderr=subprocess.STDOUT,\r\n # Avoid FileNotFoundError: [Errno 2] No such file or directory:\r\n@@ -180,7 +190,7 @@ def ds_localhost_http_server():\r\n ds_proc.terminate()\r\n \r\n \r\n-@pytest.fixture(scope=\"session\")\r\n+@pytest.fixture\r\n def ds_localhost_https_server(tmp_path_factory):\r\n cert_directory = tmp_path_factory.mktemp(\"certs\")\r\n ca = trustme.CA()\r\n@@ -194,6 +204,8 @@ def ds_localhost_https_server(tmp_path_factory):\r\n ca.cert_pem.write_to_path(path=client_cert)\r\n ds_proc = subprocess.Popen(\r\n [\r\n+ sys.executable,\r\n+ \"-m\",\r\n \"datasette\",\r\n \"--memory\",\r\n \"-p\",\r\n@@ -207,7 +219,11 @@ def ds_localhost_https_server(tmp_path_factory):\r\n stderr=subprocess.STDOUT,\r\n cwd=tempfile.gettempdir(),\r\n )\r\n- wait_until_responds(\"http://localhost:8042/\", verify=client_cert)\r\n+ wait_until_responds(\r\n+ \"http://localhost:8042/_memory.json\",\r\n+ verify=client_cert,\r\n+ headers={\"Connection\": \"close\"},\r\n+ )\r\n # Check it started successfully\r\n assert not ds_proc.poll(), ds_proc.stdout.read().decode(\"utf-8\")\r\n yield ds_proc, client_cert\r\ndiff --git a/tests/test_cli_serve_server.py b/tests/test_cli_serve_server.py\r\nindex 1c31e2a3..9320b623 100644\r\n--- a/tests/test_cli_serve_server.py\r\n+++ b/tests/test_cli_serve_server.py\r\n@@ -16,7 +16,11 @@ def test_serve_localhost_http(ds_localhost_http_server):\r\n @pytest.mark.serial\r\n def test_serve_localhost_https(ds_localhost_https_server):\r\n _, client_cert = ds_localhost_https_server\r\n- response = httpx.get(\"https://localhost:8042/_memory.json\", verify=client_cert)\r\n+ response = httpx.get(\r\n+ \"https://localhost:8042/_memory.json\",\r\n+ verify=client_cert,\r\n+ headers={\"Connection\": \"close\"},\r\n+ )\r\n assert {\r\n \"database\": \"_memory\",\r\n \"path\": \"/_memory\",\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1356627931", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1356627931, "node_id": "IC_kwDOBm6k_c5Q3H_b", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-18T02:13:01Z", "updated_at": "2022-12-18T02:13:01Z", "author_association": "OWNER", "body": "Rather than continue to bang my head against this, I'm tempted to rewrite this test to happen outside of Python world - in a bash script run by GitHub Actions, for example.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1356627331", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1356627331, "node_id": "IC_kwDOBm6k_c5Q3H2D", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-18T02:11:17Z", "updated_at": "2022-12-18T02:11:17Z", "author_association": "OWNER", "body": "This issue might be relevant, but I tried the suggested fix in there (`Connection: close` on the incoming requests) and it didn't fix my problem:\r\n- https://github.com/encode/httpx/discussions/2056", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1356626334", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1356626334, "node_id": "IC_kwDOBm6k_c5Q3Hme", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-18T02:04:01Z", "updated_at": "2022-12-18T02:04:07Z", "author_association": "OWNER", "body": "I used the steps to test manually from this comment: https://github.com/simonw/datasette/issues/1221#issuecomment-777901052\r\n\r\nIn one terminal:\r\n```\r\ncd /tmp\r\npython -m trustme\r\ndatasette --memory --ssl-keyfile=/tmp/server.key --ssl-certfile=/tmp/server.pem -p 8003\r\n```\r\nThen in another terminal:\r\n```\r\ncurl --cacert /tmp/client.pem 'https://localhost:8003/_memory.json'\r\n```\r\nThis worked correctly, outputting the expected JSON.\r\n\r\nSo the feature still works, it's just the test that is broken for some reason.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1356625642", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1356625642, "node_id": "IC_kwDOBm6k_c5Q3Hbq", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-18T02:00:57Z", "updated_at": "2022-12-18T02:00:57Z", "author_association": "OWNER", "body": "I added the TLS support here:\r\n- https://github.com/simonw/datasette/issues/1221", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1356625556", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1356625556, "node_id": "IC_kwDOBm6k_c5Q3HaU", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-18T02:00:18Z", "updated_at": "2022-12-18T02:00:18Z", "author_association": "OWNER", "body": "Maybe the reason the ASGI lifespan stuff broke was this line: https://github.com/simonw/datasette/blob/8b73fc6b47dffd8836f5c58aae1e57c1f66a5754/datasette/cli.py#L630-L632", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1356620233", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1356620233, "node_id": "IC_kwDOBm6k_c5Q3GHJ", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-18T01:31:10Z", "updated_at": "2022-12-18T01:31:10Z", "author_association": "OWNER", "body": "During the polling loop it constantly raises:\r\n\r\n`httpx.RemoteProtocolError`: Server disconnected without sending a response", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1356618913", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1356618913, "node_id": "IC_kwDOBm6k_c5Q3Fyh", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-18T01:29:05Z", "updated_at": "2022-12-18T01:29:05Z", "author_association": "OWNER", "body": "Now the only failure is in the `https` test - which fails like this (in CI and on my laptop):\r\n\r\n```\r\n message = str(exc)\r\n> raise mapped_exc(message) from exc\r\nE httpx.RemoteProtocolError: Server disconnected without sending a response.\r\n\r\n/opt/hostedtoolcache/Python/3.11.1/x64/lib/python3.11/site-packages/httpx/_transports/default.py:77: RemoteProtocolError\r\n=========================== short test summary info ============================\r\nERROR tests/test_cli_serve_server.py::test_serve_localhost_https - httpx.RemoteProtocolError: Server disconnected without sending a response.\r\n================= 30 passed, 1264 deselected, 1 error in 6.15s =================\r\n```\r\nThat's this test: https://github.com/simonw/datasette/blob/63fb750f39cac6f49b451387fdff659ecd9edc5c/tests/test_cli_serve_server.py#L16-L24\r\n\r\nAnd this fixture: https://github.com/simonw/datasette/blob/63fb750f39cac6f49b451387fdff659ecd9edc5c/tests/conftest.py#L178-L215\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1356610089", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1356610089, "node_id": "IC_kwDOBm6k_c5Q3Dop", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-18T01:12:39Z", "updated_at": "2022-12-18T01:12:39Z", "author_association": "OWNER", "body": "... and it turns out those tests saved me. Because I forgot to check if `datasette` would actually start a server correctly!\r\n\r\n```\r\n % datasette fixtures.db -p 8852\r\nINFO: Started server process [3538]\r\nINFO: Waiting for application startup.\r\nERROR: Exception in 'lifespan' protocol\r\nTraceback (most recent call last):\r\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/uvicorn/lifespan/on.py\", line 86, in main\r\n await app(scope, self.receive, self.send)\r\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py\", line 78, in __call__\r\n return await self.app(scope, receive, send)\r\n File \"/Users/simon/Dropbox/Development/datasette/datasette/utils/asgi.py\", line 437, in __call__\r\n return await self.asgi(scope, receive, send)\r\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/asgi_csrf.py\", line 39, in app_wrapped_with_csrf\r\n await app(scope, receive, send)\r\n File \"/Users/simon/Dropbox/Development/datasette/datasette/app.py\", line 1457, in __call__\r\n path = scope[\"path\"]\r\nKeyError: 'path'\r\nERROR: Application startup failed. Exiting.\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1356609095", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1356609095, "node_id": "IC_kwDOBm6k_c5Q3DZH", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-18T01:10:43Z", "updated_at": "2022-12-18T01:10:43Z", "author_association": "OWNER", "body": "Improved version of that fixture:\r\n```diff\r\ndiff --git a/tests/conftest.py b/tests/conftest.py\r\nindex 44c44f87..69dee68b 100644\r\n--- a/tests/conftest.py\r\n+++ b/tests/conftest.py\r\n@@ -27,6 +27,17 @@ UNDOCUMENTED_PERMISSIONS = {\r\n _ds_client = None\r\n \r\n \r\n+def wait_until_responds(url, timeout=5.0, client=httpx, **kwargs):\r\n+ start = time.time()\r\n+ while time.time() - start < timeout:\r\n+ try:\r\n+ client.get(url, **kwargs)\r\n+ return\r\n+ except httpx.ConnectError:\r\n+ time.sleep(0.1)\r\n+ raise AssertionError(\"Timed out waiting for {} to respond\".format(url))\r\n+\r\n+\r\n @pytest_asyncio.fixture\r\n async def ds_client():\r\n from datasette.app import Datasette\r\n@@ -161,13 +172,7 @@ def ds_localhost_http_server():\r\n # Avoid FileNotFoundError: [Errno 2] No such file or directory:\r\n cwd=tempfile.gettempdir(),\r\n )\r\n- # Loop until port 8041 serves traffic\r\n- while True:\r\n- try:\r\n- httpx.get(\"http://localhost:8041/\")\r\n- break\r\n- except httpx.ConnectError:\r\n- time.sleep(0.1)\r\n+ wait_until_responds(\"http://localhost:8041/\")\r\n # Check it started successfully\r\n assert not ds_proc.poll(), ds_proc.stdout.read().decode(\"utf-8\")\r\n yield ds_proc\r\n@@ -202,12 +207,7 @@ def ds_localhost_https_server(tmp_path_factory):\r\n stderr=subprocess.STDOUT,\r\n cwd=tempfile.gettempdir(),\r\n )\r\n- while True:\r\n- try:\r\n- httpx.get(\"https://localhost:8042/\", verify=client_cert)\r\n- break\r\n- except httpx.ConnectError:\r\n- time.sleep(0.1)\r\n+ wait_until_responds(\"http://localhost:8042/\", verify=client_cert)\r\n # Check it started successfully\r\n assert not ds_proc.poll(), ds_proc.stdout.read().decode(\"utf-8\")\r\n yield ds_proc, client_cert\r\n@@ -231,12 +231,7 @@ def ds_unix_domain_socket_server(tmp_path_factory):\r\n # Poll until available\r\n transport = httpx.HTTPTransport(uds=uds)\r\n client = httpx.Client(transport=transport)\r\n- while True:\r\n- try:\r\n- client.get(\"http://localhost/_memory.json\")\r\n- break\r\n- except httpx.ConnectError:\r\n- time.sleep(0.1)\r\n+ wait_until_responds(\"http://localhost/_memory.json\", client=client)\r\n # Check it started successfully\r\n assert not ds_proc.poll(), ds_proc.stdout.read().decode(\"utf-8\")\r\n yield ds_proc, uds\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1356600917", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1356600917, "node_id": "IC_kwDOBm6k_c5Q3BZV", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-18T01:02:26Z", "updated_at": "2022-12-18T01:02:26Z", "author_association": "OWNER", "body": "This bit here looks like it could hang!\r\n```python\r\n # Loop until port 8041 serves traffic \r\n while True: \r\n try: \r\n httpx.get(\"http://localhost:8041/\") \r\n break \r\n except httpx.ConnectError: \r\n time.sleep(0.1) \r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1356599930", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1356599930, "node_id": "IC_kwDOBm6k_c5Q3BJ6", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-18T01:01:47Z", "updated_at": "2022-12-18T01:01:47Z", "author_association": "OWNER", "body": "I think that's this test: https://github.com/simonw/datasette/blob/63fb750f39cac6f49b451387fdff659ecd9edc5c/tests/test_cli_serve_server.py#L6-L13\r\n\r\nUsing this fixture: https://github.com/simonw/datasette/blob/63fb750f39cac6f49b451387fdff659ecd9edc5c/tests/conftest.py#L155-L175", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1356596740", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1356596740, "node_id": "IC_kwDOBm6k_c5Q3AYE", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-18T00:59:47Z", "updated_at": "2022-12-18T00:59:47Z", "author_association": "OWNER", "body": "Hitting `Ctrl+C` while using `--full-trace` gave me more clues:\r\n\r\n```\r\n% pytest -m serial tests/test_cli_serve_server.py --full-trace\r\n======================================================= test session starts ========================================================\r\nplatform darwin -- Python 3.10.3, pytest-7.1.3, pluggy-1.0.0\r\nSQLite: 3.39.4\r\nrootdir: /Users/simon/Dropbox/Development/datasette, configfile: pytest.ini\r\nplugins: anyio-3.6.1, xdist-2.5.0, forked-1.4.0, asyncio-0.19.0, timeout-2.1.0, profiling-1.7.0\r\nasyncio: mode=strict\r\ncollected 3 items \r\n\r\ntests/test_cli_serve_server.py ^C^C\r\n\r\n====================================================== no tests ran in 3.49s =======================================================\r\nTraceback (most recent call last):\r\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/httpcore/_exceptions.py\", line 8, in map_exceptions\r\n yield\r\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/httpcore/backends/sync.py\", line 86, in connect_tcp\r\n sock = socket.create_connection(\r\n File \"/Users/simon/.pyenv/versions/3.10.3/lib/python3.10/socket.py\", line 845, in create_connection\r\n raise err\r\n File \"/Users/simon/.pyenv/versions/3.10.3/lib/python3.10/socket.py\", line 833, in create_connection\r\n sock.connect(sa)\r\nConnectionRefusedError: [Errno 61] Connection refused\r\n[...]\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1356595665", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1356595665, "node_id": "IC_kwDOBm6k_c5Q3AHR", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-18T00:58:16Z", "updated_at": "2022-12-18T00:58:16Z", "author_association": "OWNER", "body": "`pytest -m serial` on my Mac laptop also freezes:\r\n\r\n```\r\n(datasette) datasette % pytest -m serial\r\n======================================================= test session starts ========================================================\r\nplatform darwin -- Python 3.10.3, pytest-7.1.3, pluggy-1.0.0\r\nSQLite: 3.39.4\r\nrootdir: /Users/simon/Dropbox/Development/datasette, configfile: pytest.ini\r\nplugins: anyio-3.6.1, xdist-2.5.0, forked-1.4.0, asyncio-0.19.0, timeout-2.1.0, profiling-1.7.0\r\nasyncio: mode=strict\r\ncollected 1295 items / 1264 deselected / 31 selected \r\n\r\ntests/test_package.py . [ 3%]\r\ntests/test_cli_serve_server.py \r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1356489200", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1356489200, "node_id": "IC_kwDOBm6k_c5Q2mHw", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-17T22:29:51Z", "updated_at": "2022-12-17T22:29:51Z", "author_association": "OWNER", "body": "No, it still causes the tests to hang (I let them run for 12 minutes):\r\n\r\n\"image\"\r\n\r\nInteresting that the regular tests passed an then the `pytest -m serial` ones seem to have failed.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1356487139", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1356487139, "node_id": "IC_kwDOBm6k_c5Q2lnj", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-17T22:16:52Z", "updated_at": "2022-12-17T22:16:52Z", "author_association": "OWNER", "body": "I'm trying this fix again, after a bunch of work on the test suite in:\r\n- #1959", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1353701674", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1353701674, "node_id": "IC_kwDOBm6k_c5Qr9kq", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T21:00:51Z", "updated_at": "2022-12-15T21:00:51Z", "author_association": "OWNER", "body": "OK, I've broken the test suite here.\r\n\r\nI'm going to revert these two commits:\r\n\r\n- https://github.com/simonw/datasette/commit/dc18f62089e5672d03176f217d7840cdafa5c447\r\n- https://github.com/simonw/datasette/commit/51ee8caa4a697fa3f4120e93b1c205b714a6cdc7\r\n\r\nThen I'll do a bunch of work making the test suite more robust before I try this again.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1353694582", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1353694582, "node_id": "IC_kwDOBm6k_c5Qr712", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T20:52:46Z", "updated_at": "2022-12-15T20:52:46Z", "author_association": "OWNER", "body": "Just noticed this: https://github.com/simonw/datasette/actions/runs/3706504228/jobs/6281796135\r\n\r\n\"image\"\r\n\r\nThis suggests that the regular tests passed in CI fine, but the non-serial ones failed.\r\n\r\nI'm going to try running everything using `pytest -n auto` without splitting serial and non-serial tests. Maybe the serial thing isn't needed any more?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1353683238", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1353683238, "node_id": "IC_kwDOBm6k_c5Qr5Em", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T20:42:18Z", "updated_at": "2022-12-15T20:42:18Z", "author_association": "OWNER", "body": "Possibly related issue:\n- https://github.com/pytest-dev/pytest-xdist/issues/60", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1353680261", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1353680261, "node_id": "IC_kwDOBm6k_c5Qr4WF", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T20:39:19Z", "updated_at": "2022-12-15T20:39:19Z", "author_association": "OWNER", "body": "When I hit `Ctr+C` here's the traceback I get:\n```\n^C^CException ignored in: \nTraceback (most recent call last):\n File \"/Users/simon/.pyenv/versions/3.10.3/lib/python3.10/threading.py\", line 1530, in _shutdown\n!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! KeyboardInterrupt !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!\n/Users/simon/.pyenv/versions/3.10.3/lib/python3.10/threading.py:324: KeyboardInterrupt\n(to show a full traceback on KeyboardInterrupt use --full-trace)\nTraceback (most recent call last):\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/bin/pytest\", line 8, in \n atexit_call()\n File \"/Users/simon/.pyenv/versions/3.10.3/lib/python3.10/concurrent/futures/thread.py\", line 31, in _python_exit\n sys.exit(console_main())\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/_pytest/config/__init__.py\", line 187, in console_main\n t.join()\n File \"/Users/simon/.pyenv/versions/3.10.3/lib/python3.10/threading.py\", line 1089, in join\n self._wait_for_tstate_lock()\n File \"/Users/simon/.pyenv/versions/3.10.3/lib/python3.10/threading.py\", line 1109, in _wait_for_tstate_lock\n if lock.acquire(block, timeout):\nKeyboardInterrupt: \n code = main()\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/_pytest/config/__init__.py\", line 164, in main\n ret: Union[ExitCode, int] = config.hook.pytest_cmdline_main(\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/pluggy/_hooks.py\", line 265, in __call__\n return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/pluggy/_manager.py\", line 80, in _hookexec\n return self._inner_hookexec(hook_name, methods, kwargs, firstresult)\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/pluggy/_callers.py\", line 60, in _multicall\n return outcome.get_result()\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/pluggy/_result.py\", line 60, in get_result\n raise ex[1].with_traceback(ex[2])\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/pluggy/_callers.py\", line 39, in _multicall\n res = hook_impl.function(*args)\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/_pytest/main.py\", line 315, in pytest_cmdline_main\n return wrap_session(config, _main)\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/_pytest/main.py\", line 303, in wrap_session\n config.hook.pytest_sessionfinish(\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/pluggy/_hooks.py\", line 265, in __call__\n return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/pluggy/_manager.py\", line 80, in _hookexec\n return self._inner_hookexec(hook_name, methods, kwargs, firstresult)\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/pluggy/_callers.py\", line 55, in _multicall\n gen.send(outcome)\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/_pytest/terminal.py\", line 798, in pytest_sessionfinish\n outcome.get_result()\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/pluggy/_result.py\", line 60, in get_result\n raise ex[1].with_traceback(ex[2])\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/pluggy/_callers.py\", line 39, in _multicall\n res = hook_impl.function(*args)\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/xdist/dsession.py\", line 88, in pytest_sessionfinish\n nm.teardown_nodes()\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/xdist/workermanage.py\", line 79, in teardown_nodes\n self.group.terminate(self.EXIT_TIMEOUT)\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/execnet/multi.py\", line 215, in terminate\n safe_terminate(\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/execnet/multi.py\", line 311, in safe_terminate\n reply.get()\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/execnet/gateway_base.py\", line 206, in get\n self.waitfinish(timeout)\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/execnet/gateway_base.py\", line 213, in waitfinish\n if not self._result_ready.wait(timeout):\n File \"/Users/simon/.pyenv/versions/3.10.3/lib/python3.10/threading.py\", line 600, in wait\n signaled = self._cond.wait(timeout)\n File \"/Users/simon/.pyenv/versions/3.10.3/lib/python3.10/threading.py\", line 320, in wait\n waiter.acquire()\nKeyboardInterrupt\n```\nIt looks to me like this relates to `pytest-xdist` istelf - it's waiting on some locks but `site-packages/xdist/workermanage.py` shows up in that track.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1353516572", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1353516572, "node_id": "IC_kwDOBm6k_c5QrQYc", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T18:15:28Z", "updated_at": "2022-12-15T18:15:28Z", "author_association": "OWNER", "body": "I added `return` to the first line of that test to disable it, then ran again - and now it's hanging at about the same progress point through the tests but in a different test:\n\n![Image](https://user-images.githubusercontent.com/9599/207936587-30ebf780-c0da-4e62-b20b-e274e0adaa19.png)\n\nSo this time it was hanging at `test_urlsafe_components()`.\n\nSo it's clearly not the individual tests themselves that are the problem - something about running the entire test suite in one go is incompatible with this change for some reason.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1353512099", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1353512099, "node_id": "IC_kwDOBm6k_c5QrPSj", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T18:11:27Z", "updated_at": "2022-12-15T18:11:27Z", "author_association": "OWNER", "body": "This is surprising!\n\n![Image](https://user-images.githubusercontent.com/9599/207935885-e1f51983-0621-4490-86a6-fafd4c876f41.png)\n\nThe logs suggest that the test suite hung running this test here:\n\nhttps://github.com/simonw/datasette/blob/dc18f62089e5672d03176f217d7840cdafa5c447/tests/test_utils.py#L55-L58\n\nI find that very hard to believe.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1353509776", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1353509776, "node_id": "IC_kwDOBm6k_c5QrOuQ", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T18:09:26Z", "updated_at": "2022-12-15T18:09:26Z", "author_association": "OWNER", "body": "I added this to `conftest.py`:\n\n```python\n@pytest.fixture(autouse=True)\ndef log_name_of_test_before_test(request):\n # To help identify tests that are hanging\n name = str(request.node)\n with open(\"/tmp/test.log\", \"a\") as f:\n f.write(name + \"\\n\")\n yield\n```\nThis logs out the name of each test to `/tmp/test.log` before running the test - so I can wait until it hangs and see which test it was that caused that.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1353473571", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1353473571, "node_id": "IC_kwDOBm6k_c5QrF4j", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T17:43:28Z", "updated_at": "2022-12-15T17:43:48Z", "author_association": "OWNER", "body": "Running:\r\n\r\n pytest -n auto -x -v\r\n\r\nOn may laptop to see if I can replicate.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1353473086", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1353473086, "node_id": "IC_kwDOBm6k_c5QrFw-", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T17:43:08Z", "updated_at": "2022-12-15T17:43:08Z", "author_association": "OWNER", "body": "It looks like that fix _almost_ works... except it seems to push the tests into an infinite loop or similar? They're not finishing their runs from what I can see.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1353448095", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1353448095, "node_id": "IC_kwDOBm6k_c5Qq_qf", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T17:25:05Z", "updated_at": "2022-12-15T17:25:05Z", "author_association": "OWNER", "body": "So actually that `setup_db()` function I wrote back in 2019 has not been executing for most of Datasette's tests. Which seems bad.\r\n\r\nI'm inclined to ditch `AsgiLifespan` entirely in favour of the mechanism I described above, where `invoke_startup()` is called for every request on the first request processed by the server.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1353443718", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1353443718, "node_id": "IC_kwDOBm6k_c5Qq-mG", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T17:23:12Z", "updated_at": "2022-12-15T17:23:55Z", "author_association": "OWNER", "body": "That may not be the best fix here. It turns out this pattern:\r\n```python\r\n async def get(self, path, **kwargs):\r\n async with httpx.AsyncClient(app=self.app) as client:\r\n return await client.get(self._fix(path), **kwargs)\r\n```\r\nDoesn't trigger that `AsgiLifespan` class.\r\n\r\nI wrote about that previously in this TIL: https://til.simonwillison.net/asgi/lifespan-test-httpx", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1353423584", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1353423584, "node_id": "IC_kwDOBm6k_c5Qq5rg", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T17:13:18Z", "updated_at": "2022-12-15T17:22:59Z", "author_association": "OWNER", "body": "Wow, just spotted this in the code - it turns out I solved this problem a different (and better) way long before i introduced `invoke_startup()`!\r\n\r\nhttps://github.com/simonw/datasette/blob/e054704fb64d1f23154ec43b81b6c9481ff8202f/datasette/app.py#L1416-L1440", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1352674924", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1352674924, "node_id": "IC_kwDOBm6k_c5QoC5s", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T07:46:36Z", "updated_at": "2022-12-15T07:46:36Z", "author_association": "OWNER", "body": "It's possible the fix for this might be for the first incoming HTTP request to trigger `invoke_startup()` if it hasn't been called yet - similar to the hack I put in place for `datasette.client.get()` in tests:\r\n\r\nhttps://github.com/simonw/datasette/blob/e054704fb64d1f23154ec43b81b6c9481ff8202f/datasette/app.py#L1728-L1731\r\n\r\nThis would be a much more elegant fix, I could remove those multiple `invoke_startup()` calls entirely - and remove this tip from the documentation too: https://docs.datasette.io/en/0.63.2/testing_plugins.html#setting-up-a-datasette-test-instance", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1352643333", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1352643333, "node_id": "IC_kwDOBm6k_c5Qn7MF", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T07:07:29Z", "updated_at": "2022-12-15T07:07:29Z", "author_association": "OWNER", "body": "Datasette 0.63 is the release that broke this, thanks to this issue:\r\n\r\n- https://github.com/simonw/datasette/issues/1809", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1352643049", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1352643049, "node_id": "IC_kwDOBm6k_c5Qn7Hp", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T07:07:10Z", "updated_at": "2022-12-15T07:07:10Z", "author_association": "OWNER", "body": "This is definitely a regression: Datasette is meant to work in those environments, and I didn't think to test them when I added the `invoke_startup()` hook.\r\n\r\nCoincidentally I actually built a plugin for running Datasette with Gunicorn just a couple of months ago:\r\n\r\nhttps://datasette.io/plugins/datasette-gunicorn\r\n\r\nAnd I just tested and it has the same bug you describe here! Filed:\r\n\r\n- https://github.com/simonw/datasette-gunicorn/issues/5\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null}