{"html_url": "https://github.com/simonw/datasette/pull/1960#issuecomment-1353812913", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1960", "id": 1353812913, "node_id": "IC_kwDOBm6k_c5QsYux", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T22:48:54Z", "updated_at": "2022-12-15T22:48:54Z", "author_association": "OWNER", "body": "This is all very broken:\r\n```\r\n% pytest -x --pdb\r\n================================================================================== test session starts ==================================================================================\r\nplatform darwin -- Python 3.10.3, pytest-7.1.3, pluggy-1.0.0\r\nSQLite: 3.39.4\r\nrootdir: /Users/simon/Dropbox/Development/datasette, configfile: pytest.ini\r\nplugins: anyio-3.6.1, xdist-2.5.0, forked-1.4.0, asyncio-0.19.0, timeout-2.1.0, profiling-1.7.0\r\nasyncio: mode=strict\r\ncollected 1295 items \r\n\r\ntests/test_package.py .. [ 0%]\r\ntests/test_cli.py . [ 0%]\r\ntests/test_cli_serve_get.py .. [ 0%]\r\ntests/test_cli.py . [ 0%]\r\ntests/test_black.py . [ 0%]\r\ntests/test_api.py E\r\n>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> traceback >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>\r\n\r\nfixturedef = , request = >\r\n\r\n @pytest.hookimpl(hookwrapper=True)\r\n def pytest_fixture_setup(\r\n fixturedef: FixtureDef, request: SubRequest\r\n ) -> Optional[object]:\r\n \"\"\"Adjust the event loop policy when an event loop is produced.\"\"\"\r\n if fixturedef.argname == \"event_loop\":\r\n outcome = yield\r\n> loop = outcome.get_result()\r\n\r\n/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/pytest_asyncio/plugin.py:377: \r\n_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _\r\n/Users/simon/Dropbox/Development/datasette/tests/conftest.py:30: in event_loop\r\n return asyncio.get_event_loop()\r\n_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _\r\n\r\nself = \r\n\r\n def get_event_loop(self):\r\n \"\"\"Get the event loop for the current context.\r\n \r\n Returns an instance of EventLoop or raises an exception.\r\n \"\"\"\r\n if (self._local._loop is None and\r\n not self._local._set_called and\r\n threading.current_thread() is threading.main_thread()):\r\n self.set_event_loop(self.new_event_loop())\r\n \r\n if self._local._loop is None:\r\n> raise RuntimeError('There is no current event loop in thread %r.'\r\n % threading.current_thread().name)\r\nE RuntimeError: There is no current event loop in thread 'MainThread'.\r\n\r\n/Users/simon/.pyenv/versions/3.10.3/lib/python3.10/asyncio/events.py:656: RuntimeError\r\n>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> entering PDB >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>\r\n\r\n>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PDB post_mortem (IO-capturing turned off) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>\r\n> /Users/simon/.pyenv/versions/3.10.3/lib/python3.10/asyncio/events.py(656)get_event_loop()\r\n-> raise RuntimeError('There is no current event loop in thread %r.'\r\n(Pdb) q\r\n\r\n\r\n=================================================================================== warnings summary ====================================================================================\r\ntests/test_cli.py::test_inspect_cli_writes_to_file\r\ntests/test_cli.py::test_inspect_cli\r\n /Users/simon/Dropbox/Development/datasette/datasette/cli.py:163: DeprecationWarning: There is no current event loop\r\n loop = asyncio.get_event_loop()\r\n\r\ntests/test_cli_serve_get.py::test_serve_with_get_exit_code_for_error\r\ntests/test_cli_serve_get.py::test_serve_with_get\r\n /Users/simon/Dropbox/Development/datasette/datasette/cli.py:596: DeprecationWarning: There is no current event loop\r\n asyncio.get_event_loop().run_until_complete(ds.invoke_startup())\r\n\r\ntests/test_cli_serve_get.py::test_serve_with_get_exit_code_for_error\r\ntests/test_cli_serve_get.py::test_serve_with_get\r\n /Users/simon/Dropbox/Development/datasette/datasette/cli.py:599: DeprecationWarning: There is no current event loop\r\n asyncio.get_event_loop().run_until_complete(check_databases(ds))\r\n\r\ntests/test_api.py::test_homepage\r\n /Users/simon/Dropbox/Development/datasette/tests/conftest.py:30: DeprecationWarning: There is no current event loop\r\n return asyncio.get_event_loop()\r\n\r\n-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html\r\n================================================================================ short test summary info ================================================================================\r\nERROR tests/test_api.py::test_homepage - RuntimeError: There is no current event loop in thread 'MainThread'.\r\n!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!\r\n!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! _pytest.outcomes.Exit: Quitting debugger !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!\r\n======================================================================== 7 passed, 7 warnings, 1 error in 19.15s ========================================================================\r\n(datasette) datasette % \r\n```\r\nThe problem looks to be caused by this:\r\n\r\nhttps://github.com/simonw/datasette/blob/87737aa1ace82fa7b54c60c41471ec9a661f5299/tests/conftest.py#L28-L30\r\n\r\nWhich I found necessary in order to have `async def` fixtures that could be shared on the `scope=\"session\"` basis.\r\n\r\nCan I work around this, or is `scope=\"session\"` for async fixtures incompatible with my test suite for some reason?\r\n\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1499150951, "label": "Port as many tests as possible to async def tests against ds_client"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/pull/1960#issuecomment-1353805839", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1960", "id": 1353805839, "node_id": "IC_kwDOBm6k_c5QsXAP", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T22:38:37Z", "updated_at": "2022-12-15T22:38:37Z", "author_association": "OWNER", "body": "I'm going to make `.status_code` work on `TestClient` response too, so I don't have to worry about using both `status` or `status_code` depending on which kind of object I am using.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1499150951, "label": "Port as many tests as possible to async def tests against ds_client"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/pull/1960#issuecomment-1353765125", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1960", "id": 1353765125, "node_id": "IC_kwDOBm6k_c5QsNEF", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T22:00:04Z", "updated_at": "2022-12-15T22:00:04Z", "author_association": "OWNER", "body": "I'm going to punt on that for the moment and continue to use `app_client` for tests that use that mechanism.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1499150951, "label": "Port as many tests as possible to async def tests against ds_client"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/pull/1960#issuecomment-1353763837", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1960", "id": 1353763837, "node_id": "IC_kwDOBm6k_c5QsMv9", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T21:59:05Z", "updated_at": "2022-12-15T21:59:05Z", "author_association": "OWNER", "body": "Here's an annoying error:\r\n```\r\n> response4 = await ds_client.post(\r\n \"/-/logout\",\r\n csrftoken_from=True,\r\n cookies={\"ds_actor\": ds_client.actor_cookie({\"id\": \"test\"})},\r\n )\r\n\r\ntests/test_auth.py:88: \r\n_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _\r\n\r\nself = , path = '/-/logout'\r\nkwargs = {'cookies': {'ds_actor': 'eyJhIjp7ImlkIjoidGVzdCJ9fQ.fuFCTJG5XE-RNnUM7dcnXx9sPvE'}, 'csrftoken_from': True}, client = \r\n\r\n async def post(self, path, **kwargs):\r\n await self.ds.invoke_startup()\r\n async with httpx.AsyncClient(app=self.app) as client:\r\n> return await client.post(self._fix(path), **kwargs)\r\nE TypeError: AsyncClient.post() got an unexpected keyword argument 'csrftoken_from'\r\n```\r\nI need an alternative to the `csrftoken_from` mechanism I built for `TestClient`:\r\n\r\nhttps://github.com/simonw/datasette/blob/0b68996cc511b3a801f0cd0157bd66332d75f46f/datasette/utils/testing.py#L77-L103", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1499150951, "label": "Port as many tests as possible to async def tests against ds_client"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/pull/1960#issuecomment-1353749401", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1960", "id": 1353749401, "node_id": "IC_kwDOBm6k_c5QsJOZ", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T21:47:27Z", "updated_at": "2022-12-15T21:47:27Z", "author_association": "OWNER", "body": "I'm using this new mark:\r\n```python\r\n@pytest.mark.ds_client\r\n```\r\nPurely so I can run all of the tests that I've refactored using:\r\n\r\n```\r\npytest -m ds_client\r\n```\r\nI'll likely remove this once the test refactoring project is complete.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1499150951, "label": "Port as many tests as possible to async def tests against ds_client"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1959#issuecomment-1353747370", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1959", "id": 1353747370, "node_id": "IC_kwDOBm6k_c5QsIuq", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T21:45:14Z", "updated_at": "2022-12-15T21:45:14Z", "author_association": "OWNER", "body": "I'm going to do this in a PR.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1499081664, "label": "Refactor test suite to use mostly `async def` tests"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1959#issuecomment-1353738075", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1959", "id": 1353738075, "node_id": "IC_kwDOBm6k_c5QsGdb", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T21:35:56Z", "updated_at": "2022-12-15T21:35:56Z", "author_association": "OWNER", "body": "I built that `OldResponse` class:\r\n```diff\r\ndiff --git a/tests/utils.py b/tests/utils.py\r\nindex 191ead9b..f39ac434 100644\r\n--- a/tests/utils.py\r\n+++ b/tests/utils.py\r\n@@ -30,3 +30,25 @@ def inner_html(soup):\r\n def has_load_extension():\r\n conn = sqlite3.connect(\":memory:\")\r\n return hasattr(conn, \"enable_load_extension\")\r\n+\r\n+\r\n+class OldResponse:\r\n+ \"Transform an HTTPX response to simulate the older TestClient responses\"\r\n+ # https://github.com/simonw/datasette/issues/1959#issuecomment-1353721091\r\n+ def __init__(self, response):\r\n+ self.response = response\r\n+ self._json = None\r\n+\r\n+ @property\r\n+ def headers(self):\r\n+ return self.response.headers\r\n+\r\n+ @property\r\n+ def status(self):\r\n+ return self.response.status_code\r\n+\r\n+ @property\r\n+ def json(self):\r\n+ if self._json is None:\r\n+ self._json = self.response.json()\r\n+ return self._json\r\n```\r\nI can use it in tests like this:\r\n```python\r\n@pytest.mark.asyncio\r\nasync def test_homepage(ds_client):\r\n response = OldResponse(await ds_client.get(\"/.json\"))\r\n assert response.status == 200\r\n assert \"application/json; charset=utf-8\" == response.headers[\"content-type\"]\r\n assert response.json.keys() == {\"fixtures\": 0}.keys()\r\n d = response.json[\"fixtures\"]\r\n assert d[\"name\"] == \"fixtures\"\r\n assert d[\"tables_count\"] == 24\r\n assert len(d[\"tables_and_views_truncated\"]) == 5\r\n assert d[\"tables_and_views_more\"] is True\r\n # 4 hidden FTS tables + no_primary_key (hidden in metadata)\r\n assert d[\"hidden_tables_count\"] == 6\r\n # 201 in no_primary_key, plus 6 in other hidden tables:\r\n assert d[\"hidden_table_rows_sum\"] == 207, response.json\r\n assert d[\"views_count\"] == 4\r\n```\r\nBut as I work through the tests I'm finding it's actually not too hard to port them over, so I likely won't use it after all.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1499081664, "label": "Refactor test suite to use mostly `async def` tests"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1959#issuecomment-1353728682", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1959", "id": 1353728682, "node_id": "IC_kwDOBm6k_c5QsEKq", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T21:28:35Z", "updated_at": "2022-12-15T21:28:35Z", "author_association": "OWNER", "body": "Got this error trying to have two tests use the same `ds_client` async fixture when I added `scope=\"session\"` to that fixture:\r\n\r\n- https://github.com/tortoise/tortoise-orm/issues/638\r\n\r\nAdding this to `conftest.py` (as suggested in that issue thread) seemed to fix it:\r\n\r\n```python\r\n@pytest.fixture(scope=\"session\")\r\ndef event_loop():\r\n return asyncio.get_event_loop()\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1499081664, "label": "Refactor test suite to use mostly `async def` tests"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1619#issuecomment-1353721442", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1619", "id": 1353721442, "node_id": "IC_kwDOBm6k_c5QsCZi", "user": {"value": 2090382, "label": "noslouch"}, "created_at": "2022-12-15T21:20:53Z", "updated_at": "2022-12-15T21:20:53Z", "author_association": "NONE", "body": "i'm also getting bit by this. I'm trying to set up an nginx reverse proxy in front of multiple datasette backends.\r\n\r\nWhen I run it locally or behind the proxy, I see the `base_url` value added a second time to the path for various action links on table pages (view as JSON, sort by column, etc).", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1121583414, "label": "JSON link on row page is 404 if base_url setting is used"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1959#issuecomment-1353721091", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1959", "id": 1353721091, "node_id": "IC_kwDOBm6k_c5QsCUD", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T21:20:32Z", "updated_at": "2022-12-15T21:20:32Z", "author_association": "OWNER", "body": "Rather than tediously rewriting every single test to the new shape I'm going to try a wrapper for that HTTPX response that transforms it into an imitation of the one returned by the existing `TestClient` class.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1499081664, "label": "Refactor test suite to use mostly `async def` tests"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1959#issuecomment-1353720559", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1959", "id": 1353720559, "node_id": "IC_kwDOBm6k_c5QsCLv", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T21:19:56Z", "updated_at": "2022-12-15T21:19:56Z", "author_association": "OWNER", "body": "Here's a port of the first `def ...(app_client)` test. Note that the TestClient object works slightly differently from the HTTPX response returned by `await datasette.client.get(...)`:\r\n\r\n```diff\r\ndiff --git a/datasette/app.py b/datasette/app.py\r\nindex f3cb8876..b770b469 100644\r\n--- a/datasette/app.py\r\n+++ b/datasette/app.py\r\n@@ -281,7 +281,7 @@ class Datasette:\r\n raise\r\n self.crossdb = crossdb\r\n self.nolock = nolock\r\n- if memory or crossdb or not self.files:\r\n+ if memory or crossdb or (not self.files and memory is not False):\r\n self.add_database(\r\n Database(self, is_mutable=False, is_memory=True), name=\"_memory\"\r\n )\r\ndiff --git a/pytest.ini b/pytest.ini\r\nindex 559e518c..0bcb0d1e 100644\r\n--- a/pytest.ini\r\n+++ b/pytest.ini\r\n@@ -8,4 +8,5 @@ filterwarnings=\r\n ignore:.*current_task.*:PendingDeprecationWarning\r\n markers =\r\n serial: tests to avoid using with pytest-xdist\r\n+ ds_client: tests using the ds_client fixture\r\n asyncio_mode = strict\r\ndiff --git a/tests/conftest.py b/tests/conftest.py\r\nindex cd735e12..648423ba 100644\r\n--- a/tests/conftest.py\r\n+++ b/tests/conftest.py\r\n@@ -2,6 +2,7 @@ import httpx\r\n import os\r\n import pathlib\r\n import pytest\r\n+import pytest_asyncio\r\n import re\r\n import subprocess\r\n import tempfile\r\n@@ -23,6 +24,22 @@ UNDOCUMENTED_PERMISSIONS = {\r\n }\r\n \r\n \r\n+@pytest_asyncio.fixture\r\n+async def ds_client():\r\n+ from datasette.app import Datasette\r\n+ from .fixtures import METADATA, PLUGINS_DIR\r\n+ ds = Datasette(memory=False, metadata=METADATA, plugins_dir=PLUGINS_DIR)\r\n+ from .fixtures import TABLES, TABLE_PARAMETERIZED_SQL\r\n+ db = ds.add_memory_database(\"fixtures\")\r\n+ def prepare(conn):\r\n+ conn.executescript(TABLES)\r\n+ for sql, params in TABLE_PARAMETERIZED_SQL:\r\n+ with conn:\r\n+ conn.execute(sql, params)\r\n+ await db.execute_write_fn(prepare)\r\n+ return ds.client\r\n+\r\n+\r\n def pytest_report_header(config):\r\n return \"SQLite: {}\".format(\r\n sqlite3.connect(\":memory:\").execute(\"select sqlite_version()\").fetchone()[0]\r\ndiff --git a/tests/test_api.py b/tests/test_api.py\r\nindex 5f2a6ea6..ddf4219c 100644\r\n--- a/tests/test_api.py\r\n+++ b/tests/test_api.py\r\n@@ -23,12 +23,15 @@ import sys\r\n import urllib\r\n \r\n \r\n-def test_homepage(app_client):\r\n- response = app_client.get(\"/.json\")\r\n- assert response.status == 200\r\n+@pytest.mark.ds_client\r\n+@pytest.mark.asyncio\r\n+async def test_homepage(ds_client):\r\n+ response = await ds_client.get(\"/.json\")\r\n+ assert response.status_code == 200\r\n assert \"application/json; charset=utf-8\" == response.headers[\"content-type\"]\r\n- assert response.json.keys() == {\"fixtures\": 0}.keys()\r\n- d = response.json[\"fixtures\"]\r\n+ data = response.json()\r\n+ assert data.keys() == {\"fixtures\": 0}.keys()\r\n+ d = data[\"fixtures\"]\r\n assert d[\"name\"] == \"fixtures\"\r\n assert d[\"tables_count\"] == 24\r\n assert len(d[\"tables_and_views_truncated\"]) == 5\r\n@@ -36,7 +39,7 @@ def test_homepage(app_client):\r\n # 4 hidden FTS tables + no_primary_key (hidden in metadata)\r\n assert d[\"hidden_tables_count\"] == 6\r\n # 201 in no_primary_key, plus 6 in other hidden tables:\r\n- assert d[\"hidden_table_rows_sum\"] == 207, response.json\r\n+ assert d[\"hidden_table_rows_sum\"] == 207, data\r\n assert d[\"views_count\"] == 4\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1499081664, "label": "Refactor test suite to use mostly `async def` tests"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1959#issuecomment-1353707828", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1959", "id": 1353707828, "node_id": "IC_kwDOBm6k_c5Qr_E0", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T21:06:29Z", "updated_at": "2022-12-15T21:06:29Z", "author_association": "OWNER", "body": "Previous, abandoned attempt at this work (for #1843):\r\n```diff\r\ndiff --git a/datasette/app.py b/datasette/app.py\r\nindex 7e682498..cf35c3a2 100644\r\n--- a/datasette/app.py\r\n+++ b/datasette/app.py\r\n@@ -228,7 +228,7 @@ class Datasette:\r\n template_dir=None,\r\n plugins_dir=None,\r\n static_mounts=None,\r\n- memory=False,\r\n+ memory=None,\r\n settings=None,\r\n secret=None,\r\n version_note=None,\r\n@@ -238,6 +238,7 @@ class Datasette:\r\n nolock=False,\r\n ):\r\n self._startup_invoked = False\r\n+ self._extra_on_startup = []\r\n assert config_dir is None or isinstance(\r\n config_dir, Path\r\n ), \"config_dir= should be a pathlib.Path\"\r\n@@ -278,7 +279,7 @@ class Datasette:\r\n raise\r\n self.crossdb = crossdb\r\n self.nolock = nolock\r\n- if memory or crossdb or not self.files:\r\n+ if memory or crossdb or (not self.files and memory is not False):\r\n self.add_database(\r\n Database(self, is_mutable=False, is_memory=True), name=\"_memory\"\r\n )\r\n@@ -391,6 +392,9 @@ class Datasette:\r\n self._root_token = secrets.token_hex(32)\r\n self.client = DatasetteClient(self)\r\n \r\n+ def _add_on_startup(self, fn):\r\n+ self._extra_on_startup.append(fn)\r\n+\r\n async def refresh_schemas(self):\r\n if self._refresh_schemas_lock.locked():\r\n return\r\n@@ -431,6 +435,8 @@ class Datasette:\r\n # This must be called for Datasette to be in a usable state\r\n if self._startup_invoked:\r\n return\r\n+ for fn in self._extra_on_startup:\r\n+ await fn()\r\n # Register permissions, but watch out for duplicate name/abbr\r\n names = {}\r\n abbrs = {}\r\n@@ -1431,9 +1437,9 @@ class Datasette:\r\n )\r\n if self.setting(\"trace_debug\"):\r\n asgi = AsgiTracer(asgi)\r\n- asgi = AsgiRunOnFirstRequest(asgi, on_startup=[setup_db, self.invoke_startup])\r\n for wrapper in pm.hook.asgi_wrapper(datasette=self):\r\n asgi = wrapper(asgi)\r\n+ asgi = AsgiRunOnFirstRequest(asgi, on_startup=[setup_db, self.invoke_startup])\r\n return asgi\r\n \r\n \r\ndiff --git a/datasette/utils/asgi.py b/datasette/utils/asgi.py\r\nindex 56690251..986755cb 100644\r\n--- a/datasette/utils/asgi.py\r\n+++ b/datasette/utils/asgi.py\r\n@@ -423,9 +423,9 @@ class AsgiFileDownload:\r\n \r\n \r\n class AsgiRunOnFirstRequest:\r\n- def __init__(self, asgi, on_startup):\r\n+ def __init__(self, app, on_startup):\r\n assert isinstance(on_startup, list)\r\n- self.asgi = asgi\r\n+ self.app = app\r\n self.on_startup = on_startup\r\n self._started = False\r\n \r\n@@ -434,4 +434,4 @@ class AsgiRunOnFirstRequest:\r\n self._started = True\r\n for hook in self.on_startup:\r\n await hook()\r\n- return await self.asgi(scope, receive, send)\r\n+ return await self.app(scope, receive, send)\r\ndiff --git a/tests/conftest.py b/tests/conftest.py\r\nindex cd735e12..d1301943 100644\r\n--- a/tests/conftest.py\r\n+++ b/tests/conftest.py\r\n@@ -23,6 +23,15 @@ UNDOCUMENTED_PERMISSIONS = {\r\n }\r\n \r\n \r\n+# @pytest.fixture(autouse=True)\r\n+# def log_name_of_test_before_test(request):\r\n+# # To help identify tests that are hanging\r\n+# name = str(request.node)\r\n+# with open(\"/tmp/test.log\", \"a\") as f:\r\n+# f.write(name + \"\\n\")\r\n+# yield\r\n+\r\n+\r\n def pytest_report_header(config):\r\n return \"SQLite: {}\".format(\r\n sqlite3.connect(\":memory:\").execute(\"select sqlite_version()\").fetchone()[0]\r\ndiff --git a/tests/fixtures.py b/tests/fixtures.py\r\nindex a6700239..18d3f1b7 100644\r\n--- a/tests/fixtures.py\r\n+++ b/tests/fixtures.py\r\n@@ -101,6 +101,19 @@ EXPECTED_PLUGINS = [\r\n ]\r\n \r\n \r\n+def _populate_connection(conn):\r\n+ # Drop any tables and views that exist\r\n+ to_drop = conn.execute(\r\n+ \"SELECT name, type FROM sqlite_master where type in ('table', 'view')\"\r\n+ ).fetchall()\r\n+ for name, type in to_drop:\r\n+ conn.execute(f\"DROP {type} IF EXISTS [{name}]\")\r\n+ conn.executescript(TABLES)\r\n+ for sql, params in TABLE_PARAMETERIZED_SQL:\r\n+ with conn:\r\n+ conn.execute(sql, params)\r\n+\r\n+\r\n @contextlib.contextmanager\r\n def make_app_client(\r\n sql_time_limit_ms=None,\r\n@@ -117,45 +130,22 @@ def make_app_client(\r\n metadata=None,\r\n crossdb=False,\r\n ):\r\n- with tempfile.TemporaryDirectory() as tmpdir:\r\n- filepath = os.path.join(tmpdir, filename)\r\n- if is_immutable:\r\n- files = []\r\n- immutables = [filepath]\r\n- else:\r\n- files = [filepath]\r\n- immutables = []\r\n- conn = sqlite3.connect(filepath)\r\n- conn.executescript(TABLES)\r\n- for sql, params in TABLE_PARAMETERIZED_SQL:\r\n- with conn:\r\n- conn.execute(sql, params)\r\n- # Close the connection to avoid \"too many open files\" errors\r\n- conn.close()\r\n- if extra_databases is not None:\r\n- for extra_filename, extra_sql in extra_databases.items():\r\n- extra_filepath = os.path.join(tmpdir, extra_filename)\r\n- c2 = sqlite3.connect(extra_filepath)\r\n- c2.executescript(extra_sql)\r\n- c2.close()\r\n- # Insert at start to help test /-/databases ordering:\r\n- files.insert(0, extra_filepath)\r\n- os.chdir(os.path.dirname(filepath))\r\n- settings = settings or {}\r\n- for key, value in {\r\n- \"default_page_size\": 50,\r\n- \"max_returned_rows\": max_returned_rows or 100,\r\n- \"sql_time_limit_ms\": sql_time_limit_ms or 200,\r\n- # Default is 3 but this results in \"too many open files\"\r\n- # errors when running the full test suite:\r\n- \"num_sql_threads\": 1,\r\n- }.items():\r\n- if key not in settings:\r\n- settings[key] = value\r\n+ settings = settings or {}\r\n+ for key, value in {\r\n+ \"default_page_size\": 50,\r\n+ \"max_returned_rows\": max_returned_rows or 100,\r\n+ \"sql_time_limit_ms\": sql_time_limit_ms or 200,\r\n+ # Default is 3 but this results in \"too many open files\"\r\n+ # errors when running the full test suite:\r\n+ \"num_sql_threads\": 1,\r\n+ }.items():\r\n+ if key not in settings:\r\n+ settings[key] = value\r\n+ # We can use an in-memory database, but only if we're not doing anything\r\n+ # with is_immutable or extra_databases and filename is the default\r\n+ if not is_immutable and not extra_databases and filename == \"fixtures.db\":\r\n ds = Datasette(\r\n- files,\r\n- immutables=immutables,\r\n- memory=memory,\r\n+ memory=memory or False,\r\n cors=cors,\r\n metadata=metadata or METADATA,\r\n plugins_dir=PLUGINS_DIR,\r\n@@ -165,12 +155,57 @@ def make_app_client(\r\n template_dir=template_dir,\r\n crossdb=crossdb,\r\n )\r\n+ db = ds.add_memory_database(\"fixtures\")\r\n+\r\n+ async def populate_fixtures():\r\n+ print(\"Here we go... populating fixtures\")\r\n+ await db.execute_write_fn(_populate_connection)\r\n+\r\n+ ds._add_on_startup(populate_fixtures)\r\n yield TestClient(ds)\r\n- # Close as many database connections as possible\r\n- # to try and avoid too many open files error\r\n- for db in ds.databases.values():\r\n- if not db.is_memory:\r\n- db.close()\r\n+ else:\r\n+ with tempfile.TemporaryDirectory() as tmpdir:\r\n+ filepath = os.path.join(tmpdir, filename)\r\n+ if is_immutable:\r\n+ files = []\r\n+ immutables = [filepath]\r\n+ else:\r\n+ files = [filepath]\r\n+ immutables = []\r\n+\r\n+ conn = sqlite3.connect(filepath)\r\n+ _populate_connection(conn)\r\n+ # Close the connection to reduce \"too many open files\" errors\r\n+ conn.close()\r\n+\r\n+ if extra_databases is not None:\r\n+ for extra_filename, extra_sql in extra_databases.items():\r\n+ extra_filepath = os.path.join(tmpdir, extra_filename)\r\n+ c2 = sqlite3.connect(extra_filepath)\r\n+ c2.executescript(extra_sql)\r\n+ c2.close()\r\n+ # Insert at start to help test /-/databases ordering:\r\n+ files.insert(0, extra_filepath)\r\n+ os.chdir(os.path.dirname(filepath))\r\n+ ds = Datasette(\r\n+ files,\r\n+ immutables=immutables,\r\n+ memory=memory,\r\n+ cors=cors,\r\n+ metadata=metadata or METADATA,\r\n+ plugins_dir=PLUGINS_DIR,\r\n+ settings=settings,\r\n+ inspect_data=inspect_data,\r\n+ static_mounts=static_mounts,\r\n+ template_dir=template_dir,\r\n+ crossdb=crossdb,\r\n+ )\r\n+ yield TestClient(ds)\r\n+ # Close as many database connections as possible\r\n+ # to try and avoid too many open files error\r\n+ for db in ds.databases.values():\r\n+ if not db.is_memory:\r\n+ db.close()\r\n \r\n \r\n @pytest.fixture(scope=\"session\")\r\ndiff --git a/tests/test_cli.py b/tests/test_cli.py\r\nindex d3e015fa..d9e4e457 100644\r\n--- a/tests/test_cli.py\r\n+++ b/tests/test_cli.py\r\n@@ -1,5 +1,6 @@\r\n from .fixtures import (\r\n app_client,\r\n+ app_client_with_cors,\r\n make_app_client,\r\n TestClient as _TestClient,\r\n EXPECTED_PLUGINS,\r\n@@ -38,7 +39,7 @@ def test_inspect_cli(app_client):\r\n assert expected_count == database[\"tables\"][table_name][\"count\"]\r\n \r\n \r\n-def test_inspect_cli_writes_to_file(app_client):\r\n+def test_inspect_cli_writes_to_file(app_client_with_cors):\r\n runner = CliRunner()\r\n result = runner.invoke(\r\n cli, [\"inspect\", \"fixtures.db\", \"--inspect-file\", \"foo.json\"]\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1499081664, "label": "Refactor test suite to use mostly `async def` tests"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1959#issuecomment-1353705072", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1959", "id": 1353705072, "node_id": "IC_kwDOBm6k_c5Qr-Zw", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T21:04:07Z", "updated_at": "2022-12-15T21:04:07Z", "author_association": "OWNER", "body": "I'm going to start by getting every test that uses the raw `(app_client)` fixture and nothing else (194 at the moment) to switch to `async def` using a shared Datasette instance and `datasette.client.get()`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1499081664, "label": "Refactor test suite to use mostly `async def` tests"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1353701674", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1353701674, "node_id": "IC_kwDOBm6k_c5Qr9kq", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T21:00:51Z", "updated_at": "2022-12-15T21:00:51Z", "author_association": "OWNER", "body": "OK, I've broken the test suite here.\r\n\r\nI'm going to revert these two commits:\r\n\r\n- https://github.com/simonw/datasette/commit/dc18f62089e5672d03176f217d7840cdafa5c447\r\n- https://github.com/simonw/datasette/commit/51ee8caa4a697fa3f4120e93b1c205b714a6cdc7\r\n\r\nThen I'll do a bunch of work making the test suite more robust before I try this again.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1353694582", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1353694582, "node_id": "IC_kwDOBm6k_c5Qr712", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T20:52:46Z", "updated_at": "2022-12-15T20:52:46Z", "author_association": "OWNER", "body": "Just noticed this: https://github.com/simonw/datasette/actions/runs/3706504228/jobs/6281796135\r\n\r\n![\"image\"](\"https://user-images.githubusercontent.com/9599/207964564-46c31685-db7a-494f-9aab-a611d25e5511.png\")
\r\n\r\nThis suggests that the regular tests passed in CI fine, but the non-serial ones failed.\r\n\r\nI'm going to try running everything using `pytest -n auto` without splitting serial and non-serial tests. Maybe the serial thing isn't needed any more?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1843#issuecomment-1353690591", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1843", "id": 1353690591, "node_id": "IC_kwDOBm6k_c5Qr63f", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T20:49:05Z", "updated_at": "2022-12-15T20:49:05Z", "author_association": "OWNER", "body": "I have a nasty feeling the cleaner solution for this would involve porting my entire test suite from `def test_blah(app_client)` sync functions (which work due to a `@async_to_sync` call in `TestClient`) to `async def test_blah(async_fixture):` functions instead.\r\n\r\nI've been using that latter pattern for new tests (and plugin tests) for quite a while now, but I never took on the job of refactoring all of the old ones.\r\n\r\nA search for `(app_client):` across the whole project currently returns 194 results which might be a reasonable target to try switching to the new pattern as a starting point.\r\n\r\nNo idea if it will have much impact on the \"Too many open files\" errors though.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1408757705, "label": "Intermittent \"Too many open files\" error running tests"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1353683238", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1353683238, "node_id": "IC_kwDOBm6k_c5Qr5Em", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T20:42:18Z", "updated_at": "2022-12-15T20:42:18Z", "author_association": "OWNER", "body": "Possibly related issue:\n- https://github.com/pytest-dev/pytest-xdist/issues/60", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1353680261", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1353680261, "node_id": "IC_kwDOBm6k_c5Qr4WF", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T20:39:19Z", "updated_at": "2022-12-15T20:39:19Z", "author_association": "OWNER", "body": "When I hit `Ctr+C` here's the traceback I get:\n```\n^C^CException ignored in: \nTraceback (most recent call last):\n File \"/Users/simon/.pyenv/versions/3.10.3/lib/python3.10/threading.py\", line 1530, in _shutdown\n!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! KeyboardInterrupt !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!\n/Users/simon/.pyenv/versions/3.10.3/lib/python3.10/threading.py:324: KeyboardInterrupt\n(to show a full traceback on KeyboardInterrupt use --full-trace)\nTraceback (most recent call last):\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/bin/pytest\", line 8, in \n atexit_call()\n File \"/Users/simon/.pyenv/versions/3.10.3/lib/python3.10/concurrent/futures/thread.py\", line 31, in _python_exit\n sys.exit(console_main())\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/_pytest/config/__init__.py\", line 187, in console_main\n t.join()\n File \"/Users/simon/.pyenv/versions/3.10.3/lib/python3.10/threading.py\", line 1089, in join\n self._wait_for_tstate_lock()\n File \"/Users/simon/.pyenv/versions/3.10.3/lib/python3.10/threading.py\", line 1109, in _wait_for_tstate_lock\n if lock.acquire(block, timeout):\nKeyboardInterrupt: \n code = main()\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/_pytest/config/__init__.py\", line 164, in main\n ret: Union[ExitCode, int] = config.hook.pytest_cmdline_main(\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/pluggy/_hooks.py\", line 265, in __call__\n return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/pluggy/_manager.py\", line 80, in _hookexec\n return self._inner_hookexec(hook_name, methods, kwargs, firstresult)\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/pluggy/_callers.py\", line 60, in _multicall\n return outcome.get_result()\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/pluggy/_result.py\", line 60, in get_result\n raise ex[1].with_traceback(ex[2])\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/pluggy/_callers.py\", line 39, in _multicall\n res = hook_impl.function(*args)\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/_pytest/main.py\", line 315, in pytest_cmdline_main\n return wrap_session(config, _main)\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/_pytest/main.py\", line 303, in wrap_session\n config.hook.pytest_sessionfinish(\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/pluggy/_hooks.py\", line 265, in __call__\n return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/pluggy/_manager.py\", line 80, in _hookexec\n return self._inner_hookexec(hook_name, methods, kwargs, firstresult)\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/pluggy/_callers.py\", line 55, in _multicall\n gen.send(outcome)\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/_pytest/terminal.py\", line 798, in pytest_sessionfinish\n outcome.get_result()\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/pluggy/_result.py\", line 60, in get_result\n raise ex[1].with_traceback(ex[2])\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/pluggy/_callers.py\", line 39, in _multicall\n res = hook_impl.function(*args)\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/xdist/dsession.py\", line 88, in pytest_sessionfinish\n nm.teardown_nodes()\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/xdist/workermanage.py\", line 79, in teardown_nodes\n self.group.terminate(self.EXIT_TIMEOUT)\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/execnet/multi.py\", line 215, in terminate\n safe_terminate(\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/execnet/multi.py\", line 311, in safe_terminate\n reply.get()\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/execnet/gateway_base.py\", line 206, in get\n self.waitfinish(timeout)\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/execnet/gateway_base.py\", line 213, in waitfinish\n if not self._result_ready.wait(timeout):\n File \"/Users/simon/.pyenv/versions/3.10.3/lib/python3.10/threading.py\", line 600, in wait\n signaled = self._cond.wait(timeout)\n File \"/Users/simon/.pyenv/versions/3.10.3/lib/python3.10/threading.py\", line 320, in wait\n waiter.acquire()\nKeyboardInterrupt\n```\nIt looks to me like this relates to `pytest-xdist` istelf - it's waiting on some locks but `site-packages/xdist/workermanage.py` shows up in that track.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1843#issuecomment-1353522652", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1843", "id": 1353522652, "node_id": "IC_kwDOBm6k_c5QrR3c", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T18:21:27Z", "updated_at": "2022-12-15T18:21:27Z", "author_association": "OWNER", "body": "I'll still use on-disk test databases for `is_immutable=True`, but not for the majority of tests.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1408757705, "label": "Intermittent \"Too many open files\" error running tests"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1843#issuecomment-1353522211", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1843", "id": 1353522211, "node_id": "IC_kwDOBm6k_c5QrRwj", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T18:21:02Z", "updated_at": "2022-12-15T18:21:02Z", "author_association": "OWNER", "body": "When I initially built this test suite Datasette didn't have the `memory_name=` mechanism for creating persistent in-memory databases.\r\n\r\nI'm going to see if I can switch to that for the majority of Datasette's tests. Might find that doing so both fixes this \"too many open files\" issue AND gives me a significant speed improvement to the test site too!\r\n\r\nHopefully I can do most of the work on that in this big ugly function:\r\n\r\nhttps://github.com/simonw/datasette/blob/dc18f62089e5672d03176f217d7840cdafa5c447/tests/fixtures.py#L104-L173", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1408757705, "label": "Intermittent \"Too many open files\" error running tests"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1843#issuecomment-1353520615", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1843", "id": 1353520615, "node_id": "IC_kwDOBm6k_c5QrRXn", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T18:19:25Z", "updated_at": "2022-12-15T18:19:25Z", "author_association": "OWNER", "body": "I've been seeing this error again:\r\n\r\n```\r\nERROR tests/test_api_write.py::test_create_table[input16-400-expected_response16] - OSError: [Errno 24] Too ...\r\nERROR tests/test_api_write.py::test_create_table[input17-400-expected_response17] - OSError: [Errno 24] Too ...\r\nERROR tests/test_api_write.py::test_create_table[input18-400-expected_response18] - OSError: [Errno 24] Too ...\r\n```\r\n\r\nIt doesn't happen in CI, and it turns out that's because CI runs `pytest -n auto` which splits the tests across multiple parallel processes.\r\n\r\nI've been working around the error on my laptop using `pytest -n auto` there too, but I'd rather not have to do that.\r\n\r\nThis is also getting in my way when I try to debug other issues, like this one:\r\n\r\n- #1955", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1408757705, "label": "Intermittent \"Too many open files\" error running tests"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1353516572", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1353516572, "node_id": "IC_kwDOBm6k_c5QrQYc", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T18:15:28Z", "updated_at": "2022-12-15T18:15:28Z", "author_association": "OWNER", "body": "I added `return` to the first line of that test to disable it, then ran again - and now it's hanging at about the same progress point through the tests but in a different test:\n\n\n\nSo this time it was hanging at `test_urlsafe_components()`.\n\nSo it's clearly not the individual tests themselves that are the problem - something about running the entire test suite in one go is incompatible with this change for some reason.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1353512099", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1353512099, "node_id": "IC_kwDOBm6k_c5QrPSj", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T18:11:27Z", "updated_at": "2022-12-15T18:11:27Z", "author_association": "OWNER", "body": "This is surprising!\n\n\n\nThe logs suggest that the test suite hung running this test here:\n\nhttps://github.com/simonw/datasette/blob/dc18f62089e5672d03176f217d7840cdafa5c447/tests/test_utils.py#L55-L58\n\nI find that very hard to believe.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1353509776", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1353509776, "node_id": "IC_kwDOBm6k_c5QrOuQ", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T18:09:26Z", "updated_at": "2022-12-15T18:09:26Z", "author_association": "OWNER", "body": "I added this to `conftest.py`:\n\n```python\n@pytest.fixture(autouse=True)\ndef log_name_of_test_before_test(request):\n # To help identify tests that are hanging\n name = str(request.node)\n with open(\"/tmp/test.log\", \"a\") as f:\n f.write(name + \"\\n\")\n yield\n```\nThis logs out the name of each test to `/tmp/test.log` before running the test - so I can wait until it hangs and see which test it was that caused that.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1353473571", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1353473571, "node_id": "IC_kwDOBm6k_c5QrF4j", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T17:43:28Z", "updated_at": "2022-12-15T17:43:48Z", "author_association": "OWNER", "body": "Running:\r\n\r\n pytest -n auto -x -v\r\n\r\nOn may laptop to see if I can replicate.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1353473086", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1353473086, "node_id": "IC_kwDOBm6k_c5QrFw-", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T17:43:08Z", "updated_at": "2022-12-15T17:43:08Z", "author_association": "OWNER", "body": "It looks like that fix _almost_ works... except it seems to push the tests into an infinite loop or similar? They're not finishing their runs from what I can see.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1353448095", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1353448095, "node_id": "IC_kwDOBm6k_c5Qq_qf", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T17:25:05Z", "updated_at": "2022-12-15T17:25:05Z", "author_association": "OWNER", "body": "So actually that `setup_db()` function I wrote back in 2019 has not been executing for most of Datasette's tests. Which seems bad.\r\n\r\nI'm inclined to ditch `AsgiLifespan` entirely in favour of the mechanism I described above, where `invoke_startup()` is called for every request on the first request processed by the server.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1353443718", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1353443718, "node_id": "IC_kwDOBm6k_c5Qq-mG", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T17:23:12Z", "updated_at": "2022-12-15T17:23:55Z", "author_association": "OWNER", "body": "That may not be the best fix here. It turns out this pattern:\r\n```python\r\n async def get(self, path, **kwargs):\r\n async with httpx.AsyncClient(app=self.app) as client:\r\n return await client.get(self._fix(path), **kwargs)\r\n```\r\nDoesn't trigger that `AsgiLifespan` class.\r\n\r\nI wrote about that previously in this TIL: https://til.simonwillison.net/asgi/lifespan-test-httpx", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1353423584", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1353423584, "node_id": "IC_kwDOBm6k_c5Qq5rg", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T17:13:18Z", "updated_at": "2022-12-15T17:22:59Z", "author_association": "OWNER", "body": "Wow, just spotted this in the code - it turns out I solved this problem a different (and better) way long before i introduced `invoke_startup()`!\r\n\r\nhttps://github.com/simonw/datasette/blob/e054704fb64d1f23154ec43b81b6c9481ff8202f/datasette/app.py#L1416-L1440", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1352674924", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1352674924, "node_id": "IC_kwDOBm6k_c5QoC5s", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T07:46:36Z", "updated_at": "2022-12-15T07:46:36Z", "author_association": "OWNER", "body": "It's possible the fix for this might be for the first incoming HTTP request to trigger `invoke_startup()` if it hasn't been called yet - similar to the hack I put in place for `datasette.client.get()` in tests:\r\n\r\nhttps://github.com/simonw/datasette/blob/e054704fb64d1f23154ec43b81b6c9481ff8202f/datasette/app.py#L1728-L1731\r\n\r\nThis would be a much more elegant fix, I could remove those multiple `invoke_startup()` calls entirely - and remove this tip from the documentation too: https://docs.datasette.io/en/0.63.2/testing_plugins.html#setting-up-a-datasette-test-instance", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1958#issuecomment-1352644281", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1958", "id": 1352644281, "node_id": "IC_kwDOBm6k_c5Qn7a5", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T07:08:14Z", "updated_at": "2022-12-15T07:08:14Z", "author_association": "OWNER", "body": "Thanks for the details write-up! This looks like a bug in Datasette itself when run with Docker. Moving this issue there.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1497909798, "label": "datasette --root running in Docker doesn't reliably show the magic URL"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1352643333", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1352643333, "node_id": "IC_kwDOBm6k_c5Qn7MF", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T07:07:29Z", "updated_at": "2022-12-15T07:07:29Z", "author_association": "OWNER", "body": "Datasette 0.63 is the release that broke this, thanks to this issue:\r\n\r\n- https://github.com/simonw/datasette/issues/1809", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1955#issuecomment-1352643049", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1955", "id": 1352643049, "node_id": "IC_kwDOBm6k_c5Qn7Hp", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T07:07:10Z", "updated_at": "2022-12-15T07:07:10Z", "author_association": "OWNER", "body": "This is definitely a regression: Datasette is meant to work in those environments, and I didn't think to test them when I added the `invoke_startup()` hook.\r\n\r\nCoincidentally I actually built a plugin for running Datasette with Gunicorn just a couple of months ago:\r\n\r\nhttps://datasette.io/plugins/datasette-gunicorn\r\n\r\nAnd I just tested and it has the same bug you describe here! Filed:\r\n\r\n- https://github.com/simonw/datasette-gunicorn/issues/5\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1496652622, "label": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1953#issuecomment-1352459146", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1953", "id": 1352459146, "node_id": "IC_kwDOBm6k_c5QnOOK", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T02:02:15Z", "updated_at": "2022-12-15T02:02:15Z", "author_association": "OWNER", "body": "```\r\nThe third Datasette 1.0 alpha release adds upsert support to the JSON API, plus the ability to specify finely grained permissions when creating an API token.\r\n\r\n- New `/db/table/-/upsert` API, [documented here](https://docs.datasette.io/en/latest/json_api.html#tableupsertview). upsert is an update-or-replace: existing rows will have specified keys updated, but if no row matches the incoming primary key a brand new row will be inserted instead. ([#1878](https://github.com/simonw/datasette/issues/1878))\r\n- New [register_permissions(datasette)](https://docs.datasette.io/en/latest/plugin_hooks.html#plugin-register-permissions) plugin hook. Plugins can now register named permissions, which will then be listed in various interfaces that show available permissions. ([#1940](https://github.com/simonw/datasette/issues/1940))\r\n- The `/db/-/create` API for [creating a table](https://docs.datasette.io/en/latest/json_api.html#tablecreateview) now accepts `\"ignore\": true` and `\"replace\": true` options when called with the `\"rows\"` property that creates a new table based on an example set of rows. This means the API can be called multiple times with different rows, setting rules for what should happen if a primary key collides with an existing row. ([#1927](https://github.com/simonw/datasette/issues/1927))\r\n- Arbitrary permissions can now be configured at the instance, database and resource (table, SQL view or canned query) level in Datasette's [Metadata](https://docs.datasette.io/en/latest/metadata.html#metadata) JSON and YAML files. The new `\"permissions\"` key can be used to specify which actors should have which permissions. See [Other permissions in metadata](https://docs.datasette.io/en/latest/authentication.html#authentication-permissions-other) for details. ([#1636](https://github.com/simonw/datasette/issues/1636))\r\n- The `/-/create-token` page can now be used to create API tokens which are restricted to just a subset of actions, including against specific databases or resources. See [API Tokens](https://docs.datasette.io/en/latest/authentication.html#createtokenview) for details. ([#1947](https://github.com/simonw/datasette/issues/1947))\r\n- Likewise, the `datasette create-token` CLI command can now create tokens with [a subset of permissions](https://docs.datasette.io/en/latest/authentication.html#authentication-cli-create-token-restrict). ([#1855](https://github.com/simonw/datasette/issues/1855))\r\n- New datasette.create_token() API method ` for programmatically creating signed API tokens. ([#1951](https://github.com/simonw/datasette/issues/1951))\r\n- `/db/-/create` API now requires actor to have `insert-row` permission in order to use the `\"row\"` or `\"rows\"` properties. ([#1937](https://github.com/simonw/datasette/issues/1937))\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1495821607, "label": "Release notes for Datasette 1.0a2"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1949#issuecomment-1352411327", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1949", "id": 1352411327, "node_id": "IC_kwDOBm6k_c5QnCi_", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T00:46:27Z", "updated_at": "2022-12-15T00:46:27Z", "author_association": "OWNER", "body": "I got this far:\r\n```diff\r\ndiff --git a/datasette/handle_exception.py b/datasette/handle_exception.py\r\nindex 8b7e83e3..31d41e00 100644\r\n--- a/datasette/handle_exception.py\r\n+++ b/datasette/handle_exception.py\r\n@@ -54,7 +54,17 @@ def handle_exception(datasette, request, exception):\r\n headers = {}\r\n if datasette.cors:\r\n add_cors_headers(headers)\r\n- if request.path.split(\"?\")[0].endswith(\".json\"):\r\n+ # Return JSON error under certain conditions\r\n+ should_return_json = (\r\n+ # URL ends in .json\r\n+ request.path.split(\"?\")[0].endswith(\".json\")\r\n+ or\r\n+ # Hints from incoming request headers\r\n+ request.headers.get(\"content-type\") == \"application/json\"\r\n+ or \"application/json\" in request.headers.get(\"accept\", \"\")\r\n+ )\r\n+ breakpoint()\r\n+ if should_return_json:\r\n return Response.json(info, status=status, headers=headers)\r\n else:\r\n template = datasette.jinja_env.select_template(templates)\r\ndiff --git a/tests/test_api_write.py b/tests/test_api_write.py\r\nindex f27d143f..982543a6 100644\r\n--- a/tests/test_api_write.py\r\n+++ b/tests/test_api_write.py\r\n@@ -1140,6 +1140,38 @@ async def test_create_table_permissions(\r\n assert data[\"errors\"] == expected_errors\r\n \r\n \r\n+@pytest.mark.asyncio\r\n+@pytest.mark.parametrize(\r\n+ \"headers,expect_json\",\r\n+ (\r\n+ ({}, False),\r\n+ ({\"Accept\": \"text/html\"}, True),\r\n+ ({\"Accept\": \"application/json\"}, True),\r\n+ ({\"Content-Type\": \"application/json\"}, True),\r\n+ ({\"Accept\": \"application/json, text/plain, */*\"}, True),\r\n+ ({\"Content-Type\": \"application/json\"}, True),\r\n+ ({\"accept\": \"application/json, text/plain, */*\"}, True),\r\n+ ({\"content-type\": \"application/json\"}, True),\r\n+ ),\r\n+)\r\n+async def test_permission_errors_html_and_json(ds_write, headers, expect_json):\r\n+ request_headers = {\"Authorization\": \"Bearer bad_token\"}\r\n+ request_headers.update(headers)\r\n+ response = await ds_write.client.post(\r\n+ \"/data/-/create\",\r\n+ json={},\r\n+ headers=request_headers,\r\n+ )\r\n+ assert response.status_code == 403\r\n+ if expect_json:\r\n+ data = response.json()\r\n+ assert data[\"ok\"] is False\r\n+ assert data[\"errors\"] == [\"Permission denied\"]\r\n+ else:\r\n+ assert response.headers[\"Content-Type\"] == \"text/html; charset=utf-8\"\r\n+ assert \"Permission denied\" in response.text\r\n+\r\n+\r\n @pytest.mark.asyncio\r\n @pytest.mark.parametrize(\r\n \"input,expected_rows_after\",\r\n```\r\nThen decided I would punt this until the next milestone.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1493471221, "label": "`.json` errors should be returned as JSON"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1953#issuecomment-1352410078", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1953", "id": 1352410078, "node_id": "IC_kwDOBm6k_c5QnCPe", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T00:44:56Z", "updated_at": "2022-12-15T00:44:56Z", "author_association": "OWNER", "body": "Highlights:\r\n\r\n- `/db/table/-/upsert`\r\n- ignore and replace for `/db/-/create`\r\n- `register_permissions()` plugin hook\r\n- `datasette create-token` can create restricted tokens\r\n- `/-/create-token` can too\r\n- `datasette --get --token` option\r\n- `datasette.create_token()` API method\r\n\r\nPlus some smaller things.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1495821607, "label": "Release notes for Datasette 1.0a2"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1949#issuecomment-1352378370", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1949", "id": 1352378370, "node_id": "IC_kwDOBm6k_c5Qm6gC", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-15T00:02:08Z", "updated_at": "2022-12-15T00:04:54Z", "author_association": "OWNER", "body": "I fixed this issue to help research this further:\r\n- https://github.com/simonw/datasette-ripgrep/issues/26\r\n\r\nNow this search works:\r\n\r\n\r\n\r\nI wish I had this feature!\r\n- https://github.com/simonw/datasette-ripgrep/issues/24\r\n\r\nLooks like I have both `_error()` and `_errors()` functions in there!\r\n\r\n![\"image\"](\"https://user-images.githubusercontent.com/9599/207741634-56667a10-ba70-42e0-98ca-ca1444a86e57.png\")
\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1493471221, "label": "`.json` errors should be returned as JSON"}, "performed_via_github_app": null}