html_url,issue_url,id,node_id,user,user_label,created_at,updated_at,author_association,body,reactions,issue,issue_label,performed_via_github_app https://github.com/simonw/datasette/pull/1960#issuecomment-1356478575,https://api.github.com/repos/simonw/datasette/issues/1960,1356478575,IC_kwDOBm6k_c5Q2jhv,9599,simonw,2022-12-17T21:47:48Z,2022-12-17T21:47:48Z,OWNER,"Stick a twig in it, this will do for the moment.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1499150951,Port as many tests as possible to async def tests against ds_client, https://github.com/simonw/datasette/pull/1960#issuecomment-1354062939,https://api.github.com/repos/simonw/datasette/issues/1960,1354062939,IC_kwDOBm6k_c5QtVxb,22429695,codecov[bot],2022-12-16T01:48:06Z,2022-12-17T21:40:43Z,NONE,"# [Codecov](https://codecov.io/gh/simonw/datasette/pull/1960?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report Base: **92.19**% // Head: **92.05**% // Decreases project coverage by **`-0.13%`** :warning: > Coverage data is based on head [(`770879a`)](https://codecov.io/gh/simonw/datasette/pull/1960?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) compared to base [(`0b68996`)](https://codecov.io/gh/simonw/datasette/commit/0b68996cc511b3a801f0cd0157bd66332d75f46f?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). > Patch coverage: 100.00% of modified lines in pull request are covered. > :exclamation: Current head 770879a differs from pull request most recent head f42bca8. Consider uploading reports for the commit f42bca8 to get more accurate results
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #1960 +/- ## ========================================== - Coverage 92.19% 92.05% -0.14% ========================================== Files 38 38 Lines 5521 5527 +6 ========================================== - Hits 5090 5088 -2 - Misses 431 439 +8 ``` | [Impacted Files](https://codecov.io/gh/simonw/datasette/pull/1960?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage Δ | | |---|---|---| | [datasette/app.py](https://codecov.io/gh/simonw/datasette/pull/1960/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL2FwcC5weQ==) | `94.13% <100.00%> (-0.57%)` | :arrow_down: | | [datasette/utils/testing.py](https://codecov.io/gh/simonw/datasette/pull/1960/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL3V0aWxzL3Rlc3RpbmcucHk=) | `95.83% <100.00%> (+0.24%)` | :arrow_up: | | [datasette/views/index.py](https://codecov.io/gh/simonw/datasette/pull/1960/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL3ZpZXdzL2luZGV4LnB5) | `96.49% <0.00%> (-1.76%)` | :arrow_down: | | [datasette/database.py](https://codecov.io/gh/simonw/datasette/pull/1960/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL2RhdGFiYXNlLnB5) | `94.57% <0.00%> (-0.61%)` | :arrow_down: | Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)
[:umbrella: View full report at Codecov](https://codecov.io/gh/simonw/datasette/pull/1960?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). :loudspeaker: Do you have feedback about the report comment? [Let us know in this issue](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1499150951,Port as many tests as possible to async def tests against ds_client, https://github.com/simonw/datasette/pull/1960#issuecomment-1356476886,https://api.github.com/repos/simonw/datasette/issues/1960,1356476886,IC_kwDOBm6k_c5Q2jHW,9599,simonw,2022-12-17T21:37:05Z,2022-12-17T21:37:05Z,OWNER,"I think this test may be to blame: https://github.com/simonw/datasette/blob/5ee954e34b6eb762ccecbdb2be0791d0166fd19c/tests/test_plugins.py#L950-L972 It's over-riding `_metadata_local` and then failing to set it back to original in a `finally:` block at the end.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1499150951,Port as many tests as possible to async def tests against ds_client, https://github.com/simonw/datasette/pull/1960#issuecomment-1356476583,https://api.github.com/repos/simonw/datasette/issues/1960,1356476583,IC_kwDOBm6k_c5Q2jCn,9599,simonw,2022-12-17T21:34:51Z,2022-12-17T21:34:51Z,OWNER,"These are all the places that tests touch `_metadata_local` at the moment: ``` (venv) root@76a81d2417f5:/tmp/datasette/tests# rg _metadata_local test_facets.py 596: ds._metadata_local = { 605: ds._metadata_local[""databases""][""test_facet_size""][""tables""][""neighbourhoods""][ test_permissions.py 62: padlock_client.ds._metadata_local[""allow""] = allow 77: del padlock_client.ds._metadata_local[""allow""] 522: cascade_app_client.ds._metadata_local = updated_metadata 533: cascade_app_client.ds._metadata_local = previous_metadata 549: previous_metadata = cascade_app_client.ds._metadata_local 551: cascade_app_client.ds._metadata_local = metadata 566: cascade_app_client.ds._metadata_local = previous_metadata 842: perms_ds._metadata_local = updated_metadata 849: perms_ds._metadata_local = previous_metadata test_html.py 1114: orig = ds_client.ds._metadata_local 1115: ds_client.ds._metadata_local = metadata 1123: ds_client.ds._metadata_local = orig test_plugins.py 1034: ds_client.ds._metadata_local = { ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1499150951,Port as many tests as possible to async def tests against ds_client, https://github.com/simonw/datasette/pull/1960#issuecomment-1356056018,https://api.github.com/repos/simonw/datasette/issues/1960,1356056018,IC_kwDOBm6k_c5Q08XS,9599,simonw,2022-12-17T05:33:26Z,2022-12-17T05:33:26Z,OWNER,"I think I've found the problem. The failing test is this one: `paginated_view.json-201-9` That's this example: https://github.com/simonw/datasette/blob/cede1efeedbc3d928397d53d5a1611eecc598fde/tests/test_table_api.py#L179-L180 Why is it expected to take 9 pages and not 5, when the definition of that view is this: ```sql CREATE VIEW paginated_view AS SELECT content, '- ' || content || ' -' AS content_extra FROM no_primary_key; ``` Because `paginated_view` has extra configuration in `metadata.json`: https://github.com/simonw/datasette/blob/5ee954e34b6eb762ccecbdb2be0791d0166fd19c/tests/fixtures.py#L357 So this test is showing that `metadata` can be used to set an alternative page size for a view. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1499150951,Port as many tests as possible to async def tests against ds_client, https://github.com/simonw/datasette/pull/1960#issuecomment-1356038242,https://api.github.com/repos/simonw/datasette/issues/1960,1356038242,IC_kwDOBm6k_c5Q04Bi,9599,simonw,2022-12-17T04:56:22Z,2022-12-17T04:57:04Z,OWNER,"May have spotted the problem with that `test_paginate_tables_and_views` test: ``` (Pdb) path, expected_rows, expected_pages (None, 201, 9) ``` `path` should not be `None` here: https://github.com/simonw/datasette/blob/cede1efeedbc3d928397d53d5a1611eecc598fde/tests/test_table_api.py#L175-L212 No that's not it either - `path` is reassigned on purpose.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1499150951,Port as many tests as possible to async def tests against ds_client, https://github.com/simonw/datasette/pull/1960#issuecomment-1355823260,https://api.github.com/repos/simonw/datasette/issues/1960,1355823260,IC_kwDOBm6k_c5Q0Dic,9599,simonw,2022-12-16T23:36:07Z,2022-12-16T23:36:07Z,OWNER,"I ran `pytest --pdb -x` to drop into the debugger on the first failing test. ``` assert expected_rows == len(fetched) > assert expected_pages == count E assert 9 == 5 ``` That's in `test_paginate_tables_and_views()`. On a hunch, I checked the current settings: ``` (Pdb) ds_client.ds._settings {'default_page_size': 50, 'max_returned_rows': 100, 'max_insert_rows': 100, 'num_sql_threads': 1, 'sql_time_limit_ms': 200, 'default_facet_size': 30, 'facet_time_limit_ms': 200, 'facet_suggest_time_limit_ms': 50, 'allow_facet': True, 'allow_download': True, 'allow_signed_tokens': True, 'max_signed_tokens_ttl': 0, 'suggest_facets': True, 'default_cache_ttl': 5, 'cache_size_kb': 0, 'allow_csv_stream': True, 'max_csv_mb': 100, 'truncate_cells_html': 2048, 'force_https_urls': False, 'template_debug': False, 'trace_debug': False, 'base_url': '/'} ``` Looks like something changed `default_page_size` to 50 and forgot to change it back!","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1499150951,Port as many tests as possible to async def tests against ds_client, https://github.com/simonw/datasette/pull/1960#issuecomment-1355685828,https://api.github.com/repos/simonw/datasette/issues/1960,1355685828,IC_kwDOBm6k_c5Qzh_E,9599,simonw,2022-12-16T21:50:01Z,2022-12-16T21:50:01Z,OWNER,"Looks like that `@pytest.mark.ds_client` mark I've been using isn't necessary - I added that so I could easily run `pytest -m ds_client` to execute all tests that I had ported to the new feature, but actually this achieves the same thing: pytest -k ds_client So I'm going to remove the mark.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1499150951,Port as many tests as possible to async def tests against ds_client, https://github.com/simonw/datasette/pull/1960#issuecomment-1355478743,https://api.github.com/repos/simonw/datasette/issues/1960,1355478743,IC_kwDOBm6k_c5QyvbX,9599,simonw,2022-12-16T19:27:12Z,2022-12-16T19:27:12Z,OWNER,"Bad news: they're definitely caused by tests that are subtly affected by other tests. This passes without errors: pytest -k test_paginate_tables_and_views","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1499150951,Port as many tests as possible to async def tests against ds_client, https://github.com/simonw/datasette/pull/1960#issuecomment-1355475671,https://api.github.com/repos/simonw/datasette/issues/1960,1355475671,IC_kwDOBm6k_c5QyurX,9599,simonw,2022-12-16T19:26:08Z,2022-12-16T19:26:08Z,OWNER,"Great news! The test failures I got running on my laptop (with that fresh Ubuntu Docker image) look like they match the failures I saw in CI: ``` ======== short test summary info ======== FAILED tests/test_table_api.py::test_paginate_tables_and_views[/fixtures/paginated_view.json-201-9] - assert 9 == 5 FAILED tests/test_table_api.py::test_sortable_columns_metadata - KeyError: 'error' FAILED tests/test_table_api.py::test_searchable_views[/fixtures/searchable_view_configured_by_metadata.json?_search=weasel-expected_rows0] - AssertionError: assert [[2, 'terry d...sel', 'puma']] == [[1, 'barry c...sel', 'puma']] FAILED tests/test_table_api.py::test_unit_filters - KeyError: 'distance' FAILED tests/test_table_api.py::test_custom_query_with_unicode_characters - AssertionError: assert {'error': 'Ta...'title': None} == [{'id': 1, 'n...n Francisco'}] FAILED tests/test_table_html.py::test_sort_links - AssertionError: assert [{'a_href': N...', ...}}, ...] == [{'a_href': N...', ...}}, ...] FAILED tests/test_table_html.py::test_table_html_no_primary_key - AttributeError: 'NoneType' object has no attribute 'strip' FAILED tests/test_table_html.py::test_table_html_foreign_key_custom_label_column - assert [['']] == [['1']] FAILED tests/test_table_html.py::test_view_html - assert upper_content is None FAILED tests/test_table_html.py::test_table_metadata - AssertionError: assert 'This <em&...t; is escaped' == 'simple_primary_key' FAILED tests/test_table_html.py::test_metadata_sort - AssertionError: assert ['id', 'name\xa0▼'] == ['id\xa0▼', 'name'] FAILED tests/test_table_html.py::test_metadata_sort_desc - AssertionError: assert ['pk\xa0▲', 'name'] == ['pk\xa0▼', 'name'] FAILED tests/test_table_html.py::test_column_metadata - AttributeError: 'NoneType' object has no attribute 'findAll' ======== 13 failed, 1279 passed, 3 skipped, 57 warnings in 572.40s (0:09:32) ======== ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1499150951,Port as many tests as possible to async def tests against ds_client, https://github.com/simonw/datasette/pull/1960#issuecomment-1355471341,https://api.github.com/repos/simonw/datasette/issues/1960,1355471341,IC_kwDOBm6k_c5Qytnt,9599,simonw,2022-12-16T19:23:33Z,2022-12-16T19:23:33Z,OWNER,"Trying this instead: ``` docker run -it ubuntu:22.04 /bin/bash ``` Then in that shell: ``` apt-get update apt-get install python3.11 python3.11-venv git -y cd /tmp git clone https://github.com/simonw/datasette cd datasette git checkout async-tests python3.11 -m venv venv pip install -e '.[test]' pytest ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1499150951,Port as many tests as possible to async def tests against ds_client, https://github.com/simonw/datasette/pull/1960#issuecomment-1355445710,https://api.github.com/repos/simonw/datasette/issues/1960,1355445710,IC_kwDOBm6k_c5QynXO,9599,simonw,2022-12-16T19:06:49Z,2022-12-16T19:09:35Z,OWNER,"This would be much easier to debug if I could use Docker to run the GitHub Actions image directly on my own laptop. https://github.com/actions/runner-images/blob/releases/ubuntu22/20221212/images/linux/Ubuntu2204-Readme.md is the README for their most recent image. Not sure if there's an easy way to run it in Docker though. https://github.com/actions/runner-images/blob/main/docs/create-image-and-azure-resources.md is instructions for building them locally - looks fiddly though, involves https://www.packer.io/","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1499150951,Port as many tests as possible to async def tests against ds_client, https://github.com/simonw/datasette/pull/1960#issuecomment-1355325426,https://api.github.com/repos/simonw/datasette/issues/1960,1355325426,IC_kwDOBm6k_c5QyJ_y,9599,simonw,2022-12-16T18:00:40Z,2022-12-16T18:00:40Z,OWNER,Many of the failing tests pass on my laptop but fail in CI.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1499150951,Port as many tests as possible to async def tests against ds_client, https://github.com/simonw/datasette/pull/1960#issuecomment-1355319541,https://api.github.com/repos/simonw/datasette/issues/1960,1355319541,IC_kwDOBm6k_c5QyIj1,9599,simonw,2022-12-16T17:58:24Z,2022-12-16T17:58:46Z,OWNER,"> I tried adding `invoke_startup()` to the `ds_client()` fixture to see if that would fix this. It did not: I'm still seeing those same failures. Frustrating: https://github.com/simonw/datasette/actions/runs/3715317653/jobs/6300336884 ====== 11 failed, 1252 passed, 1 skipped, 1 warning in 185.77s (0:03:05) =======","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1499150951,Port as many tests as possible to async def tests against ds_client, https://github.com/simonw/datasette/pull/1960#issuecomment-1355317369,https://api.github.com/repos/simonw/datasette/issues/1960,1355317369,IC_kwDOBm6k_c5QyIB5,9599,simonw,2022-12-16T17:57:14Z,2022-12-16T17:57:14Z,OWNER,"Opened a follow-up issue here: - #1962","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1499150951,Port as many tests as possible to async def tests against ds_client, https://github.com/simonw/datasette/pull/1960#issuecomment-1355313058,https://api.github.com/repos/simonw/datasette/issues/1960,1355313058,IC_kwDOBm6k_c5QyG-i,9599,simonw,2022-12-16T17:53:17Z,2022-12-16T17:53:17Z,OWNER,"Got some surprising test failures here: https://github.com/simonw/datasette/actions/runs/3715317653/jobs/6300336626 ``` > assert response.json() == [{""id"": 1, ""name"": ""San Francisco""}] E AssertionError: assert {'error': 'Ta...'title': None} == [{'id': 1, 'n...n Francisco'}] E Full diff: E - [{'id': 1, 'name': 'San Francisco'}] E + {'error': 'Table not found: 𝐜𝐢𝐭𝐢𝐞𝐬', 'ok': False, 'status': 404, 'title': None} ``` A hunch: this failure suggests that maybe the fixtures tables were not correctly created when this test run. Maybe that can happen when `python -n auto` runs a bunch of separate processes and hence one of the tests randomly gets run in a fresh process and executes before the in-memory fixtures database has been fully populated. I tried adding `invoke_startup()` to the `ds_client()` fixture to see if that would fix this.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1499150951,Port as many tests as possible to async def tests against ds_client, https://github.com/simonw/datasette/pull/1960#issuecomment-1355300217,https://api.github.com/repos/simonw/datasette/issues/1960,1355300217,IC_kwDOBm6k_c5QyD15,9599,simonw,2022-12-16T17:44:55Z,2022-12-16T17:44:55Z,OWNER,That's enough for this round. I'll get the tests passing and land this.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1499150951,Port as many tests as possible to async def tests against ds_client, https://github.com/simonw/datasette/pull/1960#issuecomment-1354269873,https://api.github.com/repos/simonw/datasette/issues/1960,1354269873,IC_kwDOBm6k_c5QuISx,9599,simonw,2022-12-16T06:11:43Z,2022-12-16T06:11:43Z,OWNER,"This is quite fast: ``` % pytest -m ds_client -n auto ================================================================================== test session starts ================================================================================== platform darwin -- Python 3.10.3, pytest-7.1.3, pluggy-1.0.0 SQLite: 3.39.4 rootdir: /Users/simon/Dropbox/Development/datasette, configfile: pytest.ini plugins: anyio-3.6.1, xdist-2.5.0, forked-1.4.0, asyncio-0.19.0, timeout-2.1.0, profiling-1.7.0 asyncio: mode=strict gw0 [291] / gw1 [291] / gw2 [291] / gw3 [291] / gw4 [291] / gw5 [291] ................................................................................................................................................................................. [ 60%] .................................................................................................................. [100%] ================================================================================== 291 passed in 6.30s ================================================================================== ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1499150951,Port as many tests as possible to async def tests against ds_client, https://github.com/simonw/datasette/pull/1960#issuecomment-1354148139,https://api.github.com/repos/simonw/datasette/issues/1960,1354148139,IC_kwDOBm6k_c5Qtqkr,9599,simonw,2022-12-16T03:32:52Z,2022-12-16T03:32:52Z,OWNER,"Got that done to: 68 passed in 14.92s By implementing my own global variable - since `pytest` won't use a global fixture for me, I decided to do it for myself.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1499150951,Port as many tests as possible to async def tests against ds_client, https://github.com/simonw/datasette/pull/1960#issuecomment-1354072344,https://api.github.com/repos/simonw/datasette/issues/1960,1354072344,IC_kwDOBm6k_c5QtYEY,9599,simonw,2022-12-16T02:00:07Z,2022-12-16T02:00:07Z,OWNER,"It did NOT speed it up: 68 passed in 26.26s","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1499150951,Port as many tests as possible to async def tests against ds_client, https://github.com/simonw/datasette/pull/1960#issuecomment-1354061440,https://api.github.com/repos/simonw/datasette/issues/1960,1354061440,IC_kwDOBm6k_c5QtVaA,9599,simonw,2022-12-16T01:45:38Z,2022-12-16T01:45:38Z,OWNER,"I'm going to do `test_table_html.py` next. Currently: 68 passed in 17.20s Will this speed it up?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1499150951,Port as many tests as possible to async def tests against ds_client, https://github.com/simonw/datasette/pull/1960#issuecomment-1354053151,https://api.github.com/repos/simonw/datasette/issues/1960,1354053151,IC_kwDOBm6k_c5QtTYf,9599,simonw,2022-12-16T01:33:22Z,2022-12-16T01:33:22Z,OWNER,"The thing with `Datasette(memory=False)` is tripping me up. The problem is that the tests written against `app_client` - which I want to replace - all assume that there is no `_memory` database, because when you start Datasette with at least one database file it doesn't enable `_memory` unless you explicitly tell it to. But the new `ds_client` fixture works by creating a named in-memory database called `fixtures`, which it does with a call to `ds.add_memory_database(""fixtures"")` after the object has been instantiated. This results in a datasette instance that DOES have a `_memory` database, when we didn't want one. My initial solution attempt was a huge hack - I decided that if you pass `memory=False` to the `Datasette` constructor it should mean ""don't add a `_memory` database even though I didn't pass any files"". I set a the default `memory` argument to `None`. This is weird and surprising (`memory=False` no does something different from `memory=None`?) and I found other tests that it broke, like this one: ```python def test_sql_errors_logged_to_stderr(): runner = CliRunner(mix_stderr=False) result = runner.invoke(cli, [""--get"", ""/_memory.json?sql=select+blah""]) assert result.exit_code == 1 assert ""sql = 'select blah', params = {}: no such column: blah\n"" in result.stderr ``` It ended up with no `_memory` database because it turns out `datasette serve ...` passes `memory=False` without me realizing it. So I'm going to undo that hack and teach the fixture to do this instead: ```python db = ds.add_memory_database(""fixtures"") ds.remove_database(""_memory"") ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1499150951,Port as many tests as possible to async def tests against ds_client, https://github.com/simonw/datasette/pull/1960#issuecomment-1354046627,https://api.github.com/repos/simonw/datasette/issues/1960,1354046627,IC_kwDOBm6k_c5QtRyj,9599,simonw,2022-12-16T01:23:21Z,2022-12-16T01:23:21Z,OWNER,"This does seem to help: ```diff diff --git a/tests/conftest.py b/tests/conftest.py index 1306c407..af9c7696 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -25,12 +25,7 @@ UNDOCUMENTED_PERMISSIONS = { } -@pytest.fixture(scope=""session"") -def event_loop(): - return asyncio.get_event_loop() - - -@pytest_asyncio.fixture(scope=""session"") +@pytest_asyncio.fixture async def ds_client(): from datasette.app import Datasette from .fixtures import METADATA, PLUGINS_DIR @@ -53,10 +48,11 @@ async def ds_client(): db = ds.add_memory_database(""fixtures"") def prepare(conn): - conn.executescript(TABLES) - for sql, params in TABLE_PARAMETERIZED_SQL: - with conn: - conn.execute(sql, params) + if not conn.execute(""select count(*) from sqlite_master"").fetchone()[0]: + conn.executescript(TABLES) + for sql, params in TABLE_PARAMETERIZED_SQL: + with conn: + conn.execute(sql, params) await db.execute_write_fn(prepare) return ds.client diff --git a/tests/plugins/my_plugin_2.py b/tests/plugins/my_plugin_2.py index 4f7bf08c..d588342c 100644 --- a/tests/plugins/my_plugin_2.py +++ b/tests/plugins/my_plugin_2.py @@ -117,7 +117,12 @@ def actor_from_request(datasette, request): def permission_allowed(datasette, actor, action): # Testing asyncio version of permission_allowed async def inner(): - assert 2 == (await datasette.get_database().execute(""select 1 + 1"")).first()[0] + assert ( + 2 + == ( + await datasette.get_database(""_internal"").execute(""select 1 + 1"") + ).first()[0] + ) if action == ""this_is_allowed_async"": return True elif action == ""this_is_denied_async"": ``` `pytest -m ds_client` now passes 134 tests. Need to get `pytest -n auto` passing too.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1499150951,Port as many tests as possible to async def tests against ds_client, https://github.com/simonw/datasette/pull/1960#issuecomment-1354036967,https://api.github.com/repos/simonw/datasette/issues/1960,1354036967,IC_kwDOBm6k_c5QtPbn,9599,simonw,2022-12-16T01:10:12Z,2022-12-16T01:10:12Z,OWNER,"If it does turn out that I can't use `scope=""session""` on this fixture it might not actually be a showstopper: I can take advantage of the fact that `memory_name=""...""` databases stay present in memory for the duration of the process, so I could have it such that each test that uses the `ds_client` fixture DOES construct a fresh `Datasette` instance, but doesn't need to populate the database since they can re-use the in-memory database from the previous object.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1499150951,Port as many tests as possible to async def tests against ds_client, https://github.com/simonw/datasette/pull/1960#issuecomment-1353812913,https://api.github.com/repos/simonw/datasette/issues/1960,1353812913,IC_kwDOBm6k_c5QsYux,9599,simonw,2022-12-15T22:48:54Z,2022-12-15T22:48:54Z,OWNER,"This is all very broken: ``` % pytest -x --pdb ================================================================================== test session starts ================================================================================== platform darwin -- Python 3.10.3, pytest-7.1.3, pluggy-1.0.0 SQLite: 3.39.4 rootdir: /Users/simon/Dropbox/Development/datasette, configfile: pytest.ini plugins: anyio-3.6.1, xdist-2.5.0, forked-1.4.0, asyncio-0.19.0, timeout-2.1.0, profiling-1.7.0 asyncio: mode=strict collected 1295 items tests/test_package.py .. [ 0%] tests/test_cli.py . [ 0%] tests/test_cli_serve_get.py .. [ 0%] tests/test_cli.py . [ 0%] tests/test_black.py . [ 0%] tests/test_api.py E >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> traceback >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fixturedef = , request = > @pytest.hookimpl(hookwrapper=True) def pytest_fixture_setup( fixturedef: FixtureDef, request: SubRequest ) -> Optional[object]: """"""Adjust the event loop policy when an event loop is produced."""""" if fixturedef.argname == ""event_loop"": outcome = yield > loop = outcome.get_result() /Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/pytest_asyncio/plugin.py:377: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /Users/simon/Dropbox/Development/datasette/tests/conftest.py:30: in event_loop return asyncio.get_event_loop() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = def get_event_loop(self): """"""Get the event loop for the current context. Returns an instance of EventLoop or raises an exception. """""" if (self._local._loop is None and not self._local._set_called and threading.current_thread() is threading.main_thread()): self.set_event_loop(self.new_event_loop()) if self._local._loop is None: > raise RuntimeError('There is no current event loop in thread %r.' % threading.current_thread().name) E RuntimeError: There is no current event loop in thread 'MainThread'. /Users/simon/.pyenv/versions/3.10.3/lib/python3.10/asyncio/events.py:656: RuntimeError >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> entering PDB >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PDB post_mortem (IO-capturing turned off) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > /Users/simon/.pyenv/versions/3.10.3/lib/python3.10/asyncio/events.py(656)get_event_loop() -> raise RuntimeError('There is no current event loop in thread %r.' (Pdb) q =================================================================================== warnings summary ==================================================================================== tests/test_cli.py::test_inspect_cli_writes_to_file tests/test_cli.py::test_inspect_cli /Users/simon/Dropbox/Development/datasette/datasette/cli.py:163: DeprecationWarning: There is no current event loop loop = asyncio.get_event_loop() tests/test_cli_serve_get.py::test_serve_with_get_exit_code_for_error tests/test_cli_serve_get.py::test_serve_with_get /Users/simon/Dropbox/Development/datasette/datasette/cli.py:596: DeprecationWarning: There is no current event loop asyncio.get_event_loop().run_until_complete(ds.invoke_startup()) tests/test_cli_serve_get.py::test_serve_with_get_exit_code_for_error tests/test_cli_serve_get.py::test_serve_with_get /Users/simon/Dropbox/Development/datasette/datasette/cli.py:599: DeprecationWarning: There is no current event loop asyncio.get_event_loop().run_until_complete(check_databases(ds)) tests/test_api.py::test_homepage /Users/simon/Dropbox/Development/datasette/tests/conftest.py:30: DeprecationWarning: There is no current event loop return asyncio.get_event_loop() -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html ================================================================================ short test summary info ================================================================================ ERROR tests/test_api.py::test_homepage - RuntimeError: There is no current event loop in thread 'MainThread'. !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! _pytest.outcomes.Exit: Quitting debugger !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! ======================================================================== 7 passed, 7 warnings, 1 error in 19.15s ======================================================================== (datasette) datasette % ``` The problem looks to be caused by this: https://github.com/simonw/datasette/blob/87737aa1ace82fa7b54c60c41471ec9a661f5299/tests/conftest.py#L28-L30 Which I found necessary in order to have `async def` fixtures that could be shared on the `scope=""session""` basis. Can I work around this, or is `scope=""session""` for async fixtures incompatible with my test suite for some reason? ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1499150951,Port as many tests as possible to async def tests against ds_client, https://github.com/simonw/datasette/pull/1960#issuecomment-1353805839,https://api.github.com/repos/simonw/datasette/issues/1960,1353805839,IC_kwDOBm6k_c5QsXAP,9599,simonw,2022-12-15T22:38:37Z,2022-12-15T22:38:37Z,OWNER,"I'm going to make `.status_code` work on `TestClient` response too, so I don't have to worry about using both `status` or `status_code` depending on which kind of object I am using.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1499150951,Port as many tests as possible to async def tests against ds_client, https://github.com/simonw/datasette/pull/1960#issuecomment-1353765125,https://api.github.com/repos/simonw/datasette/issues/1960,1353765125,IC_kwDOBm6k_c5QsNEF,9599,simonw,2022-12-15T22:00:04Z,2022-12-15T22:00:04Z,OWNER,I'm going to punt on that for the moment and continue to use `app_client` for tests that use that mechanism.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1499150951,Port as many tests as possible to async def tests against ds_client, https://github.com/simonw/datasette/pull/1960#issuecomment-1353763837,https://api.github.com/repos/simonw/datasette/issues/1960,1353763837,IC_kwDOBm6k_c5QsMv9,9599,simonw,2022-12-15T21:59:05Z,2022-12-15T21:59:05Z,OWNER,"Here's an annoying error: ``` > response4 = await ds_client.post( ""/-/logout"", csrftoken_from=True, cookies={""ds_actor"": ds_client.actor_cookie({""id"": ""test""})}, ) tests/test_auth.py:88: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = , path = '/-/logout' kwargs = {'cookies': {'ds_actor': 'eyJhIjp7ImlkIjoidGVzdCJ9fQ.fuFCTJG5XE-RNnUM7dcnXx9sPvE'}, 'csrftoken_from': True}, client = async def post(self, path, **kwargs): await self.ds.invoke_startup() async with httpx.AsyncClient(app=self.app) as client: > return await client.post(self._fix(path), **kwargs) E TypeError: AsyncClient.post() got an unexpected keyword argument 'csrftoken_from' ``` I need an alternative to the `csrftoken_from` mechanism I built for `TestClient`: https://github.com/simonw/datasette/blob/0b68996cc511b3a801f0cd0157bd66332d75f46f/datasette/utils/testing.py#L77-L103","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1499150951,Port as many tests as possible to async def tests against ds_client, https://github.com/simonw/datasette/pull/1960#issuecomment-1353749401,https://api.github.com/repos/simonw/datasette/issues/1960,1353749401,IC_kwDOBm6k_c5QsJOZ,9599,simonw,2022-12-15T21:47:27Z,2022-12-15T21:47:27Z,OWNER,"I'm using this new mark: ```python @pytest.mark.ds_client ``` Purely so I can run all of the tests that I've refactored using: ``` pytest -m ds_client ``` I'll likely remove this once the test refactoring project is complete.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1499150951,Port as many tests as possible to async def tests against ds_client,