github
html_url | issue_url | id | node_id | user | created_at | updated_at | author_association | body | reactions | issue | performed_via_github_app |
---|---|---|---|---|---|---|---|---|---|---|---|
https://github.com/simonw/datasette/pull/1960#issuecomment-1354036967 | https://api.github.com/repos/simonw/datasette/issues/1960 | 1354036967 | IC_kwDOBm6k_c5QtPbn | 9599 | 2022-12-16T01:10:12Z | 2022-12-16T01:10:12Z | OWNER | If it does turn out that I can't use `scope="session"` on this fixture it might not actually be a showstopper: I can take advantage of the fact that `memory_name="..."` databases stay present in memory for the duration of the process, so I could have it such that each test that uses the `ds_client` fixture DOES construct a fresh `Datasette` instance, but doesn't need to populate the database since they can re-use the in-memory database from the previous object. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1499150951 | |
https://github.com/simonw/datasette/pull/1960#issuecomment-1354046627 | https://api.github.com/repos/simonw/datasette/issues/1960 | 1354046627 | IC_kwDOBm6k_c5QtRyj | 9599 | 2022-12-16T01:23:21Z | 2022-12-16T01:23:21Z | OWNER | This does seem to help: ```diff diff --git a/tests/conftest.py b/tests/conftest.py index 1306c407..af9c7696 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -25,12 +25,7 @@ UNDOCUMENTED_PERMISSIONS = { } -@pytest.fixture(scope="session") -def event_loop(): - return asyncio.get_event_loop() - - -@pytest_asyncio.fixture(scope="session") +@pytest_asyncio.fixture async def ds_client(): from datasette.app import Datasette from .fixtures import METADATA, PLUGINS_DIR @@ -53,10 +48,11 @@ async def ds_client(): db = ds.add_memory_database("fixtures") def prepare(conn): - conn.executescript(TABLES) - for sql, params in TABLE_PARAMETERIZED_SQL: - with conn: - conn.execute(sql, params) + if not conn.execute("select count(*) from sqlite_master").fetchone()[0]: + conn.executescript(TABLES) + for sql, params in TABLE_PARAMETERIZED_SQL: + with conn: + conn.execute(sql, params) await db.execute_write_fn(prepare) return ds.client diff --git a/tests/plugins/my_plugin_2.py b/tests/plugins/my_plugin_2.py index 4f7bf08c..d588342c 100644 --- a/tests/plugins/my_plugin_2.py +++ b/tests/plugins/my_plugin_2.py @@ -117,7 +117,12 @@ def actor_from_request(datasette, request): def permission_allowed(datasette, actor, action): # Testing asyncio version of permission_allowed async def inner(): - assert 2 == (await datasette.get_database().execute("select 1 + 1")).first()[0] + assert ( + 2 + == ( + await datasette.get_database("_internal").execute("select 1 + 1") + ).first()[0] + ) if action == "this_is_allowed_async": return True elif action == "this_is_denied_async": ``` `pytest -m ds_client` now passes 134 tests. Need to get `pytest -n auto` passing too. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1499150951 | |
https://github.com/simonw/datasette/pull/1960#issuecomment-1354053151 | https://api.github.com/repos/simonw/datasette/issues/1960 | 1354053151 | IC_kwDOBm6k_c5QtTYf | 9599 | 2022-12-16T01:33:22Z | 2022-12-16T01:33:22Z | OWNER | The thing with `Datasette(memory=False)` is tripping me up. The problem is that the tests written against `app_client` - which I want to replace - all assume that there is no `_memory` database, because when you start Datasette with at least one database file it doesn't enable `_memory` unless you explicitly tell it to. But the new `ds_client` fixture works by creating a named in-memory database called `fixtures`, which it does with a call to `ds.add_memory_database("fixtures")` after the object has been instantiated. This results in a datasette instance that DOES have a `_memory` database, when we didn't want one. My initial solution attempt was a huge hack - I decided that if you pass `memory=False` to the `Datasette` constructor it should mean "don't add a `_memory` database even though I didn't pass any files". I set a the default `memory` argument to `None`. This is weird and surprising (`memory=False` no does something different from `memory=None`?) and I found other tests that it broke, like this one: ```python def test_sql_errors_logged_to_stderr(): runner = CliRunner(mix_stderr=False) result = runner.invoke(cli, ["--get", "/_memory.json?sql=select+blah"]) assert result.exit_code == 1 assert "sql = 'select blah', params = {}: no such column: blah\n" in result.stderr ``` It ended up with no `_memory` database because it turns out `datasette serve ...` passes `memory=False` without me realizing it. So I'm going to undo that hack and teach the fixture to do this instead: ```python db = ds.add_memory_database("fixtures") ds.remove_database("_memory") ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1499150951 | |
https://github.com/simonw/datasette/pull/1960#issuecomment-1354061440 | https://api.github.com/repos/simonw/datasette/issues/1960 | 1354061440 | IC_kwDOBm6k_c5QtVaA | 9599 | 2022-12-16T01:45:38Z | 2022-12-16T01:45:38Z | OWNER | I'm going to do `test_table_html.py` next. Currently: 68 passed in 17.20s Will this speed it up? | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1499150951 | |
https://github.com/simonw/datasette/pull/1960#issuecomment-1354072344 | https://api.github.com/repos/simonw/datasette/issues/1960 | 1354072344 | IC_kwDOBm6k_c5QtYEY | 9599 | 2022-12-16T02:00:07Z | 2022-12-16T02:00:07Z | OWNER | It did NOT speed it up: 68 passed in 26.26s | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1499150951 | |
https://github.com/simonw/datasette/pull/1960#issuecomment-1354148139 | https://api.github.com/repos/simonw/datasette/issues/1960 | 1354148139 | IC_kwDOBm6k_c5Qtqkr | 9599 | 2022-12-16T03:32:52Z | 2022-12-16T03:32:52Z | OWNER | Got that done to: 68 passed in 14.92s By implementing my own global variable - since `pytest` won't use a global fixture for me, I decided to do it for myself. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1499150951 | |
https://github.com/simonw/datasette/pull/1960#issuecomment-1354269873 | https://api.github.com/repos/simonw/datasette/issues/1960 | 1354269873 | IC_kwDOBm6k_c5QuISx | 9599 | 2022-12-16T06:11:43Z | 2022-12-16T06:11:43Z | OWNER | This is quite fast: ``` % pytest -m ds_client -n auto ================================================================================== test session starts ================================================================================== platform darwin -- Python 3.10.3, pytest-7.1.3, pluggy-1.0.0 SQLite: 3.39.4 rootdir: /Users/simon/Dropbox/Development/datasette, configfile: pytest.ini plugins: anyio-3.6.1, xdist-2.5.0, forked-1.4.0, asyncio-0.19.0, timeout-2.1.0, profiling-1.7.0 asyncio: mode=strict gw0 [291] / gw1 [291] / gw2 [291] / gw3 [291] / gw4 [291] / gw5 [291] ................................................................................................................................................................................. [ 60%] .................................................................................................................. [100%] ================================================================================== 291 passed in 6.30s ================================================================================== ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1499150951 | |
https://github.com/simonw/datasette/pull/1960#issuecomment-1355300217 | https://api.github.com/repos/simonw/datasette/issues/1960 | 1355300217 | IC_kwDOBm6k_c5QyD15 | 9599 | 2022-12-16T17:44:55Z | 2022-12-16T17:44:55Z | OWNER | That's enough for this round. I'll get the tests passing and land this. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1499150951 | |
https://github.com/simonw/datasette/pull/1960#issuecomment-1355313058 | https://api.github.com/repos/simonw/datasette/issues/1960 | 1355313058 | IC_kwDOBm6k_c5QyG-i | 9599 | 2022-12-16T17:53:17Z | 2022-12-16T17:53:17Z | OWNER | Got some surprising test failures here: https://github.com/simonw/datasette/actions/runs/3715317653/jobs/6300336626 ``` > assert response.json() == [{"id": 1, "name": "San Francisco"}] E AssertionError: assert {'error': 'Ta...'title': None} == [{'id': 1, 'n...n Francisco'}] E Full diff: E - [{'id': 1, 'name': 'San Francisco'}] E + {'error': 'Table not found: 𝐜𝐢𝐭𝐢𝐞𝐬', 'ok': False, 'status': 404, 'title': None} ``` A hunch: this failure suggests that maybe the fixtures tables were not correctly created when this test run. Maybe that can happen when `python -n auto` runs a bunch of separate processes and hence one of the tests randomly gets run in a fresh process and executes before the in-memory fixtures database has been fully populated. I tried adding `invoke_startup()` to the `ds_client()` fixture to see if that would fix this. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1499150951 | |
https://github.com/simonw/datasette/pull/1960#issuecomment-1355317369 | https://api.github.com/repos/simonw/datasette/issues/1960 | 1355317369 | IC_kwDOBm6k_c5QyIB5 | 9599 | 2022-12-16T17:57:14Z | 2022-12-16T17:57:14Z | OWNER | Opened a follow-up issue here: - #1962 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1499150951 | |
https://github.com/simonw/datasette/pull/1960#issuecomment-1355319541 | https://api.github.com/repos/simonw/datasette/issues/1960 | 1355319541 | IC_kwDOBm6k_c5QyIj1 | 9599 | 2022-12-16T17:58:24Z | 2022-12-16T17:58:46Z | OWNER | > I tried adding `invoke_startup()` to the `ds_client()` fixture to see if that would fix this. It did not: I'm still seeing those same failures. Frustrating: https://github.com/simonw/datasette/actions/runs/3715317653/jobs/6300336884 ====== 11 failed, 1252 passed, 1 skipped, 1 warning in 185.77s (0:03:05) ======= | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1499150951 | |
https://github.com/simonw/datasette/pull/1960#issuecomment-1355325426 | https://api.github.com/repos/simonw/datasette/issues/1960 | 1355325426 | IC_kwDOBm6k_c5QyJ_y | 9599 | 2022-12-16T18:00:40Z | 2022-12-16T18:00:40Z | OWNER | Many of the failing tests pass on my laptop but fail in CI. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1499150951 | |
https://github.com/simonw/datasette/pull/1960#issuecomment-1355445710 | https://api.github.com/repos/simonw/datasette/issues/1960 | 1355445710 | IC_kwDOBm6k_c5QynXO | 9599 | 2022-12-16T19:06:49Z | 2022-12-16T19:09:35Z | OWNER | This would be much easier to debug if I could use Docker to run the GitHub Actions image directly on my own laptop. https://github.com/actions/runner-images/blob/releases/ubuntu22/20221212/images/linux/Ubuntu2204-Readme.md is the README for their most recent image. Not sure if there's an easy way to run it in Docker though. https://github.com/actions/runner-images/blob/main/docs/create-image-and-azure-resources.md is instructions for building them locally - looks fiddly though, involves https://www.packer.io/ | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1499150951 | |
https://github.com/simonw/datasette/pull/1960#issuecomment-1355471341 | https://api.github.com/repos/simonw/datasette/issues/1960 | 1355471341 | IC_kwDOBm6k_c5Qytnt | 9599 | 2022-12-16T19:23:33Z | 2022-12-16T19:23:33Z | OWNER | Trying this instead: ``` docker run -it ubuntu:22.04 /bin/bash ``` Then in that shell: ``` apt-get update apt-get install python3.11 python3.11-venv git -y cd /tmp git clone https://github.com/simonw/datasette cd datasette git checkout async-tests python3.11 -m venv venv pip install -e '.[test]' pytest ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1499150951 | |
https://github.com/simonw/datasette/pull/1960#issuecomment-1355475671 | https://api.github.com/repos/simonw/datasette/issues/1960 | 1355475671 | IC_kwDOBm6k_c5QyurX | 9599 | 2022-12-16T19:26:08Z | 2022-12-16T19:26:08Z | OWNER | Great news! The test failures I got running on my laptop (with that fresh Ubuntu Docker image) look like they match the failures I saw in CI: ``` ======== short test summary info ======== FAILED tests/test_table_api.py::test_paginate_tables_and_views[/fixtures/paginated_view.json-201-9] - assert 9 == 5 FAILED tests/test_table_api.py::test_sortable_columns_metadata - KeyError: 'error' FAILED tests/test_table_api.py::test_searchable_views[/fixtures/searchable_view_configured_by_metadata.json?_search=weasel-expected_rows0] - AssertionError: assert [[2, 'terry d...sel', 'puma']] == [[1, 'barry c...sel', 'puma']] FAILED tests/test_table_api.py::test_unit_filters - KeyError: 'distance' FAILED tests/test_table_api.py::test_custom_query_with_unicode_characters - AssertionError: assert {'error': 'Ta...'title': None} == [{'id': 1, 'n...n Francisco'}] FAILED tests/test_table_html.py::test_sort_links - AssertionError: assert [{'a_href': N...', ...}}, ...] == [{'a_href': N...', ...}}, ...] FAILED tests/test_table_html.py::test_table_html_no_primary_key - AttributeError: 'NoneType' object has no attribute 'strip' FAILED tests/test_table_html.py::test_table_html_foreign_key_custom_label_column - assert [['<td class=...1</em></td>']] == [['<td class=...>1</a></td>']] FAILED tests/test_table_html.py::test_view_html - assert <a href="/fixtures/simple_view?_size=3&_sort=upper_content" rel="nofollow">upper_content</a> is None FAILED tests/test_table_html.py::test_table_metadata - AssertionError: assert 'This <em&...t; is escaped' == 'simple_primary_key' FAILED tests/test_table_html.py::test_metadata_sort - AssertionError: assert ['id', 'name\xa0▼'] == ['id\xa0▼', 'name'] FAILED tests/test_table_html.py::test_metadata_sort_desc - AssertionError: assert ['pk\xa0▲', 'name'] == ['pk\xa0▼', 'name'] FAILED tests/test_table_html.py::test_column_metadata - AttributeError: 'NoneType' object has no attribute 'findAll' ======== 13 failed, 1279 passed, 3 skipped, 57 warnings in 572.40s (0:09:32) ======== ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1499150951 | |
https://github.com/simonw/datasette/pull/1960#issuecomment-1355478743 | https://api.github.com/repos/simonw/datasette/issues/1960 | 1355478743 | IC_kwDOBm6k_c5QyvbX | 9599 | 2022-12-16T19:27:12Z | 2022-12-16T19:27:12Z | OWNER | Bad news: they're definitely caused by tests that are subtly affected by other tests. This passes without errors: pytest -k test_paginate_tables_and_views | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1499150951 | |
https://github.com/simonw/datasette/pull/1960#issuecomment-1355685828 | https://api.github.com/repos/simonw/datasette/issues/1960 | 1355685828 | IC_kwDOBm6k_c5Qzh_E | 9599 | 2022-12-16T21:50:01Z | 2022-12-16T21:50:01Z | OWNER | Looks like that `@pytest.mark.ds_client` mark I've been using isn't necessary - I added that so I could easily run `pytest -m ds_client` to execute all tests that I had ported to the new feature, but actually this achieves the same thing: pytest -k ds_client So I'm going to remove the mark. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1499150951 | |
https://github.com/simonw/datasette/pull/1960#issuecomment-1355823260 | https://api.github.com/repos/simonw/datasette/issues/1960 | 1355823260 | IC_kwDOBm6k_c5Q0Dic | 9599 | 2022-12-16T23:36:07Z | 2022-12-16T23:36:07Z | OWNER | I ran `pytest --pdb -x` to drop into the debugger on the first failing test. ``` assert expected_rows == len(fetched) > assert expected_pages == count E assert 9 == 5 ``` That's in `test_paginate_tables_and_views()`. On a hunch, I checked the current settings: ``` (Pdb) ds_client.ds._settings {'default_page_size': 50, 'max_returned_rows': 100, 'max_insert_rows': 100, 'num_sql_threads': 1, 'sql_time_limit_ms': 200, 'default_facet_size': 30, 'facet_time_limit_ms': 200, 'facet_suggest_time_limit_ms': 50, 'allow_facet': True, 'allow_download': True, 'allow_signed_tokens': True, 'max_signed_tokens_ttl': 0, 'suggest_facets': True, 'default_cache_ttl': 5, 'cache_size_kb': 0, 'allow_csv_stream': True, 'max_csv_mb': 100, 'truncate_cells_html': 2048, 'force_https_urls': False, 'template_debug': False, 'trace_debug': False, 'base_url': '/'} ``` Looks like something changed `default_page_size` to 50 and forgot to change it back! | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1499150951 |