{"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1480349156", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1480349156, "node_id": "IC_kwDOBm6k_c5YPFXk", "user": {"value": 22429695, "label": "codecov[bot]"}, "created_at": "2023-03-22T22:40:58Z", "updated_at": "2023-03-22T22:48:57Z", "author_association": "NONE", "body": "## [Codecov](https://codecov.io/gh/simonw/datasette/pull/1999?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report\nPatch coverage: **`87.89`**% and project coverage change: **`-4.43`** :warning:\n> Comparison is base [(`56b0758`)](https://codecov.io/gh/simonw/datasette/commit/56b0758a5fbf85d01ff80a40c9b028469d7bb65f?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) 92.15% compared to head [(`921faae`)](https://codecov.io/gh/simonw/datasette/pull/1999?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) 87.73%.\n\n> :exclamation: Current head 921faae differs from pull request most recent head 69a31cd. Consider uploading reports for the commit 69a31cd to get more accurate results\n\n
Additional details and impacted files\n\n\n```diff\n@@ Coverage Diff @@\n## main #1999 +/- ##\n==========================================\n- Coverage 92.15% 87.73% -4.43% \n==========================================\n Files 38 38 \n Lines 5560 6066 +506 \n==========================================\n+ Hits 5124 5322 +198 \n- Misses 436 744 +308 \n```\n\n\n| [Impacted Files](https://codecov.io/gh/simonw/datasette/pull/1999?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage \u0394 | |\n|---|---|---|\n| [datasette/views/database.py](https://codecov.io/gh/simonw/datasette/pull/1999?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL3ZpZXdzL2RhdGFiYXNlLnB5) | `96.61% <\u00f8> (\u00f8)` | |\n| [datasette/views/row.py](https://codecov.io/gh/simonw/datasette/pull/1999?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL3ZpZXdzL3Jvdy5weQ==) | `87.82% <\u00f8> (\u00f8)` | |\n| [datasette/views/table.py](https://codecov.io/gh/simonw/datasette/pull/1999?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL3ZpZXdzL3RhYmxlLnB5) | `69.11% <86.76%> (-23.46%)` | :arrow_down: |\n| [datasette/renderer.py](https://codecov.io/gh/simonw/datasette/pull/1999?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL3JlbmRlcmVyLnB5) | `93.33% <90.90%> (-0.87%)` | :arrow_down: |\n| [datasette/views/base.py](https://codecov.io/gh/simonw/datasette/pull/1999?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL3ZpZXdzL2Jhc2UucHk=) | `92.78% <91.66%> (-2.39%)` | :arrow_down: |\n| [datasette/app.py](https://codecov.io/gh/simonw/datasette/pull/1999?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL2FwcC5weQ==) | `94.48% <100.00%> (-0.01%)` | :arrow_down: |\n| [datasette/cli.py](https://codecov.io/gh/simonw/datasette/pull/1999?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL2NsaS5weQ==) | `79.93% <100.00%> (\u00f8)` | |\n| [datasette/hookspecs.py](https://codecov.io/gh/simonw/datasette/pull/1999?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL2hvb2tzcGVjcy5weQ==) | `100.00% <100.00%> (\u00f8)` | |\n| [datasette/publish/cloudrun.py](https://codecov.io/gh/simonw/datasette/pull/1999?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL3B1Ymxpc2gvY2xvdWRydW4ucHk=) | `97.29% <100.00%> (\u00f8)` | |\n| [datasette/utils/\\_\\_init\\_\\_.py](https://codecov.io/gh/simonw/datasette/pull/1999?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL3V0aWxzL19faW5pdF9fLnB5) | `94.59% <100.00%> (-0.27%)` | :arrow_down: |\n\n... and [2 files with indirect coverage changes](https://codecov.io/gh/simonw/datasette/pull/1999/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)\n\nHelp us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)\n\n
\n\n[:umbrella: View full report in Codecov by Sentry](https://codecov.io/gh/simonw/datasette/pull/1999?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). \n:loudspeaker: Do you have feedback about the report comment? [Let us know in this issue](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1480343044", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1480343044, "node_id": "IC_kwDOBm6k_c5YPD4E", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-22T22:33:15Z", "updated_at": "2023-03-22T22:33:15Z", "author_association": "OWNER", "body": "This still needs documentation, but now the tests are passing I'm going to merge it into `main`!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1480339527", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1480339527, "node_id": "IC_kwDOBm6k_c5YPDBH", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-22T22:28:54Z", "updated_at": "2023-03-22T22:28:54Z", "author_association": "OWNER", "body": "I hacked at the CSV stuff until it worked.\r\n\r\nI need to clean it up though, but I can do that in this separate task:\r\n- #1101", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1480299765", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1480299765, "node_id": "IC_kwDOBm6k_c5YO5T1", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-22T21:44:03Z", "updated_at": "2023-03-22T21:45:10Z", "author_association": "OWNER", "body": "Oh this is a bit tricky.\r\n\r\nI have a failing test because a plugin that uses the `extra_css_urls` hook can't see the columns for the page.\r\n\r\nTurns out that bit comes from here:\r\n\r\nhttps://github.com/simonw/datasette/blob/56b0758a5fbf85d01ff80a40c9b028469d7bb65f/datasette/app.py#L1203-L1217\r\n\r\nWhich assumes the context has `\"columns\"` - but that's only now available if `?_extra=columns` was passed.\r\n\r\nActually I think I can cheat here, since it's still getting the HTML context in order to render the template.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1480179217", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1480179217, "node_id": "IC_kwDOBm6k_c5YOb4R", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-22T19:55:31Z", "updated_at": "2023-03-22T21:34:02Z", "author_association": "OWNER", "body": "I rebased from `main`. Now:\r\n\r\n```\r\nFAILED tests/test_csv.py::test_table_csv - assert 500 == 200\r\nFAILED tests/test_csv.py::test_table_csv_cors_headers - assert 500 == 200\r\nFAILED tests/test_csv.py::test_table_csv_no_header - assert 500 == 200\r\nFAILED tests/test_csv.py::test_table_csv_with_labels - assert 500 == 200\r\nFAILED tests/test_csv.py::test_table_csv_with_nullable_labels - assert 500 == 200\r\nFAILED tests/test_csv.py::test_table_csv_blob_columns - assert 500 == 200\r\nFAILED tests/test_csv.py::test_table_csv_download - assert 500 == 200\r\nFAILED tests/test_csv.py::test_table_csv_stream - assert 1 == 101\r\nFAILED tests/test_csv.py::test_csv_trace - AttributeError: 'NoneType' object has no attribute 'text'\r\nFAILED tests/test_plugins.py::test_hook_extra_css_urls[/fixtures/sortable-expected_decoded_object2] - AssertionError: assert {'added': 15,...ortable', ...} == {'added': 15,...ortable', ...}\r\nFAILED tests/test_plugins.py::test_hook_render_cell_demo - AttributeError: 'NoneType' object has no attribute 'string'\r\nFAILED tests/test_plugins.py::test_hook_render_cell_async[/fixtures/simple_primary_key] - assert b'RENDER_CELL_ASYNC_RESULT' in b'\\n\\n\\n Error 500\\n details.open = false);\\n});\\n\\n\\n\\n\\n\\n', ...} == {'1+1': 2, 'c... 0xXXX>', ...}\r\nFAILED tests/test_permissions.py::test_permissions_checked[/fixtures/simple_primary_key-permissions3] - assert 500 in (200, 403)\r\nFAILED tests/test_table_html.py::test_table_csv_json_export_interface - assert 500 == 200\r\nFAILED tests/test_table_html.py::test_table_metadata - assert 500 == 200\r\nFAILED tests/test_html.py::test_css_classes_on_body[/fixtures/simple_primary_key-expected_classes3] - assert 500 == 200\r\nFAILED tests/test_html.py::test_templates_considered[/fixtures/simple_primary_key-table-fixtures-simple_primary_key.html, *table.html] - assert 500 == 200\r\nFAILED tests/test_plugins.py::test_hook_table_actions[simple_view] - AssertionError: assert [] == [{'href': '/'...simple_view'}]\r\nERROR tests/test_internals_database.py::test_execute_write_fn_connection_exception - Failed: Timeout >1.0s\r\nERROR tests/test_internals_database.py::test_mtime_ns - Failed: Timeout >1.0s\r\nERROR tests/test_internals_database.py::test_mtime_ns_is_none_for_memory - Failed: Timeout >1.0s\r\nERROR tests/test_internals_database.py::test_is_mutable - Failed: Timeout >1.0s\r\nERROR tests/test_internals_database.py::test_database_memory_name - Failed: Timeout >1.0s\r\nERROR tests/test_internals_database.py::test_in_memory_databases_forbid_writes - Failed: Timeout >1.0s\r\n=============== 21 failed, 1275 passed, 2 skipped, 1 xfailed, 6 errors in 59.18s ================\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1480163485", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1480163485, "node_id": "IC_kwDOBm6k_c5YOYCd", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-22T19:48:00Z", "updated_at": "2023-03-22T19:48:00Z", "author_association": "OWNER", "body": "Getting close now! Only 13 failures left, mostly relating to CSV.\r\n```\r\nFAILED tests/test_csv.py::test_table_csv - assert 500 == 200\r\nFAILED tests/test_csv.py::test_table_csv_cors_headers - assert 500 == 200\r\nFAILED tests/test_csv.py::test_table_csv_no_header - assert 500 == 200\r\nFAILED tests/test_csv.py::test_table_csv_with_labels - assert 500 == 200\r\nFAILED tests/test_csv.py::test_table_csv_with_nullable_labels - assert 500 == 200\r\nFAILED tests/test_csv.py::test_table_csv_blob_columns - assert 500 == 200\r\nFAILED tests/test_csv.py::test_table_csv_download - assert 500 == 200\r\nFAILED tests/test_csv.py::test_table_csv_stream - assert 1 == 101\r\nFAILED tests/test_plugins.py::test_hook_extra_css_urls[/fixtures/sortable-expected_decoded_object2] - AssertionError: assert {'added': 15,...ortable', ...} == {'added': 15,...ortable', ...}\r\nFAILED tests/test_plugins.py::test_hook_register_facet_classes - KeyError: 'suggested_facets'\r\nFAILED tests/test_csv.py::test_csv_trace - AttributeError: 'NoneType' object has no attribute 'text'\r\nFAILED tests/test_plugins.py::test_hook_extra_body_script[/fixtures/sortable-expected_extra_body_script2] - AssertionError: assert {'added': 15,...ixtures', ...} == {'added': 15,...ixtures', ...}\r\nFAILED tests/test_plugins.py::test_hook_register_output_renderer_all_parameters - assert {'1+1': 2, 'c... 0xXXX>', ...} == {'1+1': 2, 'c... 0xXXX>', ...}\r\n=============== 13 failed, 1287 passed, 2 skipped, 1 xfailed in 61.57s (0:01:01) ================\r\n```\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1480154453", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1480154453, "node_id": "IC_kwDOBm6k_c5YOV1V", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-22T19:41:39Z", "updated_at": "2023-03-22T19:43:04Z", "author_association": "OWNER", "body": "To replace this code: https://github.com/simonw/datasette/blob/56b0758a5fbf85d01ff80a40c9b028469d7bb65f/datasette/views/base.py#L110-L122\r\n\r\nMaybe `datasette.render_template()` should optionally accept a list of templates.\r\n\r\nhttps://docs.datasette.io/en/stable/internals.html#await-render-template-template-context-none-request-none - turns out it does already:\r\n\r\n> If this is a list of template file names then the first one that exists will be loaded and rendered.\r\n\r\nIt doesn't have an easy way to populate that `select_templates` debug template variable though.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1477082852", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1477082852, "node_id": "IC_kwDOBm6k_c5YCn7k", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-20T23:27:25Z", "updated_at": "2023-03-20T23:27:25Z", "author_association": "OWNER", "body": "Urgh getting CSV to work is going to be _so hard_, because the logic for that currently lives in a huge chunk of code in `BaseView` which depends on the old design of the `data()` method:\r\n\r\nhttps://github.com/simonw/datasette/blob/4bb49848697e40b8b9a1557be42b8e59eac965b3/datasette/views/base.py#L177-L343", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1476860334", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1476860334, "node_id": "IC_kwDOBm6k_c5YBxmu", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-20T20:06:56Z", "updated_at": "2023-03-20T22:09:03Z", "author_association": "OWNER", "body": "`pytest -n 8` locally says:\r\n\r\n```\r\nFAILED tests/test_canned_queries.py::test_canned_query_form_csrf_hidden_field[add_name_specify_id-True] - assert '\\n\\n\\n Error 500\\n details.open = false);\\n});\\n\\n\\n\\n Error 500\\n details.open = false);\\n});\\n' in '\\n\\n\\n Error 500\\n details.open = false);\\...\r\nFAILED tests/test_canned_queries.py::test_insert - KeyError: 'ds_csrftoken'\r\nFAILED tests/test_canned_queries.py::test_insert_error - KeyError: 'ds_csrftoken'\r\nFAILED tests/test_canned_queries.py::test_magic_parameters_csrf_json[False-True] - KeyError: 'ds_csrftoken'\r\nFAILED tests/test_canned_queries.py::test_magic_parameters_csrf_json[True-True] - KeyError: 'ds_csrftoken'\r\nFAILED tests/test_canned_queries.py::test_vary_header - KeyError: 'vary'\r\nFAILED tests/test_csv.py::test_csv_trace - AssertionError: assert 'application/...charset=utf-8' == 'text/html; charset=utf-8'\r\nFAILED tests/test_csv.py::test_table_csv - AssertionError: assert 'application/...charset=utf-8' == 'text/plain; charset=utf-8'\r\nFAILED tests/test_csv.py::test_table_csv_blob_columns - AssertionError: assert 'application/...charset=utf-8' == 'text/plain; charset=utf-8'\r\nFAILED tests/test_csv.py::test_table_csv_download - AssertionError: assert 'application/...charset=utf-8' == 'text/csv; charset=utf-8'\r\nFAILED tests/test_csv.py::test_table_csv_no_header - AssertionError: assert 'application/...charset=utf-8' == 'text/plain; charset=utf-8'\r\nFAILED tests/test_csv.py::test_table_csv_stream - assert 1 == 101\r\nFAILED tests/test_csv.py::test_table_csv_stream_does_not_calculate_counts - AttributeError: 'NoneType' object has no attribute 'text'\r\nFAILED tests/test_csv.py::test_table_csv_stream_does_not_calculate_facets - AttributeError: 'NoneType' object has no attribute 'text'\r\nFAILED tests/test_csv.py::test_table_csv_with_labels - AssertionError: assert 'application/...charset=utf-8' == 'text/plain; charset=utf-8'\r\nFAILED tests/test_csv.py::test_table_csv_with_nullable_labels - AssertionError: assert 'application/...charset=utf-8' == 'text/plain; charset=utf-8'\r\nFAILED tests/test_facets.py::test_array_facet_handle_duplicate_tags - KeyError: 'facet_results'\r\nFAILED tests/test_facets.py::test_conflicting_facet_names_json - AssertionError: assert {'results', 'timed_out'} == {'created', '...gs', 'tags_2'}\r\nFAILED tests/test_facets.py::test_facet_size - KeyError: 'suggested_facets'\r\nFAILED tests/test_facets.py::test_json_array_with_blanks_and_nulls - KeyError: 'suggested_facets'\r\nFAILED tests/test_facets.py::test_other_types_of_facet_in_metadata - assert 'created (date)\\n' in '\\n\\n\\n fixtures: facetable: 15 rows\\n \\n\\n\\n\\n' in '\\n\\n\\n fixtures: simple_primary_key: 5 rows\\n ' in '\\n\\n\\n fixtures: table/with/slashes.csv: 1 row\\n ', ...}\r\nFAILED tests/test_plugins.py::test_hook_register_output_renderer_no_parameters - assert b'Hello' == b'{\"ok\": true... \"n\": null}]}'\r\nFAILED tests/test_plugins.py::test_hook_register_output_renderer_returning_broken_value - assert 200 == 500\r\nFAILED tests/test_plugins.py::test_hook_register_output_renderer_returning_response - assert {'next': None...', ...}, ...]} == {'this_is': 'json'}\r\n=== 39 failed, 1259 passed, 2 skipped, 1 xfailed in 58.07s ===\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1476898261", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1476898261, "node_id": "IC_kwDOBm6k_c5YB63V", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-20T20:37:52Z", "updated_at": "2023-03-20T20:37:52Z", "author_association": "OWNER", "body": "Manual testing spotted a bug.\r\n\r\n`/content/repos.json?owner=9599&_facet_array=topics` - does not return a `facet_results` key.\r\n\r\n`/content/repos.json?owner=9599&_facet_array=topics&_facet=owner` does.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1476854645", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1476854645, "node_id": "IC_kwDOBm6k_c5YBwN1", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-20T20:02:24Z", "updated_at": "2023-03-20T20:02:24Z", "author_association": "OWNER", "body": "```\r\n def test_routes(routes, path, expected_class, expected_matches):\r\n match, view = resolve_routes(routes, path)\r\n if expected_class is None:\r\n assert match is None\r\n else:\r\n> assert view.view_class.__name__ == expected_class\r\nE AttributeError: 'function' object has no attribute 'view_class'\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1476851525", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1476851525, "node_id": "IC_kwDOBm6k_c5YBvdF", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-20T19:59:51Z", "updated_at": "2023-03-20T20:01:40Z", "author_association": "OWNER", "body": "Three failures in `test_html.py`:\r\n```\r\nFAILED tests/test_html.py::test_templates_considered[/fixtures/simple_primary_key-table-fixtures-simple_primary_key.html, *table.html] - assert '' in '\\n\\n\\n fixtures: simple_primary_key: 5 rows\\n ' in '\\n\\n\\n fixtures: table/with/slashes.csv: 1 row\\n \\xa0']] == [['\\xa0']]\r\nFAILED tests/test_table_html.py::test_table_html_disable_foreign_key_links_with_labels - assert [['b']] == [['b']]\r\nFAILED tests/test_table_html.py::test_table_html_foreign_key_custom_label_column - assert [['']] == [['']]\r\nFAILED tests/test_table_html.py::test_table_html_filter_form_column_options[/fixtures/infinity-expected_column_options0] - AssertionError: assert ['- column -'...wid', 'value'] == ['- column -']\r\nFAILED tests/test_table_html.py::test_table_html_filter_form_column_options[/fixtures/primary_key_multiple_columns-expected_column_options1] - AssertionError: assert ['- column -'...', 'content2'] == ['- column -']\r\nFAILED tests/test_table_html.py::test_table_html_filter_form_column_options[/fixtures/compound_primary_key-expected_column_options2] - AssertionError: assert ['- column -'...2', 'content'] == ['- column -']\r\nFAILED tests/test_table_html.py::test_table_html_filter_form_still_shows_nocol_columns - AssertionError: assert ['- column -'] == ['- column -'...nulls_2', ...]\r\nFAILED tests/test_table_html.py::test_compound_primary_key_with_foreign_key_references - assert [['']] == [['']]\r\nFAILED tests/test_table_html.py::test_view_html - assert None is not None\r\nFAILED tests/test_table_html.py::test_extra_where_clauses - assert [('_where', \"...'_city_id=1')] == []\r\nFAILED tests/test_table_html.py::test_other_hidden_form_fields[/fixtures/facetable?_size=10-expected_hidden0] - AssertionError: assert [] == [('_size', '10')]\r\nFAILED tests/test_table_html.py::test_other_hidden_form_fields[/fixtures/facetable?_size=10&_ignore=1&_ignore=2-expected_hidden1] - AssertionError: assert [] == [('_size', '1...ignore', '2')]\r\nFAILED tests/test_table_html.py::test_search_and_sort_fields_not_duplicated[/fixtures/searchable?_sort=text2&_where=1-expected_hidden3] - AssertionError: assert [] == [('_where', '1')]\r\nFAILED tests/test_table_html.py::test_binary_data_display_in_table - assert [['\\xa0']] == [['\\xa0']]\r\nFAILED tests/test_table_html.py::test_metadata_sort - AttributeError: 'NoneType' object has no attribute 'string'\r\nFAILED tests/test_table_html.py::test_metadata_sort_desc - AttributeError: 'NoneType' object has no attribute 'string'\r\nFAILED tests/test_table_html.py::test_facet_more_links[5-/fixtures/facetable?_facet=_neighborhood-2-True-/fixtures/facetable?_facet=_neighborhood&_facet_size=max] - assert 0 == 2\r\nFAILED tests/test_table_html.py::test_facet_more_links[5-/fixtures/facetable?_facet=_neighborhood&_facet_size=50-5-True-/fixtures/facetable?_facet=_neighborhood&_facet_size=max] - assert 0 == 5\r\nFAILED tests/test_table_html.py::test_facet_total - assert 500 == 200\r\n```\r\n\r\nDeduped that's 30 tests:\r\n\r\n```\r\nFAILED tests/test_table_html.py::test_add_filter_redirects\r\nFAILED tests/test_table_html.py::test_binary_data_display_in_table\r\nFAILED tests/test_table_html.py::test_compound_primary_key_with_foreign_key_references\r\nFAILED tests/test_table_html.py::test_csv_json_export_links_include_labels_if_foreign_keys\r\nFAILED tests/test_table_html.py::test_empty_search_parameter_gets_removed\r\nFAILED tests/test_table_html.py::test_existing_filter_redirects\r\nFAILED tests/test_table_html.py::test_extra_where_clauses\r\nFAILED tests/test_table_html.py::test_facet_more_links\r\nFAILED tests/test_table_html.py::test_facet_total\r\nFAILED tests/test_table_html.py::test_facets_persist_through_filter_form\r\nFAILED tests/test_table_html.py::test_metadata_sort\r\nFAILED tests/test_table_html.py::test_metadata_sort_desc\r\nFAILED tests/test_table_html.py::test_next_does_not_persist_in_hidden_field\r\nFAILED tests/test_table_html.py::test_other_hidden_form_fields\r\nFAILED tests/test_table_html.py::test_reflected_hidden_form_fields\r\nFAILED tests/test_table_html.py::test_rowid_sortable_no_primary_key\r\nFAILED tests/test_table_html.py::test_search_and_sort_fields_not_duplicated\r\nFAILED tests/test_table_html.py::test_searchable_view_persists_fts_table\r\nFAILED tests/test_table_html.py::test_sort_by_desc_redirects\r\nFAILED tests/test_table_html.py::test_sort_links\r\nFAILED tests/test_table_html.py::test_table_csv_json_export_interface\r\nFAILED tests/test_table_html.py::test_table_html_compound_primary_key\r\nFAILED tests/test_table_html.py::test_table_html_disable_foreign_key_links_with_labels\r\nFAILED tests/test_table_html.py::test_table_html_filter_form_column_options\r\nFAILED tests/test_table_html.py::test_table_html_filter_form_still_shows_nocol_columns\r\nFAILED tests/test_table_html.py::test_table_html_foreign_key_custom_label_column\r\nFAILED tests/test_table_html.py::test_table_html_foreign_key_links\r\nFAILED tests/test_table_html.py::test_table_html_no_primary_key\r\nFAILED tests/test_table_html.py::test_table_html_simple_primary_key\r\nFAILED tests/test_table_html.py::test_view_html\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1475074025", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1475074025, "node_id": "IC_kwDOBm6k_c5X69fp", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-19T02:14:28Z", "updated_at": "2023-03-19T02:14:51Z", "author_association": "OWNER", "body": "I had to replicate quite a bit of this logic from `base.py`:\r\n\r\nhttps://github.com/simonw/datasette/blob/56b0758a5fbf85d01ff80a40c9b028469d7bb65f/datasette/views/base.py#L526-L544\r\n\r\nI should refactor this when I move the canned / arbitrary query views away from that base class too.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1475016834", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1475016834, "node_id": "IC_kwDOBm6k_c5X6viC", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-18T22:30:31Z", "updated_at": "2023-03-18T22:30:31Z", "author_association": "OWNER", "body": "`test_paginate_using_link_header` will be tricky.\r\n\r\nThe reason the tests are failing is that `json_renderer()` attempts to populate the `link` header using `data[\"next_url\"]` - but that's not present unless `?_extra=next_url` has been passed:\r\n\r\nhttps://github.com/simonw/datasette/blob/2f38479dcc81f11a4362f4e28511fa88afc34e61/datasette/renderer.py#L101-L102\r\n\r\nBut I can only rely on `next` being present, not `next_url`.\r\n\r\nI thought I could maybe assemble the `link` header using `next`, by turning that into a `next_url` link - but there's some custom logic which kicks in for pagination against SQL views (offset/limit pagination) to calculate the `next_url` which isn't easily replicable at the `json_renderer()` layer, in the `_next_value_and_url()` utility function:\r\n\r\nhttps://github.com/simonw/datasette/blob/2f38479dcc81f11a4362f4e28511fa88afc34e61/datasette/views/table.py#L2275-L2282\r\n\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1475003292", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1475003292, "node_id": "IC_kwDOBm6k_c5X6sOc", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-18T21:46:20Z", "updated_at": "2023-03-18T21:46:20Z", "author_association": "OWNER", "body": "Now 25 failures in `test_table_api.py`:\r\n```\r\nFAILED tests/test_table_api.py::test_expand_labels - assert {'2': {'pk': 2, 'created': '2019-01-14 08:00:00', 'planet_int': 1, 'on_earth': 1, 'state': 'CA', '_city_id': 1, '_neighborhood': 'Dogpatch', 'tags': '[\"tag1\", \"tag3\"]', 'complex_array'...\r\nFAILED tests/test_table_api.py::test_expand_label - AssertionError: assert {'1': {'pk': '1', 'foreign_key_with_label': '1', 'foreign_key_with_blank_label': '3', 'foreign_key_with_no_label': '1', 'foreign_key_compound_pk1': 'a', 'foreign_key_co...\r\nFAILED tests/test_table_api.py::test_ttl_parameter[/fixtures/facetable.json-max-age=5] - KeyError: 'Cache-Control'\r\nFAILED tests/test_table_api.py::test_ttl_parameter[/fixtures/facetable.json?_ttl=invalid-max-age=5] - KeyError: 'Cache-Control'\r\nFAILED tests/test_table_api.py::test_ttl_parameter[/fixtures/facetable.json?_ttl=10-max-age=10] - KeyError: 'Cache-Control'\r\nFAILED tests/test_table_api.py::test_ttl_parameter[/fixtures/facetable.json?_ttl=0-no-cache] - KeyError: 'Cache-Control'\r\nFAILED tests/test_table_api.py::test_infinity_returned_as_null - AssertionError: assert [{'rowid': 1, 'value': inf}, {'rowid': 2, 'value': -inf}, {'rowid': 3, 'value': 1.5}] == [{'rowid': 1, 'value': None}, {'rowid': 2, 'value': None}, {'rowid': 3, 'value'...\r\nFAILED tests/test_table_api.py::test_null_and_compound_foreign_keys_are_not_expanded - AssertionError: assert [{'pk': '1', 'foreign_key_with_label': '1', 'foreign_key_with_blank_label': '3', 'foreign_key_with_no_label': '1', 'foreign_key_compound_pk1': 'a', 'foreign_key_compoun...\r\nFAILED tests/test_table_api.py::test_binary_data_in_json[/fixtures/binary_data.json?_shape=array-expected_json0-None] - assert [{'rowid': 1, 'data': \"b'\\\\x15\\\\x1c\\\\x02\\\\xc7\\\\xad\\\\x05\\\\xfe'\"}, {'rowid': 2, 'data': \"b'\\\\x15\\\\x1c\\\\x03\\\\xc7\\\\xad\\\\x05\\\\xfe'\"}, {'rowid': 3, 'data': None}] == [{'rowid': 1, 'data': {'...\r\nFAILED tests/test_table_api.py::test_binary_data_in_json[/fixtures/binary_data.json?_shape=array&_nl=on-None-{\"rowid\": 1, \"data\": {\"$base64\": true, \"encoded\": \"FRwCx60F/g==\"}}\\n{\"rowid\": 2, \"data\": {\"$base64\": true, \"encoded\": \"FRwDx60F/g==\"}}\\n{\"rowid\": 3, \"data\": null}] - assert '{\"ok\": false, \"error\": \"Object of type bytes is not JSON serializable\", \"status\": 500, \"title\": null}' == '{\"rowid\": 1, \"data\": {\"$base64\": true, \"encoded\": \"FRwCx60F/g==\"}}\\n{\"rowid\"...\r\nFAILED tests/test_table_api.py::test_paginate_using_link_header[] - assert 1 == 21\r\nFAILED tests/test_table_api.py::test_paginate_using_link_header[?_shape=arrays] - assert 1 == 21\r\nFAILED tests/test_table_api.py::test_paginate_using_link_header[?_shape=arrayfirst] - assert 400 == 200\r\nFAILED tests/test_table_api.py::test_paginate_using_link_header[?_shape=object] - assert 1 == 21\r\nFAILED tests/test_table_api.py::test_paginate_using_link_header[?_shape=objects] - assert 1 == 21\r\nFAILED tests/test_table_api.py::test_paginate_using_link_header[?_shape=array] - assert 1 == 21\r\nFAILED tests/test_table_api.py::test_paginate_using_link_header[?_shape=array&_nl=on] - assert 1 == 21\r\nFAILED tests/test_table_api.py::test_col_nocol[/fixtures/facetable.json?_col=created-expected_columns0] - KeyError: 'columns'\r\nFAILED tests/test_table_api.py::test_col_nocol[/fixtures/facetable.json?_nocol=created-expected_columns1] - KeyError: 'columns'\r\nFAILED tests/test_table_api.py::test_col_nocol[/fixtures/facetable.json?_col=state&_col=created-expected_columns2] - KeyError: 'columns'\r\nFAILED tests/test_table_api.py::test_col_nocol[/fixtures/facetable.json?_col=state&_col=state-expected_columns3] - KeyError: 'columns'\r\nFAILED tests/test_table_api.py::test_col_nocol[/fixtures/facetable.json?_col=state&_col=created&_nocol=created-expected_columns4] - KeyError: 'columns'\r\nFAILED tests/test_table_api.py::test_col_nocol[/fixtures/facetable.json?_nocol=state&_facet=state-expected_columns5] - KeyError: 'columns'\r\nFAILED tests/test_table_api.py::test_col_nocol[/fixtures/simple_view.json?_nocol=content-expected_columns6] - KeyError: 'columns'\r\nFAILED tests/test_table_api.py::test_col_nocol[/fixtures/simple_view.json?_col=content-expected_columns7] - KeyError: 'columns'\r\n============================================================================= 25 failed, 86 passed, 1 xfailed in 7.18s =============================================================================\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1474704790", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1474704790, "node_id": "IC_kwDOBm6k_c5X5jWW", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-18T04:52:59Z", "updated_at": "2023-03-18T04:52:59Z", "author_association": "OWNER", "body": "Here are the next set of tests to get passing:\r\n```\r\n% pytest tests/test_table_api.py\r\n```\r\n```\r\nFAILED tests/test_table_api.py::test_facets[/fixtures/facetable.json?_facet=state&_facet=_city_id-expected_facet_results0] - KeyError: 'name'\r\nFAILED tests/test_table_api.py::test_facets[/fixtures/facetable.json?_facet=state&_facet=_city_id&state=MI-expected_facet_results1] - KeyError: 'name'\r\nFAILED tests/test_table_api.py::test_facets[/fixtures/facetable.json?_facet=planet_int-expected_facet_results2] - KeyError: 'name'\r\nFAILED tests/test_table_api.py::test_facets[/fixtures/facetable.json?_facet=planet_int&planet_int=1-expected_facet_results3] - KeyError: 'name'\r\nFAILED tests/test_table_api.py::test_suggested_facets - KeyError: 'suggested_facets'\r\nFAILED tests/test_table_api.py::test_allow_facet_off - KeyError: 'suggested_facets'\r\nFAILED tests/test_table_api.py::test_suggest_facets_off - KeyError: 'suggested_facets'\r\nFAILED tests/test_table_api.py::test_nofacet[True] - KeyError: 'suggested_facets'\r\nFAILED tests/test_table_api.py::test_nofacet[False] - KeyError: 'suggested_facets'\r\nFAILED tests/test_table_api.py::test_nosuggest[True] - KeyError: 'suggested_facets'\r\nFAILED tests/test_table_api.py::test_nosuggest[False] - KeyError: 'suggested_facets'\r\nFAILED tests/test_table_api.py::test_nocount[True-None] - KeyError: 'count'\r\nFAILED tests/test_table_api.py::test_nocount[False-15] - KeyError: 'count'\r\nFAILED tests/test_table_api.py::test_expand_labels - AssertionError: assert {'13': {'_cit...:00:00', ...}} == {'13': {'_cit...:00:00', ...}}\r\nFAILED tests/test_table_api.py::test_expand_label - AssertionError: assert {'1': {'forei...l': '1', ...}} == {'1': {'forei...': '1'}, ...}}\r\nFAILED tests/test_table_api.py::test_ttl_parameter[/fixtures/facetable.json-max-age=5] - KeyError: 'Cache-Control'\r\nFAILED tests/test_table_api.py::test_ttl_parameter[/fixtures/facetable.json?_ttl=invalid-max-age=5] - KeyError: 'Cache-Control'\r\nFAILED tests/test_table_api.py::test_ttl_parameter[/fixtures/facetable.json?_ttl=10-max-age=10] - KeyError: 'Cache-Control'\r\nFAILED tests/test_table_api.py::test_ttl_parameter[/fixtures/facetable.json?_ttl=0-no-cache] - KeyError: 'Cache-Control'\r\nFAILED tests/test_table_api.py::test_infinity_returned_as_null - AssertionError: assert [{'rowid': 1,...'value': 1.5}] == [{'rowid': 1,...'value': 1.5}]\r\nFAILED tests/test_table_api.py::test_null_and_compound_foreign_keys_are_not_expanded - AssertionError: assert [{'foreign_ke...': None, ...}] == [{'foreign_ke...': None, ...}]\r\nFAILED tests/test_table_api.py::test_binary_data_in_json[/fixtures/binary_data.json?_shape=array-expected_json0-None] - assert [{'data': \"b'..., 'rowid': 3}] == [{'data': {'$..., 'rowid': 3}]\r\nFAILED tests/test_table_api.py::test_binary_data_in_json[/fixtures/binary_data.json?_shape=array&_nl=on-None-{\"rowid\": 1, \"data\": {\"$base64\": true, \"encoded\": \"FRwCx60F/g==\"}}\\n{\"rowid\": 2, \"data\": {\"$base64\": true, \"encoded\": \"FRwDx60F/g==\"}}\\n{\"rowid\": 3, \"data\": null}] - assert '{\"ok\": false...title\": null}' == '{\"rowid\": 1,...\"data\": null}'\r\nFAILED tests/test_table_api.py::test_paginate_using_link_header[] - assert 1 == 21\r\nFAILED tests/test_table_api.py::test_paginate_using_link_header[?_shape=arrays] - assert 1 == 21\r\nFAILED tests/test_table_api.py::test_paginate_using_link_header[?_shape=arrayfirst] - assert 400 == 200\r\nFAILED tests/test_table_api.py::test_paginate_using_link_header[?_shape=object] - assert 1 == 21\r\nFAILED tests/test_table_api.py::test_paginate_using_link_header[?_shape=objects] - assert 1 == 21\r\nFAILED tests/test_table_api.py::test_paginate_using_link_header[?_shape=array] - assert 1 == 21\r\nFAILED tests/test_table_api.py::test_paginate_using_link_header[?_shape=array&_nl=on] - assert 1 == 21\r\nFAILED tests/test_table_api.py::test_col_nocol[/fixtures/facetable.json?_col=created-expected_columns0] - KeyError: 'columns'\r\nFAILED tests/test_table_api.py::test_col_nocol[/fixtures/facetable.json?_nocol=created-expected_columns1] - KeyError: 'columns'\r\nFAILED tests/test_table_api.py::test_col_nocol[/fixtures/facetable.json?_col=state&_col=created-expected_columns2] - KeyError: 'columns'\r\nFAILED tests/test_table_api.py::test_col_nocol[/fixtures/facetable.json?_col=state&_col=state-expected_columns3] - KeyError: 'columns'\r\nFAILED tests/test_table_api.py::test_col_nocol[/fixtures/facetable.json?_col=state&_col=created&_nocol=created-expected_columns4] - KeyError: 'columns'\r\nFAILED tests/test_table_api.py::test_col_nocol[/fixtures/facetable.json?_nocol=state&_facet=state-expected_columns5] - KeyError: 'columns'\r\nFAILED tests/test_table_api.py::test_col_nocol[/fixtures/simple_view.json?_nocol=content-expected_columns6] - KeyError: 'columns'\r\nFAILED tests/test_table_api.py::test_col_nocol[/fixtures/simple_view.json?_col=content-expected_columns7] - KeyError: 'columns'\r\n============================================================================= 38 failed, 73 passed, 1 xfailed in 7.25s =============================================================================\r\n```\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1463113856", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1463113856, "node_id": "IC_kwDOBm6k_c5XNViA", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-10T02:13:15Z", "updated_at": "2023-03-10T02:13:15Z", "author_association": "OWNER", "body": "Idea for if this change ends up making a bunch of breaking changes to the templates (which I think it should) - I can generate a GitHub diff link between the old and new templates and include that link in the 1.0 upgrade documentation to help people who wrote custom templates see what they might need to change - with minimal effort from myself to document those changes.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1463024951", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1463024951, "node_id": "IC_kwDOBm6k_c5XM_03", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-10T00:17:58Z", "updated_at": "2023-03-10T00:17:58Z", "author_association": "OWNER", "body": "Renderers have an impact on three different pages: query results, table page and row page.\r\n\r\nThe row page feature is incomplete though:\r\n\r\nhttps://congress-legislators.datasettes.com/legislators/social_media/A000055\r\n\r\n\"image\"\r\n\r\nWhy is there no `.csv` link there?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1463023674", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1463023674, "node_id": "IC_kwDOBm6k_c5XM_g6", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-10T00:16:03Z", "updated_at": "2023-03-10T00:16:03Z", "author_association": "OWNER", "body": "I also need to figure out the `renderers` stuff, so I can link to the right URLs for CSV and JSON and other formats:\r\n\r\nhttps://github.com/simonw/datasette/blob/6d07a7da1531cd749844fc6827d9a1e57009b2ea/datasette/views/base.py#L474-L518", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1463022397", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1463022397, "node_id": "IC_kwDOBm6k_c5XM_M9", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-10T00:14:21Z", "updated_at": "2023-03-10T00:14:21Z", "author_association": "OWNER", "body": "Ironically the thing I most need right now is comprehensive documentation of what variables are being passed to the templates!\r\n\r\nOne big challenge is that I need to untangle the template context that happens in `BaseView` - I'm hacking that together at the moment, but I need a real answer for how that should work in a world in which view functions aren't using that base class.\r\n\r\nhttps://github.com/simonw/datasette/blob/56b0758a5fbf85d01ff80a40c9b028469d7bb65f/datasette/views/base.py#L110-L145", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1463021383", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1463021383, "node_id": "IC_kwDOBm6k_c5XM-9H", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-10T00:12:50Z", "updated_at": "2023-03-10T00:12:50Z", "author_association": "OWNER", "body": "Now at 34 failed, 34 passed.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1463005744", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1463005744, "node_id": "IC_kwDOBm6k_c5XM7Iw", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-09T23:52:15Z", "updated_at": "2023-03-09T23:52:23Z", "author_association": "OWNER", "body": "I need to figure out what to do about `extra_context_from_filters` - which was previously passed straight to the HTML context.\r\n\r\nhttps://github.com/simonw/datasette/blob/11f7feb7a3f7166c71389786880863d60ed3d165/datasette/views/table.py#L406-L422", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1462997800", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1462997800, "node_id": "IC_kwDOBm6k_c5XM5Mo", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-09T23:39:47Z", "updated_at": "2023-03-09T23:39:47Z", "author_association": "OWNER", "body": "Found a neat trick:\r\n```diff\r\ndiff --git a/datasette/app.py b/datasette/app.py\r\nindex 186f192d..40416713 100644\r\n--- a/datasette/app.py\r\n+++ b/datasette/app.py\r\n@@ -23,7 +23,13 @@ from pathlib import Path\r\n \r\n from markupsafe import Markup, escape\r\n from itsdangerous import URLSafeSerializer\r\n-from jinja2 import ChoiceLoader, Environment, FileSystemLoader, PrefixLoader\r\n+from jinja2 import (\r\n+ ChoiceLoader,\r\n+ Environment,\r\n+ FileSystemLoader,\r\n+ PrefixLoader,\r\n+ StrictUndefined,\r\n+)\r\n from jinja2.environment import Template\r\n from jinja2.exceptions import TemplateNotFound\r\n \r\n@@ -394,7 +400,10 @@ class Datasette:\r\n ]\r\n )\r\n self.jinja_env = Environment(\r\n- loader=template_loader, autoescape=True, enable_async=True\r\n+ loader=template_loader,\r\n+ autoescape=True,\r\n+ enable_async=True,\r\n+ undefined=StrictUndefined,\r\n )\r\n self.jinja_env.filters[\"escape_css_string\"] = escape_css_string\r\n self.jinja_env.filters[\"quote_plus\"] = urllib.parse.quote_plus\r\n```\r\nThis causes Jinja to raise a hard error if there are any variables referenced in the template that are not available in the context.\r\n\r\nIt's helping me spot things that are still missing, rather than just relying on failed unit tests.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1461161256", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1461161256, "node_id": "IC_kwDOBm6k_c5XF40o", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-09T02:10:07Z", "updated_at": "2023-03-09T02:10:07Z", "author_association": "OWNER", "body": "Just ran into a `no such table: columns` error - which I think is because my new view code sometimes bypasses calling this method (currently done in `BaseView.dispatch_request()`):\r\n\r\nhttps://github.com/simonw/datasette/blob/96e94f9b7b2db53865e61390bcce6761727f26d8/datasette/views/base.py#L101-L103", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1461148579", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1461148579, "node_id": "IC_kwDOBm6k_c5XF1uj", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-09T01:54:10Z", "updated_at": "2023-03-09T01:55:33Z", "author_association": "OWNER", "body": "Or... I could temporarily build a quick additional `CannedQueryView` subclass that just does the necessary bits to get the existing code to work. I'm going to try that.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1461148254", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1461148254, "node_id": "IC_kwDOBm6k_c5XF1pe", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-09T01:53:41Z", "updated_at": "2023-03-09T01:53:41Z", "author_association": "OWNER", "body": "Solving this is proving difficult: https://github.com/simonw/datasette/blob/96e94f9b7b2db53865e61390bcce6761727f26d8/datasette/views/table.py#L1500-L1503\r\n\r\nThe problem is that calling `.data()` on `QueryView` only works here because we expect to ourselves be inside a `.data()` method, with all of the existing magic that knows how to render things that are returned by that.\r\n\r\nSo I may need to substantially re-engineer how `QueryView` works in order to get this to work.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1461075648", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1461075648, "node_id": "IC_kwDOBm6k_c5XFj7A", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-09T00:24:22Z", "updated_at": "2023-03-09T00:24:22Z", "author_association": "OWNER", "body": "`127.0.0.1:8001/fixtures/neighborhood_search` fails because the forwarding to a canned query does not yet work.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1461074526", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1461074526, "node_id": "IC_kwDOBm6k_c5XFjpe", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-09T00:23:06Z", "updated_at": "2023-03-09T00:23:06Z", "author_association": "OWNER", "body": " pytest tests/test_table_html.py\r\n\r\nCurrently 44 failed, 24 passed in 7.53s\r\n\r\nFailures here: https://gist.github.com/simonw/df0a52cd7d820b776dc3dfc50e7cb778", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1461070937", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1461070937, "node_id": "IC_kwDOBm6k_c5XFixZ", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-09T00:18:52Z", "updated_at": "2023-03-09T00:19:36Z", "author_association": "OWNER", "body": "I managed to get HTML view working! I did it by continuing to add more things to the extras and the `_html` bundle until the page loaded for me:\r\n\r\n```diff\r\n async def extra_extras():\r\n \"Available ?_extra= blocks\"\r\n return {\r\n@@ -1981,6 +2053,14 @@ async def extra_extras():\r\n \"query\",\r\n \"display_columns\",\r\n \"display_rows\",\r\n+ \"database\",\r\n+ \"table\",\r\n+ \"database_color\",\r\n+ \"table_actions\",\r\n+ \"filters\",\r\n+ \"renderers\",\r\n+ \"custom_table_templates\",\r\n+ \"sorted_facet_results\",\r\n ]\r\n }\r\n \r\n@@ -2006,6 +2086,14 @@ async def extra_extras():\r\n extra_query,\r\n extra_metadata,\r\n extra_extras,\r\n+ extra_database,\r\n+ extra_table,\r\n+ extra_database_color,\r\n+ extra_table_actions,\r\n+ extra_filters,\r\n+ extra_renderers,\r\n+ extra_custom_table_templates,\r\n+ extra_sorted_facet_results,\r\n )\r\n```\r\nI'll probably refactor this into something cleaner, and maybe but a bunch of them in a `\"html\"` dictionary and update the templates to use `{{ html.filters }}` or similar. Will look at that once the tests are passing.\r\n\r\n\"image\"\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1461047607", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1461047607, "node_id": "IC_kwDOBm6k_c5XFdE3", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T23:51:46Z", "updated_at": "2023-03-08T23:51:46Z", "author_association": "OWNER", "body": "This feels quite nice:\r\n\r\n\"image\"\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1461044477", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1461044477, "node_id": "IC_kwDOBm6k_c5XFcT9", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T23:47:26Z", "updated_at": "2023-03-08T23:47:26Z", "author_association": "OWNER", "body": "I want to package together all of the extras that are needed for the HTML format. A few options for doing that:\r\n\r\n- Introduce `?_extra=_html` where the leading underscore indicates that this is a \"bundle\" of extras, then define a bundle that's everything needed for the HTML renderer\r\n- Have some other mechanism whereby different renderers can request a bundle of extras.\r\n\r\nI'm leaning towards the first option. I'll try that and see what it looks like.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1461023559", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1461023559, "node_id": "IC_kwDOBm6k_c5XFXNH", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T23:23:02Z", "updated_at": "2023-03-08T23:23:02Z", "author_association": "OWNER", "body": "To get this unblocked, I'm going to allow myself to pass non-JSON-serializable objects to the HTML template version of things. If I can get that working (and get the existing tests to pass) I can consider a later change that makes those JSON serializable - or admit that it's OK for the templates to have non-JSON data passed to them and figure out how best to document those variables independently from the JSON documentation.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1461002039", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1461002039, "node_id": "IC_kwDOBm6k_c5XFR83", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T22:58:16Z", "updated_at": "2023-03-08T23:02:09Z", "author_association": "OWNER", "body": "The reason for that `Row` thing is that it allows custom templates that do things like this:\r\n\r\nhttps://docs.datasette.io/en/stable/changelog.html#easier-custom-templates-for-table-rows\r\n\r\n```html+jinja\r\n{% for row in display_rows %}\r\n
\r\n

{{ row[\"title\"] }}

\r\n

{{ row[\"description\"] }}\r\n

Category: {{ row.display(\"category_id\") }}

\r\n
\r\n{% endfor %}\r\n```\r\nIs that a good design? the `.display()` thing feels weird - I wonder if anyone has ever actually used that.\r\n\r\nIt's documented here: https://docs.datasette.io/en/0.64.2/custom_templates.html#custom-templates\r\n\r\n> If you want to output the rendered HTML version of a column, including any links to foreign keys, you can use `{{ row.display(\"column_name\") }}`.\r\n\r\nI can't see any examples of anyone using it in this code search: https://cs.github.com/?scopeName=All+repos&scope=&q=datasette+row.display\r\n\r\nIt is however useful to have some kind of abstraction layer here that insulates the SQLite `Row` object, since having an extra layer will help if Datasette ever grows support for alternative database backends such as DuckDB or PostgreSQL.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1460988975", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1460988975, "node_id": "IC_kwDOBm6k_c5XFOwv", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T22:42:57Z", "updated_at": "2023-03-08T22:42:57Z", "author_association": "OWNER", "body": "Aside idea: it might be interesting if there were \"lazy\" template variables available in the context: things that are not actually executed unless a template author requests them.\r\n\r\nImagine if `metadata` was a lazy template reference, such that custom templates that don't display any metadata don't trigger it to be resolved (which might involve additional database queries some day).", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1460986533", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1460986533, "node_id": "IC_kwDOBm6k_c5XFOKl", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T22:40:28Z", "updated_at": "2023-03-08T22:40:28Z", "author_association": "OWNER", "body": "Figuring out what to do with `display_columns_and_rows()` is hard. That returns rows as this special kind of object, which is designed to be accessed from the HTML templates:\r\n\r\nhttps://github.com/simonw/datasette/blob/96e94f9b7b2db53865e61390bcce6761727f26d8/datasette/views/table.py#L45-L71", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1460970807", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1460970807, "node_id": "IC_kwDOBm6k_c5XFKU3", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T22:31:49Z", "updated_at": "2023-03-08T22:33:03Z", "author_association": "OWNER", "body": "For the HTML version, I need to decide where all of the stuff that happens in `async def extra_template()` is going to live.\r\n\r\nI think it's another one of those extra functions, triggered for `?_extra=context`.\r\n\r\nhttps://github.com/simonw/datasette/blob/96e94f9b7b2db53865e61390bcce6761727f26d8/datasette/views/table.py#L813-L912", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1460943097", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1460943097, "node_id": "IC_kwDOBm6k_c5XFDj5", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T22:09:24Z", "updated_at": "2023-03-08T22:09:47Z", "author_association": "OWNER", "body": "The ease with which I added that `?_extra=query` feature in https://github.com/simonw/datasette/pull/1999/commits/96e94f9b7b2db53865e61390bcce6761727f26d8 made me feel really confident that this architecture is going in the right direction.\r\n\r\n```diff\r\ndiff --git a/datasette/views/table.py b/datasette/views/table.py\r\nindex 8d3bb2c930..3e1db9c85f 100644\r\n--- a/datasette/views/table.py\r\n+++ b/datasette/views/table.py\r\n@@ -1913,6 +1913,13 @@ async def extra_request():\r\n \"args\": request.args._data,\r\n }\r\n \r\n+ async def extra_query():\r\n+ \"Details of the underlying SQL query\"\r\n+ return {\r\n+ \"sql\": sql,\r\n+ \"params\": params,\r\n+ }\r\n+\r\n async def extra_extras():\r\n \"Available ?_extra= blocks\"\r\n return {\r\n@@ -1938,6 +1945,7 @@ async def extra_extras():\r\n extra_primary_keys,\r\n extra_debug,\r\n extra_request,\r\n+ extra_query,\r\n extra_extras,\r\n )\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1460916405", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1460916405, "node_id": "IC_kwDOBm6k_c5XE9C1", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T21:43:27Z", "updated_at": "2023-03-08T21:43:27Z", "author_association": "OWNER", "body": "Just noticed that `_json=colname` is not working, and that's because it's handled by the renderer here:\r\n\r\nhttps://github.com/simonw/datasette/blob/56b0758a5fbf85d01ff80a40c9b028469d7bb65f/datasette/renderer.py#L29-L40\r\n\r\nBut that's not currently being called by my new code.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1460907148", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1460907148, "node_id": "IC_kwDOBm6k_c5XE6yM", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T21:34:30Z", "updated_at": "2023-03-08T21:34:30Z", "author_association": "OWNER", "body": "I'm going to hold off on that refactor until later, when I have tests to show me if the refactor works or not.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1460906741", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1460906741, "node_id": "IC_kwDOBm6k_c5XE6r1", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T21:34:08Z", "updated_at": "2023-03-08T21:34:08Z", "author_association": "OWNER", "body": "So maybe I can refactor it to look a bit more like this:\r\n\r\nhttps://github.com/simonw/datasette/blob/db1a88f4e17a1f50bdaa681e8beddb2276503e7c/datasette/views/table.py#L1602-L1604\r\n\r\nOne thing that's useful here is that `is_view` is handled early, like this:\r\n\r\nhttps://github.com/simonw/datasette/blob/db1a88f4e17a1f50bdaa681e8beddb2276503e7c/datasette/views/table.py#L466-L472\r\n\r\nSo if I omit the `is_view` bit from the extracted function I can simplify more.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1460905469", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1460905469, "node_id": "IC_kwDOBm6k_c5XE6X9", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T21:32:56Z", "updated_at": "2023-03-08T21:32:56Z", "author_association": "OWNER", "body": "I'd really like to extract this ugly logic out into a helper function:\r\n\r\nhttps://github.com/simonw/datasette/blob/db1a88f4e17a1f50bdaa681e8beddb2276503e7c/datasette/views/table.py#L1617-L1684\r\n\r\nI copied it in and asked ChatGPT to \"List all of the variable that are either modified or assigned to by the above code\":\r\n\r\n> Modified variables:\r\n> \r\n> offset\r\n> where_clauses\r\n> params\r\n> order_by\r\n> \r\n> Assigned variables:\r\n> \r\n> sort_value\r\n> next_by_pk_clauses\r\n> order_by (when sort or sort_desc is not None)\r\n\r\nThen I asked which variables were used as inputs, and argued with it a bit about whether it should be counting functions. Eventually got to this:\r\n\r\n> My apologies for the oversight. Here are the variables needed as input by the above code, excluding any functions:\r\n> \r\n> _next\r\n> is_view\r\n> sort\r\n> sort_desc\r\n> use_rowid\r\n> pks\r\n> order_by\r\n\r\nNote that `use_rowid` is actually defined earlier in terms of two of those other variables: https://github.com/simonw/datasette/blob/db1a88f4e17a1f50bdaa681e8beddb2276503e7c/datasette/views/table.py#L1540", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1460760116", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1460760116, "node_id": "IC_kwDOBm6k_c5XEW40", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T19:48:52Z", "updated_at": "2023-03-08T19:48:52Z", "author_association": "OWNER", "body": "I'm trying to get `http://127.0.0.1:8001/fixtures/compound_three_primary_keys?_next=a,d,v` to return the correct results.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1421988953", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1421988953, "node_id": "IC_kwDOBm6k_c5UwdRZ", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-02-08T04:35:44Z", "updated_at": "2023-02-08T05:27:48Z", "author_association": "OWNER", "body": "Next step: get `?_next=...` working (it is ignored at the moment, even though the returned JSON includes the `\"next\"` key).\r\n\r\nThen... figure out how to render HTML and other requested formats.\r\n\r\nThen get the tests to pass!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1399343659", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1399343659, "node_id": "IC_kwDOBm6k_c5TaEor", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-01-21T22:19:20Z", "updated_at": "2023-02-06T23:02:12Z", "author_association": "OWNER", "body": "HTML mode needs a list of renderers so it can show links to `.geojson` etc - can do that as a hidden extra (maybe called `renderers`), repeating this code:\r\n\r\nhttps://github.com/simonw/datasette/blob/e4ebef082de90db4e1b8527abc0d582b7ae0bc9d/datasette/views/base.py#L477-L497", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1399341658", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1399341658, "node_id": "IC_kwDOBm6k_c5TaEJa", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-01-21T22:06:29Z", "updated_at": "2023-01-21T22:07:30Z", "author_association": "OWNER", "body": "Relevant:\r\n- #1101\r\n- #1672\r\n- #1062", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1399341151", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1399341151, "node_id": "IC_kwDOBm6k_c5TaEBf", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-01-21T22:03:20Z", "updated_at": "2023-01-21T22:03:20Z", "author_association": "OWNER", "body": "I think I'm going to have to write a new view function from scratch which completely ignores the existing BaseView/DataView/TableView hierarchy.\r\n\r\nHere's what I get on the incoming request:\r\n```\r\n(Pdb) request.url, request.full_path, request.host, request.url_vars\r\n('http://127.0.0.1:8001/content/repos.json', '/content/repos.json', '127.0.0.1:8001',\r\n {'database': 'content', 'table': 'repos', 'format': 'json'})\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null}