{"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1461047607", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1461047607, "node_id": "IC_kwDOBm6k_c5XFdE3", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T23:51:46Z", "updated_at": "2023-03-08T23:51:46Z", "author_association": "OWNER", "body": "This feels quite nice:\r\n\r\n\"image\"\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1461044477", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1461044477, "node_id": "IC_kwDOBm6k_c5XFcT9", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T23:47:26Z", "updated_at": "2023-03-08T23:47:26Z", "author_association": "OWNER", "body": "I want to package together all of the extras that are needed for the HTML format. A few options for doing that:\r\n\r\n- Introduce `?_extra=_html` where the leading underscore indicates that this is a \"bundle\" of extras, then define a bundle that's everything needed for the HTML renderer\r\n- Have some other mechanism whereby different renderers can request a bundle of extras.\r\n\r\nI'm leaning towards the first option. I'll try that and see what it looks like.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1461023559", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1461023559, "node_id": "IC_kwDOBm6k_c5XFXNH", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T23:23:02Z", "updated_at": "2023-03-08T23:23:02Z", "author_association": "OWNER", "body": "To get this unblocked, I'm going to allow myself to pass non-JSON-serializable objects to the HTML template version of things. If I can get that working (and get the existing tests to pass) I can consider a later change that makes those JSON serializable - or admit that it's OK for the templates to have non-JSON data passed to them and figure out how best to document those variables independently from the JSON documentation.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1461002039", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1461002039, "node_id": "IC_kwDOBm6k_c5XFR83", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T22:58:16Z", "updated_at": "2023-03-08T23:02:09Z", "author_association": "OWNER", "body": "The reason for that `Row` thing is that it allows custom templates that do things like this:\r\n\r\nhttps://docs.datasette.io/en/stable/changelog.html#easier-custom-templates-for-table-rows\r\n\r\n```html+jinja\r\n{% for row in display_rows %}\r\n
\r\n

{{ row[\"title\"] }}

\r\n

{{ row[\"description\"] }}\r\n

Category: {{ row.display(\"category_id\") }}

\r\n
\r\n{% endfor %}\r\n```\r\nIs that a good design? the `.display()` thing feels weird - I wonder if anyone has ever actually used that.\r\n\r\nIt's documented here: https://docs.datasette.io/en/0.64.2/custom_templates.html#custom-templates\r\n\r\n> If you want to output the rendered HTML version of a column, including any links to foreign keys, you can use `{{ row.display(\"column_name\") }}`.\r\n\r\nI can't see any examples of anyone using it in this code search: https://cs.github.com/?scopeName=All+repos&scope=&q=datasette+row.display\r\n\r\nIt is however useful to have some kind of abstraction layer here that insulates the SQLite `Row` object, since having an extra layer will help if Datasette ever grows support for alternative database backends such as DuckDB or PostgreSQL.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1460988975", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1460988975, "node_id": "IC_kwDOBm6k_c5XFOwv", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T22:42:57Z", "updated_at": "2023-03-08T22:42:57Z", "author_association": "OWNER", "body": "Aside idea: it might be interesting if there were \"lazy\" template variables available in the context: things that are not actually executed unless a template author requests them.\r\n\r\nImagine if `metadata` was a lazy template reference, such that custom templates that don't display any metadata don't trigger it to be resolved (which might involve additional database queries some day).", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1460986533", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1460986533, "node_id": "IC_kwDOBm6k_c5XFOKl", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T22:40:28Z", "updated_at": "2023-03-08T22:40:28Z", "author_association": "OWNER", "body": "Figuring out what to do with `display_columns_and_rows()` is hard. That returns rows as this special kind of object, which is designed to be accessed from the HTML templates:\r\n\r\nhttps://github.com/simonw/datasette/blob/96e94f9b7b2db53865e61390bcce6761727f26d8/datasette/views/table.py#L45-L71", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1460970807", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1460970807, "node_id": "IC_kwDOBm6k_c5XFKU3", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T22:31:49Z", "updated_at": "2023-03-08T22:33:03Z", "author_association": "OWNER", "body": "For the HTML version, I need to decide where all of the stuff that happens in `async def extra_template()` is going to live.\r\n\r\nI think it's another one of those extra functions, triggered for `?_extra=context`.\r\n\r\nhttps://github.com/simonw/datasette/blob/96e94f9b7b2db53865e61390bcce6761727f26d8/datasette/views/table.py#L813-L912", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1460943097", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1460943097, "node_id": "IC_kwDOBm6k_c5XFDj5", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T22:09:24Z", "updated_at": "2023-03-08T22:09:47Z", "author_association": "OWNER", "body": "The ease with which I added that `?_extra=query` feature in https://github.com/simonw/datasette/pull/1999/commits/96e94f9b7b2db53865e61390bcce6761727f26d8 made me feel really confident that this architecture is going in the right direction.\r\n\r\n```diff\r\ndiff --git a/datasette/views/table.py b/datasette/views/table.py\r\nindex 8d3bb2c930..3e1db9c85f 100644\r\n--- a/datasette/views/table.py\r\n+++ b/datasette/views/table.py\r\n@@ -1913,6 +1913,13 @@ async def extra_request():\r\n \"args\": request.args._data,\r\n }\r\n \r\n+ async def extra_query():\r\n+ \"Details of the underlying SQL query\"\r\n+ return {\r\n+ \"sql\": sql,\r\n+ \"params\": params,\r\n+ }\r\n+\r\n async def extra_extras():\r\n \"Available ?_extra= blocks\"\r\n return {\r\n@@ -1938,6 +1945,7 @@ async def extra_extras():\r\n extra_primary_keys,\r\n extra_debug,\r\n extra_request,\r\n+ extra_query,\r\n extra_extras,\r\n )\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1460916405", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1460916405, "node_id": "IC_kwDOBm6k_c5XE9C1", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T21:43:27Z", "updated_at": "2023-03-08T21:43:27Z", "author_association": "OWNER", "body": "Just noticed that `_json=colname` is not working, and that's because it's handled by the renderer here:\r\n\r\nhttps://github.com/simonw/datasette/blob/56b0758a5fbf85d01ff80a40c9b028469d7bb65f/datasette/renderer.py#L29-L40\r\n\r\nBut that's not currently being called by my new code.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1460907148", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1460907148, "node_id": "IC_kwDOBm6k_c5XE6yM", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T21:34:30Z", "updated_at": "2023-03-08T21:34:30Z", "author_association": "OWNER", "body": "I'm going to hold off on that refactor until later, when I have tests to show me if the refactor works or not.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1460906741", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1460906741, "node_id": "IC_kwDOBm6k_c5XE6r1", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T21:34:08Z", "updated_at": "2023-03-08T21:34:08Z", "author_association": "OWNER", "body": "So maybe I can refactor it to look a bit more like this:\r\n\r\nhttps://github.com/simonw/datasette/blob/db1a88f4e17a1f50bdaa681e8beddb2276503e7c/datasette/views/table.py#L1602-L1604\r\n\r\nOne thing that's useful here is that `is_view` is handled early, like this:\r\n\r\nhttps://github.com/simonw/datasette/blob/db1a88f4e17a1f50bdaa681e8beddb2276503e7c/datasette/views/table.py#L466-L472\r\n\r\nSo if I omit the `is_view` bit from the extracted function I can simplify more.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1460905469", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1460905469, "node_id": "IC_kwDOBm6k_c5XE6X9", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T21:32:56Z", "updated_at": "2023-03-08T21:32:56Z", "author_association": "OWNER", "body": "I'd really like to extract this ugly logic out into a helper function:\r\n\r\nhttps://github.com/simonw/datasette/blob/db1a88f4e17a1f50bdaa681e8beddb2276503e7c/datasette/views/table.py#L1617-L1684\r\n\r\nI copied it in and asked ChatGPT to \"List all of the variable that are either modified or assigned to by the above code\":\r\n\r\n> Modified variables:\r\n> \r\n> offset\r\n> where_clauses\r\n> params\r\n> order_by\r\n> \r\n> Assigned variables:\r\n> \r\n> sort_value\r\n> next_by_pk_clauses\r\n> order_by (when sort or sort_desc is not None)\r\n\r\nThen I asked which variables were used as inputs, and argued with it a bit about whether it should be counting functions. Eventually got to this:\r\n\r\n> My apologies for the oversight. Here are the variables needed as input by the above code, excluding any functions:\r\n> \r\n> _next\r\n> is_view\r\n> sort\r\n> sort_desc\r\n> use_rowid\r\n> pks\r\n> order_by\r\n\r\nNote that `use_rowid` is actually defined earlier in terms of two of those other variables: https://github.com/simonw/datasette/blob/db1a88f4e17a1f50bdaa681e8beddb2276503e7c/datasette/views/table.py#L1540", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2036#issuecomment-1460866243", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2036", "id": 1460866243, "node_id": "IC_kwDOBm6k_c5XEwzD", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T20:57:34Z", "updated_at": "2023-03-08T20:57:34Z", "author_association": "OWNER", "body": "This fix is released in 0.64.2 https://docs.datasette.io/en/stable/changelog.html#v0-64-2", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1615862295, "label": "`publish cloudrun` reuses image tags, which can lead to very surprising deploy problems"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2036#issuecomment-1460848869", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2036", "id": 1460848869, "node_id": "IC_kwDOBm6k_c5XEsjl", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T20:40:55Z", "updated_at": "2023-03-08T20:40:55Z", "author_association": "OWNER", "body": "Here's the https://latest.datasette.io/ deployment that just went out, further demonstrating that this change is working correctly:\r\n\r\n\"image\"\r\n\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1615862295, "label": "`publish cloudrun` reuses image tags, which can lead to very surprising deploy problems"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2037#issuecomment-1460840620", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2037", "id": 1460840620, "node_id": "IC_kwDOBm6k_c5XEqis", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T20:33:00Z", "updated_at": "2023-03-08T20:33:00Z", "author_association": "OWNER", "body": "Got the same failure again for a recent commit: https://github.com/simonw/datasette/actions/runs/4368239376/jobs/7640567282", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1615891776, "label": "Test failure: FAILED tests/test_cli.py::test_install_requirements - FileNotFoundError"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2037#issuecomment-1460838797", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2037", "id": 1460838797, "node_id": "IC_kwDOBm6k_c5XEqGN", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T20:31:15Z", "updated_at": "2023-03-08T20:31:15Z", "author_association": "OWNER", "body": "It's this test here:\r\n\r\nhttps://github.com/simonw/datasette/blob/1ad92a1d87d79084ebe524ed186c900ff042328c/tests/test_cli.py#L181-L189\r\n\r\nAdded in:\r\n- #2033 ", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1615891776, "label": "Test failure: FAILED tests/test_cli.py::test_install_requirements - FileNotFoundError"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2037#issuecomment-1460838109", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2037", "id": 1460838109, "node_id": "IC_kwDOBm6k_c5XEp7d", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T20:30:36Z", "updated_at": "2023-03-08T20:30:36Z", "author_association": "OWNER", "body": "Instead of using `isolated_filesystem()` I could use a `tmpdir` fixture instead.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1615891776, "label": "Test failure: FAILED tests/test_cli.py::test_install_requirements - FileNotFoundError"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2036#issuecomment-1460827178", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2036", "id": 1460827178, "node_id": "IC_kwDOBm6k_c5XEnQq", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T20:25:10Z", "updated_at": "2023-03-08T20:25:10Z", "author_association": "OWNER", "body": "https://console.cloud.google.com/run/detail/us-central1/new-service/revisions?project=datasette-222320 confirms that the image deployed is:\r\n\r\n\"image\"\r\n\r\nCompared to https://console.cloud.google.com/run/detail/us-central1/datasette-io/revisions?project=datasette-222320 which shows that `datasette.io` is running:\r\n\r\n\"image\"\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1615862295, "label": "`publish cloudrun` reuses image tags, which can lead to very surprising deploy problems"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2036#issuecomment-1460816528", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2036", "id": 1460816528, "node_id": "IC_kwDOBm6k_c5XEkqQ", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T20:22:50Z", "updated_at": "2023-03-08T20:23:20Z", "author_association": "OWNER", "body": "Testing this manually:\r\n\r\n```\r\n% datasette publish cloudrun content.db --service new-service\r\nCreating temporary tarball archive of 2 file(s) totalling 13.8 MiB before compression.\r\nUploading tarball of [.] to [gs://datasette-222320_cloudbuild/source/1678306859.271661-805303f364144b6094cc9c8532ab5133.tgz]\r\nCreated [https://cloudbuild.googleapis.com/v1/projects/datasette-222320/locations/global/builds/290f41a4-e29a-443c-a1e5-c54513c6143d].\r\nLogs are available at [ https://console.cloud.google.com/cloud-build/builds/290f41a4-e29a-443c-a1e5-c54513c6143d?project=99025868001 ].\r\n---- REMOTE BUILD OUTPUT ----\r\nstarting build \"290f41a4-e29a-443c-a1e5-c54513c6143d\"\r\n\r\nFETCHSOURCE\r\nFetching storage object: gs://datasette-222320_cloudbuild/source/1678306859.271661-805303f364144b6094cc9c8532ab5133.tgz#1678306862810483\r\nCopying gs://datasette-222320_cloudbuild/source/1678306859.271661-805303f364144b6094cc9c8532ab5133.tgz#1678306862810483...\r\n/ [1 files][ 3.9 MiB/ 3.9 MiB] \r\nOperation completed over 1 objects/3.9 MiB.\r\nBUILD\r\nAlready have image (with digest): gcr.io/cloud-builders/docker\r\nSending build context to Docker daemon 14.52MB\r\nStep 1/9 : FROM python:3.11.0-slim-bullseye\r\n...\r\nInstalling collected packages: rfc3986, typing-extensions, sniffio, PyYAML, python-multipart, pluggy, pint, mergedeep, MarkupSafe, itsdangerous, idna, hupper, h11, click, certifi, asgiref, aiofiles, uvicorn, Jinja2, janus, click-default-group-wheel, asgi-csrf, anyio, httpcore, httpx, datasette\r\nSuccessfully installed Jinja2-3.1.2 MarkupSafe-2.1.2 PyYAML-6.0 aiofiles-23.1.0 anyio-3.6.2 asgi-csrf-0.9 asgiref-3.6.0 certifi-2022.12.7 click-8.1.3 click-default-group-wheel-1.2.2 datasette-0.64.1 h11-0.14.0 httpcore-0.16.3 httpx-0.23.3 hupper-1.11 idna-3.4 itsdangerous-2.1.2 janus-1.0.0 mergedeep-1.3.4 pint-0.20.1 pluggy-1.0.0 python-multipart-0.0.6 rfc3986-1.5.0 sniffio-1.3.0 typing-extensions-4.5.0 uvicorn-0.20.0\r\nWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\r\n\r\n[notice] A new release of pip available: 22.3 -> 23.0.1\r\n[notice] To update, run: pip install --upgrade pip\r\nRemoving intermediate container 8ccebfebebc9\r\n ---> b972c85b38bb\r\n...\r\nSuccessfully built 606b7c286d7f\r\nSuccessfully tagged gcr.io/datasette-222320/datasette-new-service:latest\r\nPUSH\r\nPushing gcr.io/datasette-222320/datasette-new-service\r\nThe push refers to repository [gcr.io/datasette-222320/datasette-new-service]\r\n667b1dc69e5e: Preparing\r\n...\r\nd8ddfcff216f: Pushed\r\nlatest: digest: sha256:452daffb2d3d7a8579c2ab39854be285155252c9428b4c1c50caac6a3a269e3f size: 2004\r\nDONE\r\n---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\r\nID CREATE_TIME DURATION SOURCE IMAGES STATUS\r\n290f41a4-e29a-443c-a1e5-c54513c6143d 2023-03-08T20:21:03+00:00 39S gs://datasette-222320_cloudbuild/source/1678306859.271661-805303f364144b6094cc9c8532ab5133.tgz gcr.io/datasette-222320/datasette-new-service (+1 more) SUCCESS\r\nDeploying container to Cloud Run service [new-service] in project [datasette-222320] region [us-central1]\r\n\u2713 Deploying new service... Done. \r\n \u2713 Creating Revision... \r\n \u2713 Routing traffic... \r\n \u2713 Setting IAM Policy... \r\nDone. \r\nService [new-service] revision [new-service-00001-zon] has been deployed and is serving 100 percent of traffic.\r\nService URL: https://new-service-j7hipcg4aq-uc.a.run.app\r\n```\r\nhttps://new-service-j7hipcg4aq-uc.a.run.app/ was deployed successfully.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1615862295, "label": "`publish cloudrun` reuses image tags, which can lead to very surprising deploy problems"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2036#issuecomment-1460810523", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2036", "id": 1460810523, "node_id": "IC_kwDOBm6k_c5XEjMb", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T20:17:01Z", "updated_at": "2023-03-08T20:17:01Z", "author_association": "OWNER", "body": "I'm going to solve this by using the service name in that `image_id` instead:\r\n\r\n```python\r\nimage_id = f\"gcr.io/{project}/{service_name}\"\r\n```\r\nThis is a nasty bug, so I'm going to backport it to a `0.64.2` release as well.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1615862295, "label": "`publish cloudrun` reuses image tags, which can lead to very surprising deploy problems"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2036#issuecomment-1460809643", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2036", "id": 1460809643, "node_id": "IC_kwDOBm6k_c5XEi-r", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T20:16:10Z", "updated_at": "2023-03-08T20:16:10Z", "author_association": "OWNER", "body": "I think the code at fault is here:\r\n\r\nhttps://github.com/simonw/datasette/blob/1ad92a1d87d79084ebe524ed186c900ff042328c/datasette/publish/cloudrun.py#L176-L182\r\n\r\nThat name ends up defaulting to `datasette` - so multiple different projects may end up deploying to the same `image_id`.\r\n\r\nWhat I think happened in the `datasette.io` bug is that this workflow: https://github.com/simonw/simonwillisonblog-backup/blob/bfb573e96d8622ab52b22fdcd54724fe6e59fd24/.github/workflows/backup.yml and this workflow: https://github.com/simonw/datasette.io/blob/4676db5bf4a3fc9f792ee270ec0c59eb902cd2c3/.github/workflows/deploy.yml both happened to run at the exact same time.\r\n\r\nAnd so the image that was pushed to `gcr.io/datasette-222320/datasette:latest` by the `simonw/simonwillisonblog-backup` action was then deployed by the `simonw/datasette.io/` action, which broke the site.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1615862295, "label": "`publish cloudrun` reuses image tags, which can lead to very surprising deploy problems"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1460760116", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1460760116, "node_id": "IC_kwDOBm6k_c5XEW40", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T19:48:52Z", "updated_at": "2023-03-08T19:48:52Z", "author_association": "OWNER", "body": "I'm trying to get `http://127.0.0.1:8001/fixtures/compound_three_primary_keys?_next=a,d,v` to return the correct results.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2035#issuecomment-1460682625", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2035", "id": 1460682625, "node_id": "IC_kwDOBm6k_c5XED-B", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T18:40:57Z", "updated_at": "2023-03-08T18:40:57Z", "author_association": "OWNER", "body": "Pushed that prototype to a branch: https://github.com/simonw/datasette/commit/0fe844e9adb006a0138e83102ced1329d9155c59 / https://github.com/simonw/datasette/compare/sql-list-parameters", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1615692818, "label": "Potential feature: special support for `?a=1&a=2` on the query page"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2035#issuecomment-1460679434", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2035", "id": 1460679434, "node_id": "IC_kwDOBm6k_c5XEDMK", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T18:39:35Z", "updated_at": "2023-03-08T18:39:35Z", "author_association": "OWNER", "body": "I should consider the existing design of magic parameters here: https://docs.datasette.io/en/stable/sql_queries.html#magic-parameters\r\n\r\n- `_actor_*`\r\n- `_header_*`\r\n- `_cookie_`\r\n- `_now_epoch`\r\n- `_now_date_utc`\r\n- `_now_datetime_utc`\r\n- `_random_chars_*`\r\n\r\nShould this new `id__list` syntax look more like those magic parameters, or is it OK to use `name__magic` syntax here instead?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1615692818, "label": "Potential feature: special support for `?a=1&a=2` on the query page"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2035#issuecomment-1460668431", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2035", "id": 1460668431, "node_id": "IC_kwDOBm6k_c5XEAgP", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T18:35:34Z", "updated_at": "2023-03-08T18:35:34Z", "author_association": "OWNER", "body": "To implement this properly need to do the following:\r\n- Get the page to display multiple `id: [ text input here ]` fields such that re-submission works\r\n- Figure out how this should work for canned queries and for writable canned queries\r\n- Tests that cover queries, canned queries, writable canned queries\r\n\r\nAnd a bonus feature: what if the Datasette UI layer spotted `:id__list` parameters and used them to add a bit of JavaScript that allowed users to click a `+` button next to an `id` form field to add another one?\r\n\r\nAlso, when a page is re-displayed for on of these queries it could potentially add an extra form field allowing people to add another value.\r\n\r\nThough this has an annoying problem: how to tell the difference between an additional `id` input field that the user chose not to populate, v.s. one that is supposed to represent an empty string?\r\n\r\nMaybe only support multiple `id` fields for users with JavaScript in order to avoid this problem.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1615692818, "label": "Potential feature: special support for `?a=1&a=2` on the query page"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2035#issuecomment-1460664619", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2035", "id": 1460664619, "node_id": "IC_kwDOBm6k_c5XD_kr", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T18:32:29Z", "updated_at": "2023-03-08T18:32:29Z", "author_association": "OWNER", "body": "Got a prototype working:\r\n```diff\r\ndiff --git a/datasette/views/database.py b/datasette/views/database.py\r\nindex 8d289105..6f9d8a44 100644\r\n--- a/datasette/views/database.py\r\n+++ b/datasette/views/database.py\r\n@@ -226,6 +226,12 @@ class QueryView(DataView):\r\n ):\r\n db = await self.ds.resolve_database(request)\r\n database = db.name\r\n+ # Disallow x__list query string parameters\r\n+ invalid_params = [k for k in request.args if k.endswith(\"__list\")]\r\n+ if invalid_params:\r\n+ raise DatasetteError(\r\n+ \"Invalid query string parameters: {}\".format(\", \".join(invalid_params))\r\n+ )\r\n params = {key: request.args.get(key) for key in request.args}\r\n if \"sql\" in params:\r\n params.pop(\"sql\")\r\n@@ -258,6 +264,11 @@ class QueryView(DataView):\r\n for named_parameter in named_parameters\r\n if not named_parameter.startswith(\"_\")\r\n }\r\n+ # Handle any __list parameters\r\n+ for named_parameter in named_parameters:\r\n+ if named_parameter.endswith(\"__list\"):\r\n+ list_values = request.args.getlist(named_parameter[:-6])\r\n+ params[named_parameter] = json.dumps(list_values)\r\n \r\n # Set to blank string if missing from params\r\n for named_parameter in named_parameters:\r\n```\r\nThis isn't yet doing the right thing on form re-submission: it breaks because it attempts to pass through the `?id__list=` invalid parameter. But I did manage to get it to do this through careful editing of the URL:\r\n\r\n\"image\"\r\n\r\nThat was this URL: `http://127.0.0.1:8034/content?sql=select+%3Aid__list%2C*+from+releases+where+id+in+(select+value+from+json_each(%3Aid__list))&id=62642726&id=18402901&id=38714866`", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1615692818, "label": "Potential feature: special support for `?a=1&a=2` on the query page"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2035#issuecomment-1460659382", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2035", "id": 1460659382, "node_id": "IC_kwDOBm6k_c5XD-S2", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T18:28:00Z", "updated_at": "2023-03-08T18:28:00Z", "author_association": "OWNER", "body": "Also: `datasette-explain` may need to be updated to understand how to handle this:\r\n\r\n`ERROR: conn=, sql = 'explain select * from releases where id in (select id from json_each(:id__list))', params = None: You did not supply a value for binding parameter :id__list.`\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1615692818, "label": "Potential feature: special support for `?a=1&a=2` on the query page"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2035#issuecomment-1460654136", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2035", "id": 1460654136, "node_id": "IC_kwDOBm6k_c5XD9A4", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T18:25:46Z", "updated_at": "2023-03-08T18:25:46Z", "author_association": "OWNER", "body": "Trickiest part of the implementation here is that it needs to know to output three `id` HTML form fields on the page, such that their values are persisted when the form is submitted a second time.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1615692818, "label": "Potential feature: special support for `?a=1&a=2` on the query page"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2035#issuecomment-1460639749", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2035", "id": 1460639749, "node_id": "IC_kwDOBm6k_c5XD5gF", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T18:17:31Z", "updated_at": "2023-03-08T18:17:31Z", "author_association": "OWNER", "body": "Since we are pre-1.0 it's still OK to implement a feature that disallows `?id__list=` in the URL, but allows `:id__list` in SQL queries to reference the JSON list of parameters.\r\n\r\nSo I'm going to prototype this as the `:id__list` feature and see how it feels.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1615692818, "label": "Potential feature: special support for `?a=1&a=2` on the query page"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2035#issuecomment-1460637906", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2035", "id": 1460637906, "node_id": "IC_kwDOBm6k_c5XD5DS", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T18:16:31Z", "updated_at": "2023-03-08T18:16:31Z", "author_association": "OWNER", "body": "I'm pretty sold on this as a feature now. The main question I have is which of these options to implement:\r\n\r\n1. `?id=1&?id=2` results in `:id` in the query being `[\"1\", \"2\"]` - no additional syntax required\r\n2. `:id` in the query continues to reference just the first of those parameters - but `:id__list` (or some other custom syntax) instead gets `[\"1\", \"2\"]` - or, if the URL is `?id=1` - gets `[\"1\"]`\r\n\r\nActually on writing these out I realize that option 2 is the ONLY valid option. It's no good building a query that works against a JSON list if the user might pass just a single ID, `?id=1`, resulting in their query breaking.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1615692818, "label": "Potential feature: special support for `?a=1&a=2` on the query page"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2035#issuecomment-1460632758", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2035", "id": 1460632758, "node_id": "IC_kwDOBm6k_c5XD3y2", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T18:13:49Z", "updated_at": "2023-03-08T18:13:49Z", "author_association": "OWNER", "body": "https://github.com/rclement/datasette-dashboards/issues/54 makes the excellent point that the `\r\n \r\n \r\n \r\n \r\n\r\n\r\n\r\n```\r\nSubmitting that form with the middle two options selected navigates to: `https://www.example.com/?id=32&id=15`", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1615692818, "label": "Potential feature: special support for `?a=1&a=2` on the query page"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2035#issuecomment-1460628199", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2035", "id": 1460628199, "node_id": "IC_kwDOBm6k_c5XD2rn", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T18:11:31Z", "updated_at": "2023-03-08T18:11:31Z", "author_association": "OWNER", "body": "One variant on this idea: maybe you have to specify in your query that you want it to be the JSON list version, not the single item (first `?id=` parameter version)? Maybe with syntax like this:\r\n\r\n where id in (select value from json_each(:id__list))\r\n\r\nDatasette would automatically pass `{\"id\": \"11\", \"id__list\": '[\"11\", \"32\", \"62\"]'}` as arguments to the `db.execute()` method, if the page was called with `?id=11&id=32&id=62`.\r\n\r\nThis is more explicit, though the syntax is a bit uglier (maybe there's a nicer design for this?). I also worry about `?id__list=` conflicting with this, but I think that's a risk I can take - tell people not to do that, or even block `?id__list=` style parameters entirely.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1615692818, "label": "Potential feature: special support for `?a=1&a=2` on the query page"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2035#issuecomment-1460621871", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2035", "id": 1460621871, "node_id": "IC_kwDOBm6k_c5XD1Iv", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T18:08:25Z", "updated_at": "2023-03-08T18:09:04Z", "author_association": "OWNER", "body": "My current preferred solution is to lean into SQLite's JSON support.\r\n\r\nWhat if the query page spotted `?id=11&id=32&id=62` and turned that into a JSON string called `:id:` with a value of `[\"11\", \"32\", \"62\"]`?\r\n\r\nNote that this is still a string, not a list. This avoids a nasty problem that occurred in PHP world, where `?id[]=1&id[]=2` would result in an actual PHP array object, which often broke underlying code that had expected `$_GET[\"id\"]` to be a string, not an array.\r\n\r\nSo in a query you'd be able to do this:\r\n\r\n where id in (select value from json_each(:id))\r\n\r\nAnd then call it with `?id=11&id=32&id=62`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1615692818, "label": "Potential feature: special support for `?a=1&a=2` on the query page"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2035#issuecomment-1460618433", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2035", "id": 1460618433, "node_id": "IC_kwDOBm6k_c5XD0TB", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T18:06:34Z", "updated_at": "2023-03-08T18:06:34Z", "author_association": "OWNER", "body": "One way to do this would be to dynamically generate the `where id in (?, ?, ?)` with the correct number of question marks, then feed in a list from `request.args.getlist(\"id\")` - but that would require rewriting the SQL query text to add those question marks.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1615692818, "label": "Potential feature: special support for `?a=1&a=2` on the query page"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2033#issuecomment-1457117383", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2033", "id": 1457117383, "node_id": "IC_kwDOBm6k_c5W2djH", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-06T22:28:55Z", "updated_at": "2023-03-06T22:28:55Z", "author_association": "OWNER", "body": "Documentation: https://docs.datasette.io/en/latest/plugins.html#installing-plugins", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1612296210, "label": "`datasette install -r requirements.txt`"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/2031#issuecomment-1456997425", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2031", "id": 1456997425, "node_id": "IC_kwDOBm6k_c5W2AQx", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-06T21:04:27Z", "updated_at": "2023-03-06T21:06:34Z", "author_association": "OWNER", "body": "This is a very neat fix, for something I've been wanting for a while.\r\n\r\nAdd a unit test for the row HTML page - I suggest against this page: https://latest.datasette.io/fixtures/foreign_key_references/1 - and I'll land this PR.\r\n\r\nYou can model it on this test here: https://github.com/simonw/datasette/blob/a53b893c46453f35decc8c145c138671cee6140c/tests/test_table_html.py#L609-L632\r\n\r\nI think adding it to `test_table_html.py` is OK, even though it's technically for the row page and not the table page.\r\n\r\nThanks!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1605481359, "label": "Expand foreign key references in row view as well"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/2028#issuecomment-1456914694", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2028", "id": 1456914694, "node_id": "IC_kwDOBm6k_c5W1sEG", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-06T20:19:37Z", "updated_at": "2023-03-06T20:19:37Z", "author_association": "OWNER", "body": "Thanks!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1590839187, "label": "add Python 3.11 classifier"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2024#issuecomment-1426031395", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2024", "id": 1426031395, "node_id": "IC_kwDOBm6k_c5U_4Mj", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-02-10T16:11:53Z", "updated_at": "2023-02-10T16:11:53Z", "author_association": "OWNER", "body": "Relevant: https://til.simonwillison.net/sqlite/enabling-wal-mode", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1579973223, "label": "Mention WAL mode in documentation"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/262#issuecomment-1423067724", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/262", "id": 1423067724, "node_id": "IC_kwDOBm6k_c5U0kpM", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-02-08T18:33:32Z", "updated_at": "2023-02-08T18:36:48Z", "author_association": "OWNER", "body": "Just realized that it's useful to be able to tell what parameters were used to generate a page... but reflecting things like `_next` back in the JSON is confusing in the presence of `next`.\r\n\r\nSo I'm going to add an extra for that information too.\r\n\r\nNot sure what to call it though:\r\n\r\n- `params` - confusing because in the code that's usually used for params passed to SQL queries\r\n- `query_string` - wouldn't that be a string, not params as a dictionary?\r\n\r\nI'm going to experiment with a `request` extra that returns some bits of information about the request.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 323658641, "label": "Add ?_extra= mechanism for requesting extra properties in JSON"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1421988953", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1421988953, "node_id": "IC_kwDOBm6k_c5UwdRZ", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-02-08T04:35:44Z", "updated_at": "2023-02-08T05:27:48Z", "author_association": "OWNER", "body": "Next step: get `?_next=...` working (it is ignored at the moment, even though the returned JSON includes the `\"next\"` key).\r\n\r\nThen... figure out how to render HTML and other requested formats.\r\n\r\nThen get the tests to pass!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2019#issuecomment-1421784930", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2019", "id": 1421784930, "node_id": "IC_kwDOBm6k_c5Uvrdi", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-02-08T01:28:25Z", "updated_at": "2023-02-08T01:40:46Z", "author_association": "OWNER", "body": "Rather than duplicate this rather awful hack:\r\n\r\nhttps://github.com/simonw/datasette/blob/0b4a28691468b5c758df74fa1d72a823813c96bf/datasette/views/table.py#L694-L714\r\n\r\nI'm tempted to say that the code that calls the new pagination helper needs to ensure that the `sort` or `sort_desc` columns are selected. If it wants to ditch them later (e.g. because they were not included in `?_col=`) it can do that later once the results have come back.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1573424830, "label": "Refactor out the keyset pagination code"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2019#issuecomment-1421600789", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2019", "id": 1421600789, "node_id": "IC_kwDOBm6k_c5Uu-gV", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-02-07T23:12:40Z", "updated_at": "2023-02-07T23:16:20Z", "author_association": "OWNER", "body": "Most complicated example of a paginated query: https://latest.datasette.io/fixtures?sql=select%0D%0A++pk1%2C%0D%0A++pk2%2C%0D%0A++content%2C%0D%0A++sortable%2C%0D%0A++sortable_with_nulls%2C%0D%0A++sortable_with_nulls_2%2C%0D%0A++text%0D%0Afrom%0D%0A++sortable%0D%0Awhere%0D%0A++(%0D%0A++++sortable_with_nulls+is+null%0D%0A++++and+(%0D%0A++++++(pk1+%3E+%3Ap0)%0D%0A++++++or+(%0D%0A++++++++pk1+%3D+%3Ap0%0D%0A++++++++and+pk2+%3E+%3Ap1%0D%0A++++++)%0D%0A++++)%0D%0A++)%0D%0Aorder+by%0D%0A++sortable_with_nulls+desc%2C%0D%0A++pk1%2C%0D%0A++pk2%0D%0Alimit%0D%0A++101&p0=h&p1=r\r\n\r\n```sql\r\nselect\r\n pk1,\r\n pk2,\r\n content,\r\n sortable,\r\n sortable_with_nulls,\r\n sortable_with_nulls_2,\r\n text\r\nfrom\r\n sortable\r\nwhere\r\n (\r\n sortable_with_nulls is null\r\n and (\r\n (pk1 > :p0)\r\n or (\r\n pk1 = :p0\r\n and pk2 > :p1\r\n )\r\n )\r\n )\r\norder by\r\n sortable_with_nulls desc,\r\n pk1,\r\n pk2\r\nlimit\r\n 101\r\n```\r\nGenerated by this page: https://latest.datasette.io/fixtures/sortable?_next=%24null%2Ch%2Cr&_sort_desc=sortable_with_nulls\r\n\r\nThe `_next=` parameter there decodes as `$null,h,r` - and those components are tilde-encoded, so this can be distinguished from an actual `$null` value which would be represented as `~24null`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1573424830, "label": "Refactor out the keyset pagination code"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2019#issuecomment-1421274434", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2019", "id": 1421274434, "node_id": "IC_kwDOBm6k_c5Utu1C", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-02-07T18:42:42Z", "updated_at": "2023-02-07T18:42:42Z", "author_association": "OWNER", "body": "I'm going to build completely separate tests for this in `test_pagination.py`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1573424830, "label": "Refactor out the keyset pagination code"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2019#issuecomment-1420109153", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2019", "id": 1420109153, "node_id": "IC_kwDOBm6k_c5UpSVh", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-02-07T02:32:36Z", "updated_at": "2023-02-07T02:32:36Z", "author_association": "OWNER", "body": "Doing this as a class makes sense to me. There are a few steps:\r\n\r\n- Instantiate the class with the information it needs, which includes sort order, page size, tiebreaker columns and SQL query and parameters\r\n- Generate the new SQL query that will actually be executed - maybe this takes the optional `_next` parameter? This returns the SQL and params that should be executed, where the SQL now includes pagination logic plus order by and limit\r\n- The calling code then gets to execute the SQL query to fetch the rows\r\n- Last step: those rows are passed to a paginator method which returns `(rows, next)` - where `rows` is the rows truncated to the correct length (really just with the last one cut off if it's too long for the length) and `next` is either `None` or a token, depending on if there should be a next page.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1573424830, "label": "Refactor out the keyset pagination code"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2019#issuecomment-1420106315", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2019", "id": 1420106315, "node_id": "IC_kwDOBm6k_c5UpRpL", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-02-07T02:28:03Z", "updated_at": "2023-02-07T02:28:36Z", "author_association": "OWNER", "body": "So I think I can write an abstraction that applies keyset pagination to ANY arbitrary SQL query provided it is given the query, the existing params (so it can pick names for the new params that won't overlap with them), the desired sort order, any existing `_next` token AND the columns that should be used to tie-break any duplicates.\r\n\r\nThose tie breakers will be either the primary key(s) or `rowid` if none are provided.\r\n\r\nWhat about the case of SQL views, where offset/limit should be used instead? I'm inclined to have that as a separate pagination abstraction entirely, with the calling code deciding which pagination helper to use based on if keyset pagination makes sense or not.\r\n\r\nMight be easier to design a class structure for this starting with `OffsetPaginator`, then using that to inform the design of `KeysetPaginator`.\r\n\r\nMight put these in `datasette.utils.pagination` to start off with, then maybe extract them out to `sqlite-utils` later once they've proven themselves.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1573424830, "label": "Refactor out the keyset pagination code"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2019#issuecomment-1420104254", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2019", "id": 1420104254, "node_id": "IC_kwDOBm6k_c5UpRI-", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-02-07T02:24:46Z", "updated_at": "2023-02-07T02:24:46Z", "author_association": "OWNER", "body": "Even more complicated: https://latest.datasette.io/fixtures/sortable?sortable_with_nulls__notnull=1&_next=0~2E692704598586882%2Ce%2Cr&_sort=sortable_with_nulls_2\r\n\r\nThe rewritten SQL for that is:\r\n\r\n```sql\r\nselect * from (select pk1, pk2, content, sortable, sortable_with_nulls, sortable_with_nulls_2, text from sortable where \"sortable_with_nulls\" is not null)\r\n where (sortable_with_nulls_2 > :p2 or (sortable_with_nulls_2 = :p2 and ((pk1 > :p0)\r\n or\r\n(pk1 = :p0 and pk2 > :p1)))) order by sortable_with_nulls_2, pk1, pk2 limit 101\r\n```\r\nAnd it still has the same number of explain steps as the current SQL witohut the subselect.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1573424830, "label": "Refactor out the keyset pagination code"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2019#issuecomment-1420101175", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2019", "id": 1420101175, "node_id": "IC_kwDOBm6k_c5UpQY3", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-02-07T02:22:11Z", "updated_at": "2023-02-07T02:22:11Z", "author_association": "OWNER", "body": "A more complex example: https://latest.datasette.io/fixtures/sortable?_next=0~2E2650566289400591%2Ca%2Cu&_sort=sortable_with_nulls_2\r\n\r\nSQL:\r\n\r\n```sql\r\nselect pk1, pk2, content, sortable, sortable_with_nulls, sortable_with_nulls_2, text from sortable where (sortable_with_nulls_2 > :p2 or (sortable_with_nulls_2 = :p2 and ((pk1 > :p0)\r\n or\r\n(pk1 = :p0 and pk2 > :p1)))) order by sortable_with_nulls_2, pk1, pk2 limit 101\r\n```\r\n\r\nhttps://latest.datasette.io/fixtures?sql=select+pk1%2C+pk2%2C+content%2C+sortable%2C+sortable_with_nulls%2C+sortable_with_nulls_2%2C+text+from+sortable+where+%28sortable_with_nulls_2+%3E+%3Ap2+or+%28sortable_with_nulls_2+%3D+%3Ap2+and+%28%28pk1+%3E+%3Ap0%29%0A++or%0A%28pk1+%3D+%3Ap0+and+pk2+%3E+%3Ap1%29%29%29%29+order+by+sortable_with_nulls_2%2C+pk1%2C+pk2+limit+101&p0=a&p1=u&p2=0.2650566289400591\r\n\r\nHere's the explain: 49 steps long https://latest.datasette.io/fixtures?sql=explain+select+pk1%2C+pk2%2C+content%2C+sortable%2C+sortable_with_nulls%2C+sortable_with_nulls_2%2C+text+from+sortable+where+%28sortable_with_nulls_2+%3E+%3Ap2+or+%28sortable_with_nulls_2+%3D+%3Ap2+and+%28%28pk1+%3E+%3Ap0%29%0D%0A++or%0D%0A%28pk1+%3D+%3Ap0+and+pk2+%3E+%3Ap1%29%29%29%29+order+by+sortable_with_nulls_2%2C+pk1%2C+pk2+limit+101&p2=0.2650566289400591&p0=a&p1=u\r\n\r\nRewritten with a subselect:\r\n\r\n```sql\r\nselect * from (\r\n select pk1, pk2, content, sortable, sortable_with_nulls, sortable_with_nulls_2, text from sortable\r\n)\r\nwhere (sortable_with_nulls_2 > :p2 or (sortable_with_nulls_2 = :p2 and ((pk1 > :p0)\r\n or\r\n(pk1 = :p0 and pk2 > :p1)))) order by sortable_with_nulls_2, pk1, pk2 limit 101\r\n```\r\nhttps://latest.datasette.io/fixtures?sql=select+*+from+(%0D%0A++select+pk1%2C+pk2%2C+content%2C+sortable%2C+sortable_with_nulls%2C+sortable_with_nulls_2%2C+text+from+sortable%0D%0A)%0D%0Awhere+(sortable_with_nulls_2+%3E+%3Ap2+or+(sortable_with_nulls_2+%3D+%3Ap2+and+((pk1+%3E+%3Ap0)%0D%0A++or%0D%0A(pk1+%3D+%3Ap0+and+pk2+%3E+%3Ap1))))+order+by+sortable_with_nulls_2%2C+pk1%2C+pk2+limit+101&p2=0.2650566289400591&p0=a&p1=u\r\n\r\nAnd here's the explain for that - also 49 steps: https://latest.datasette.io/fixtures?sql=explain+select+*+from+%28%0D%0A++select+pk1%2C+pk2%2C+content%2C+sortable%2C+sortable_with_nulls%2C+sortable_with_nulls_2%2C+text+from+sortable%0D%0A%29%0D%0Awhere+%28sortable_with_nulls_2+%3E+%3Ap2+or+%28sortable_with_nulls_2+%3D+%3Ap2+and+%28%28pk1+%3E+%3Ap0%29%0D%0A++or%0D%0A%28pk1+%3D+%3Ap0+and+pk2+%3E+%3Ap1%29%29%29%29+order+by+sortable_with_nulls_2%2C+pk1%2C+pk2+limit+101&p2=0.2650566289400591&p0=a&p1=u", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1573424830, "label": "Refactor out the keyset pagination code"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2019#issuecomment-1420094396", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2019", "id": 1420094396, "node_id": "IC_kwDOBm6k_c5UpOu8", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-02-07T02:18:11Z", "updated_at": "2023-02-07T02:19:16Z", "author_association": "OWNER", "body": "For the SQL underlying this page (the second page in that compound primary key paginated sequence): https://latest.datasette.io/fixtures/compound_three_primary_keys?_next=a%2Cd%2Cv\r\n\r\nThe explain for the default query: https://latest.datasette.io/fixtures?sql=explain+select%0D%0A++pk1%2C%0D%0A++pk2%2C%0D%0A++pk3%2C%0D%0A++content%0D%0Afrom%0D%0A++compound_three_primary_keys%0D%0Awhere%0D%0A++%28%0D%0A++++%28pk1+%3E+%3Ap0%29%0D%0A++++or+%28%0D%0A++++++pk1+%3D+%3Ap0%0D%0A++++++and+pk2+%3E+%3Ap1%0D%0A++++%29%0D%0A++++or+%28%0D%0A++++++pk1+%3D+%3Ap0%0D%0A++++++and+pk2+%3D+%3Ap1%0D%0A++++++and+pk3+%3E+%3Ap2%0D%0A++++%29%0D%0A++%29%0D%0Aorder+by%0D%0A++pk1%2C%0D%0A++pk2%2C%0D%0A++pk3%0D%0Alimit%0D%0A++101&p0=a&p1=d&p2=v\r\n\r\nThe explain for that query rewritten as this:\r\n\r\n```sql\r\nexplain\r\nselect\r\n *\r\nfrom\r\n (\r\n select\r\n pk1,\r\n pk2,\r\n pk3,\r\n content\r\n from\r\n compound_three_primary_keys\r\n )\r\nwhere\r\n (\r\n (pk1 > :p0)\r\n or (\r\n pk1 = :p0\r\n and pk2 > :p1\r\n )\r\n or (\r\n pk1 = :p0\r\n and pk2 = :p1\r\n and pk3 > :p2\r\n )\r\n )\r\norder by\r\n pk1,\r\n pk2,\r\n pk3\r\nlimit\r\n 101\r\n```\r\nhttps://latest.datasette.io/fixtures?sql=explain+select+*+from+%28select+%0D%0A++pk1%2C%0D%0A++pk2%2C%0D%0A++pk3%2C%0D%0A++content%0D%0Afrom%0D%0A++compound_three_primary_keys%0D%0A%29%0D%0A++where%0D%0A++%28%0D%0A++++%28pk1+%3E+%3Ap0%29%0D%0A++++or+%28%0D%0A++++++pk1+%3D+%3Ap0%0D%0A++++++and+pk2+%3E+%3Ap1%0D%0A++++%29%0D%0A++++or+%28%0D%0A++++++pk1+%3D+%3Ap0%0D%0A++++++and+pk2+%3D+%3Ap1%0D%0A++++++and+pk3+%3E+%3Ap2%0D%0A++++%29%0D%0A++%29%0D%0Aorder+by%0D%0A++pk1%2C%0D%0A++pk2%2C%0D%0A++pk3%0D%0Alimit%0D%0A++101&p0=a&p1=d&p2=v\r\n\r\nBoth explains have 31 steps and look pretty much identical.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1573424830, "label": "Refactor out the keyset pagination code"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2019#issuecomment-1420088670", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2019", "id": 1420088670, "node_id": "IC_kwDOBm6k_c5UpNVe", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-02-07T02:14:35Z", "updated_at": "2023-02-07T02:14:35Z", "author_association": "OWNER", "body": "Maybe the correct level of abstraction here is that pagination is something that happens to a SQL query that is defined as SQL and params, without an order by or limit. That's then wrapped in a sub-select and those things are added to it, plus the necessary `where` clauses depending on the page.\r\n\r\nNeed to check that the query plan for pagination of a subquery isn't slower than the plan for pagination as it works today.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1573424830, "label": "Refactor out the keyset pagination code"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2019#issuecomment-1419953256", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2019", "id": 1419953256, "node_id": "IC_kwDOBm6k_c5UosRo", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-02-06T23:42:56Z", "updated_at": "2023-02-06T23:43:10Z", "author_association": "OWNER", "body": "Relevant issue:\r\n- https://github.com/simonw/datasette/issues/1773\r\n\r\nExplains this comment: https://github.com/simonw/datasette/blob/0b4a28691468b5c758df74fa1d72a823813c96bf/datasette/views/table.py#L697\r\n\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1573424830, "label": "Refactor out the keyset pagination code"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2019#issuecomment-1419928455", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2019", "id": 1419928455, "node_id": "IC_kwDOBm6k_c5UomOH", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-02-06T23:21:50Z", "updated_at": "2023-02-06T23:21:50Z", "author_association": "OWNER", "body": "Found more logic relating to this:\r\n\r\nhttps://github.com/simonw/datasette/blob/0b4a28691468b5c758df74fa1d72a823813c96bf/datasette/views/table.py#L684-L732", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1573424830, "label": "Refactor out the keyset pagination code"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2019#issuecomment-1419921228", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2019", "id": 1419921228, "node_id": "IC_kwDOBm6k_c5UokdM", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-02-06T23:14:15Z", "updated_at": "2023-02-06T23:14:15Z", "author_association": "OWNER", "body": "Crucial utility function: https://github.com/simonw/datasette/blob/0b4a28691468b5c758df74fa1d72a823813c96bf/datasette/utils/__init__.py#L137-L160", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1573424830, "label": "Refactor out the keyset pagination code"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2019#issuecomment-1419917661", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2019", "id": 1419917661, "node_id": "IC_kwDOBm6k_c5Uojld", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-02-06T23:10:51Z", "updated_at": "2023-02-06T23:10:51Z", "author_association": "OWNER", "body": "I should turn `sort` and `sort_desc` into an object representing the sort order earlier in the code.\r\n\r\nI should also create something that bundles together `pks` and `use_rowid` and maybe `is_view` as well.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1573424830, "label": "Refactor out the keyset pagination code"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2019#issuecomment-1419916684", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2019", "id": 1419916684, "node_id": "IC_kwDOBm6k_c5UojWM", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-02-06T23:09:51Z", "updated_at": "2023-02-06T23:10:13Z", "author_association": "OWNER", "body": "The inputs and outputs for this are pretty complex.\r\n\r\nInputs:\r\n\r\n- `?_next=` from the query string\r\n- `is_view` - is this for a table or view? If it's a view it uses offset/limit pagination - which could actually work for arbitrary queries too. Also views could have keyset pagination if they are known to be sorted by a particular column.\r\n- `sort` and `sort_desc` reflecting the current sort order\r\n- `use_rowid` for if the table is a rowid table with no primary key of its own\r\n- `pks` - the primary keys for the table\r\n- `params` - the current set of parameters, I think used just to count their length so new params can be added as `p5` etc without collisions. This could be handled with a `s0`, `s1` etc naming convention instead.\r\n\r\nOutputs:\r\n\r\n- `where_clauses` - a list of where clauses to add to the query\r\n- `params` - additional parameters to use with the query due to the new where clauses\r\n- `order_by` - the order by clause", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1573424830, "label": "Refactor out the keyset pagination code"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1399343659", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1399343659, "node_id": "IC_kwDOBm6k_c5TaEor", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-01-21T22:19:20Z", "updated_at": "2023-02-06T23:02:12Z", "author_association": "OWNER", "body": "HTML mode needs a list of renderers so it can show links to `.geojson` etc - can do that as a hidden extra (maybe called `renderers`), repeating this code:\r\n\r\nhttps://github.com/simonw/datasette/blob/e4ebef082de90db4e1b8527abc0d582b7ae0bc9d/datasette/views/base.py#L477-L497", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/262#issuecomment-1418288327", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/262", "id": 1418288327, "node_id": "IC_kwDOBm6k_c5UiVzH", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-02-05T22:57:58Z", "updated_at": "2023-02-06T23:01:15Z", "author_association": "OWNER", "body": "I think that does make sense: `?_extra=table` perhaps, which would add `{\"table\": \"...\"}`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 323658641, "label": "Add ?_extra= mechanism for requesting extra properties in JSON"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2016#issuecomment-1418288077", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2016", "id": 1418288077, "node_id": "IC_kwDOBm6k_c5UiVvN", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-02-05T22:56:43Z", "updated_at": "2023-02-05T22:56:43Z", "author_association": "OWNER", "body": "This absolutely makes sense. One of the biggest goals for Datasette 1.0 is \"documented template contexts\" - for any default template in Datasette that people might want to over-ride there should be documentation that describes the available context variables, plus tests that ensure they don't accidentally get broken by future changes.\r\n\r\nEnsuring description/title/etc are available on the index page feels like it fits well into that bucket.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1571207083, "label": "Database metadata fields like description are not available in the index page template's context"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2011#issuecomment-1410827249", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2011", "id": 1410827249, "node_id": "IC_kwDOBm6k_c5UF4Px", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-01-31T17:58:54Z", "updated_at": "2023-01-31T17:58:54Z", "author_association": "OWNER", "body": "I think this is the relevant code:\r\n\r\nhttps://github.com/simonw/datasette/blob/0b4a28691468b5c758df74fa1d72a823813c96bf/datasette/facets.py#L260-L268", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1564769997, "label": "Applied facet did not result in an \"x\" icon to dismiss it"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2010#issuecomment-1409406327", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2010", "id": 1409406327, "node_id": "IC_kwDOBm6k_c5UAdV3", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-01-30T21:51:58Z", "updated_at": "2023-01-30T21:51:58Z", "author_association": "OWNER", "body": "Here's a quick prototype I knocked up for this:\r\n```diff\r\ndiff --git a/datasette/static/app.css b/datasette/static/app.css\r\nindex 71437bd4..d763bcff 100644\r\n--- a/datasette/static/app.css\r\n+++ b/datasette/static/app.css\r\n@@ -695,7 +695,48 @@ p.zero-results {\r\n \r\n \r\n \r\n+/* Force table to not be like tables anymore */\r\n+body.row table.rows-and-columns,\r\n+body.row .rows-and-columns thead,\r\n+body.row .rows-and-columns tbody,\r\n+body.row .rows-and-columns th,\r\n+body.row .rows-and-columns td,\r\n+body.row .rows-and-columns tr {\r\n+ display: block;\r\n+}\r\n+\r\n+/* Hide table headers (but not display: none;, for accessibility) */\r\n+body.row .rows-and-columns thead tr {\r\n+ position: absolute;\r\n+ top: -9999px;\r\n+ left: -9999px;\r\n+}\r\n+\r\n+body.row .rows-and-columns tr {\r\n+ border: 1px solid #ccc;\r\n+ margin-bottom: 1em;\r\n+ border-radius: 10px;\r\n+ background-color: white;\r\n+ padding: 0.2rem;\r\n+}\r\n \r\n+body.row .rows-and-columns td {\r\n+ /* Behave like a \"row\" */\r\n+ border: none;\r\n+ border-bottom: 1px solid #eee;\r\n+ padding: 0;\r\n+ padding-left: 10%;\r\n+ padding-bottom: 0.3em;\r\n+}\r\n+\r\n+body.row .rows-and-columns td:before {\r\n+ display: block;\r\n+ color: black;\r\n+ padding-bottom: 0.2em;\r\n+ font-size: 0.8em;\r\n+ font-weight: bold;\r\n+ background-color: #f5f5f5;\r\n+}\r\n \r\n \r\n /* Overrides ===============================================================*/\r\ndiff --git a/datasette/templates/row.html b/datasette/templates/row.html\r\nindex 1d1b0bfd..339eb643 100644\r\n--- a/datasette/templates/row.html\r\n+++ b/datasette/templates/row.html\r\n@@ -5,6 +5,9 @@\r\n {% block extra_head %}\r\n {{- super() -}}\r\n