{"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1461047607", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1461047607, "node_id": "IC_kwDOBm6k_c5XFdE3", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T23:51:46Z", "updated_at": "2023-03-08T23:51:46Z", "author_association": "OWNER", "body": "This feels quite nice:\r\n\r\n<img width=\"906\" alt=\"image\" src=\"https://user-images.githubusercontent.com/9599/223879235-b5ac71d9-213c-48ff-b17f-077db5b16e90.png\">\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1461044477", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1461044477, "node_id": "IC_kwDOBm6k_c5XFcT9", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T23:47:26Z", "updated_at": "2023-03-08T23:47:26Z", "author_association": "OWNER", "body": "I want to package together all of the extras that are needed for the HTML format. A few options for doing that:\r\n\r\n- Introduce `?_extra=_html` where the leading underscore indicates that this is a \"bundle\" of extras, then define a bundle that's everything needed for the HTML renderer\r\n- Have some other mechanism whereby different renderers can request a bundle of extras.\r\n\r\nI'm leaning towards the first option. I'll try that and see what it looks like.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1461023559", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1461023559, "node_id": "IC_kwDOBm6k_c5XFXNH", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T23:23:02Z", "updated_at": "2023-03-08T23:23:02Z", "author_association": "OWNER", "body": "To get this unblocked, I'm going to allow myself to pass non-JSON-serializable objects to the HTML template version of things. If I can get that working (and get the existing tests to pass) I can consider a later change that makes those JSON serializable - or admit that it's OK for the templates to have non-JSON data passed to them and figure out how best to document those variables independently from the JSON documentation.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1461002039", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1461002039, "node_id": "IC_kwDOBm6k_c5XFR83", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T22:58:16Z", "updated_at": "2023-03-08T23:02:09Z", "author_association": "OWNER", "body": "The reason for that `Row` thing is that it allows custom templates that do things like this:\r\n\r\nhttps://docs.datasette.io/en/stable/changelog.html#easier-custom-templates-for-table-rows\r\n\r\n```html+jinja\r\n{% for row in display_rows %}\r\n    <div>\r\n        <h2>{{ row[\"title\"] }}</h2>\r\n        <p>{{ row[\"description\"] }}<lp>\r\n        <p>Category: {{ row.display(\"category_id\") }}</p>\r\n    </div>\r\n{% endfor %}\r\n```\r\nIs that a good design? the `.display()` thing feels weird - I wonder if anyone has ever actually used that.\r\n\r\nIt's documented here: https://docs.datasette.io/en/0.64.2/custom_templates.html#custom-templates\r\n\r\n> If you want to output the rendered HTML version of a column, including any links to foreign keys, you can use `{{ row.display(\"column_name\") }}`.\r\n\r\nI can't see any examples of anyone using it in this code search: https://cs.github.com/?scopeName=All+repos&scope=&q=datasette+row.display\r\n\r\nIt is however useful to have some kind of abstraction layer here that insulates the SQLite `Row` object, since having an extra layer will help if Datasette ever grows support for alternative database backends such as DuckDB or PostgreSQL.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1460988975", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1460988975, "node_id": "IC_kwDOBm6k_c5XFOwv", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T22:42:57Z", "updated_at": "2023-03-08T22:42:57Z", "author_association": "OWNER", "body": "Aside idea: it might be interesting if there were \"lazy\" template variables available in the context: things that are not actually executed unless a template author requests them.\r\n\r\nImagine if `metadata` was a lazy template reference, such that custom templates that don't display any metadata don't trigger it to be resolved (which might involve additional database queries some day).", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1460986533", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1460986533, "node_id": "IC_kwDOBm6k_c5XFOKl", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T22:40:28Z", "updated_at": "2023-03-08T22:40:28Z", "author_association": "OWNER", "body": "Figuring out what to do with `display_columns_and_rows()` is hard. That returns rows as this special kind of object, which is designed to be accessed from the HTML templates:\r\n\r\nhttps://github.com/simonw/datasette/blob/96e94f9b7b2db53865e61390bcce6761727f26d8/datasette/views/table.py#L45-L71", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1460970807", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1460970807, "node_id": "IC_kwDOBm6k_c5XFKU3", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T22:31:49Z", "updated_at": "2023-03-08T22:33:03Z", "author_association": "OWNER", "body": "For the HTML version, I need to decide where all of the stuff that happens in `async def extra_template()` is going to live.\r\n\r\nI think it's another one of those extra functions, triggered for `?_extra=context`.\r\n\r\nhttps://github.com/simonw/datasette/blob/96e94f9b7b2db53865e61390bcce6761727f26d8/datasette/views/table.py#L813-L912", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1460943097", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1460943097, "node_id": "IC_kwDOBm6k_c5XFDj5", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T22:09:24Z", "updated_at": "2023-03-08T22:09:47Z", "author_association": "OWNER", "body": "The ease with which I added that  `?_extra=query` feature in https://github.com/simonw/datasette/pull/1999/commits/96e94f9b7b2db53865e61390bcce6761727f26d8 made me feel really confident that this architecture is going in the right direction.\r\n\r\n```diff\r\ndiff --git a/datasette/views/table.py b/datasette/views/table.py\r\nindex 8d3bb2c930..3e1db9c85f 100644\r\n--- a/datasette/views/table.py\r\n+++ b/datasette/views/table.py\r\n@@ -1913,6 +1913,13 @@ async def extra_request():\r\n             \"args\": request.args._data,\r\n         }\r\n \r\n+    async def extra_query():\r\n+        \"Details of the underlying SQL query\"\r\n+        return {\r\n+            \"sql\": sql,\r\n+            \"params\": params,\r\n+        }\r\n+\r\n     async def extra_extras():\r\n         \"Available ?_extra= blocks\"\r\n         return {\r\n@@ -1938,6 +1945,7 @@ async def extra_extras():\r\n         extra_primary_keys,\r\n         extra_debug,\r\n         extra_request,\r\n+        extra_query,\r\n         extra_extras,\r\n     )\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1460916405", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1460916405, "node_id": "IC_kwDOBm6k_c5XE9C1", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T21:43:27Z", "updated_at": "2023-03-08T21:43:27Z", "author_association": "OWNER", "body": "Just noticed that `_json=colname` is not working, and that's because it's handled by the renderer here:\r\n\r\nhttps://github.com/simonw/datasette/blob/56b0758a5fbf85d01ff80a40c9b028469d7bb65f/datasette/renderer.py#L29-L40\r\n\r\nBut that's not currently being called by my new code.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1460907148", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1460907148, "node_id": "IC_kwDOBm6k_c5XE6yM", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T21:34:30Z", "updated_at": "2023-03-08T21:34:30Z", "author_association": "OWNER", "body": "I'm going to hold off on that refactor until later, when I have tests to show me if the refactor works or not.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1460906741", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1460906741, "node_id": "IC_kwDOBm6k_c5XE6r1", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T21:34:08Z", "updated_at": "2023-03-08T21:34:08Z", "author_association": "OWNER", "body": "So maybe I can refactor it to look a bit more like this:\r\n\r\nhttps://github.com/simonw/datasette/blob/db1a88f4e17a1f50bdaa681e8beddb2276503e7c/datasette/views/table.py#L1602-L1604\r\n\r\nOne thing that's useful here is that `is_view` is handled early, like this:\r\n\r\nhttps://github.com/simonw/datasette/blob/db1a88f4e17a1f50bdaa681e8beddb2276503e7c/datasette/views/table.py#L466-L472\r\n\r\nSo if I omit the `is_view` bit from the extracted function I can simplify more.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1460905469", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1460905469, "node_id": "IC_kwDOBm6k_c5XE6X9", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T21:32:56Z", "updated_at": "2023-03-08T21:32:56Z", "author_association": "OWNER", "body": "I'd really like to extract this ugly logic out into a helper function:\r\n\r\nhttps://github.com/simonw/datasette/blob/db1a88f4e17a1f50bdaa681e8beddb2276503e7c/datasette/views/table.py#L1617-L1684\r\n\r\nI copied it in and asked ChatGPT to \"List all of the variable that are either modified or assigned to by the above code\":\r\n\r\n> Modified variables:\r\n> \r\n>     offset\r\n>     where_clauses\r\n>     params\r\n>     order_by\r\n> \r\n> Assigned variables:\r\n> \r\n>     sort_value\r\n>     next_by_pk_clauses\r\n>     order_by (when sort or sort_desc is not None)\r\n\r\nThen I asked which variables were used as inputs, and argued with it a bit about whether it should be counting functions. Eventually got to this:\r\n\r\n> My apologies for the oversight. Here are the variables needed as input by the above code, excluding any functions:\r\n> \r\n>     _next\r\n>     is_view\r\n>     sort\r\n>     sort_desc\r\n>     use_rowid\r\n>     pks\r\n>     order_by\r\n\r\nNote that `use_rowid` is actually defined earlier in terms of two of those other variables: https://github.com/simonw/datasette/blob/db1a88f4e17a1f50bdaa681e8beddb2276503e7c/datasette/views/table.py#L1540", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/2036#issuecomment-1460866243", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2036", "id": 1460866243, "node_id": "IC_kwDOBm6k_c5XEwzD", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T20:57:34Z", "updated_at": "2023-03-08T20:57:34Z", "author_association": "OWNER", "body": "This fix is released in 0.64.2 https://docs.datasette.io/en/stable/changelog.html#v0-64-2", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1615862295, "label": "`publish cloudrun` reuses image tags, which can lead to very surprising deploy problems"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/2036#issuecomment-1460848869", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2036", "id": 1460848869, "node_id": "IC_kwDOBm6k_c5XEsjl", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T20:40:55Z", "updated_at": "2023-03-08T20:40:55Z", "author_association": "OWNER", "body": "Here's the https://latest.datasette.io/ deployment that just went out, further demonstrating that this change is working correctly:\r\n\r\n<img width=\"565\" alt=\"image\" src=\"https://user-images.githubusercontent.com/9599/223844709-bc01813f-b4da-4bb9-bdb8-7be839700a58.png\">\r\n\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1615862295, "label": "`publish cloudrun` reuses image tags, which can lead to very surprising deploy problems"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/2037#issuecomment-1460840620", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2037", "id": 1460840620, "node_id": "IC_kwDOBm6k_c5XEqis", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T20:33:00Z", "updated_at": "2023-03-08T20:33:00Z", "author_association": "OWNER", "body": "Got the same failure again for a recent commit: https://github.com/simonw/datasette/actions/runs/4368239376/jobs/7640567282", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1615891776, "label": "Test failure: FAILED tests/test_cli.py::test_install_requirements - FileNotFoundError"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/2037#issuecomment-1460838797", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2037", "id": 1460838797, "node_id": "IC_kwDOBm6k_c5XEqGN", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T20:31:15Z", "updated_at": "2023-03-08T20:31:15Z", "author_association": "OWNER", "body": "It's this test here:\r\n\r\nhttps://github.com/simonw/datasette/blob/1ad92a1d87d79084ebe524ed186c900ff042328c/tests/test_cli.py#L181-L189\r\n\r\nAdded in:\r\n- #2033 ", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1615891776, "label": "Test failure: FAILED tests/test_cli.py::test_install_requirements - FileNotFoundError"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/2037#issuecomment-1460838109", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2037", "id": 1460838109, "node_id": "IC_kwDOBm6k_c5XEp7d", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T20:30:36Z", "updated_at": "2023-03-08T20:30:36Z", "author_association": "OWNER", "body": "Instead of using `isolated_filesystem()` I could use a `tmpdir` fixture instead.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1615891776, "label": "Test failure: FAILED tests/test_cli.py::test_install_requirements - FileNotFoundError"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/2036#issuecomment-1460827178", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2036", "id": 1460827178, "node_id": "IC_kwDOBm6k_c5XEnQq", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T20:25:10Z", "updated_at": "2023-03-08T20:25:10Z", "author_association": "OWNER", "body": "https://console.cloud.google.com/run/detail/us-central1/new-service/revisions?project=datasette-222320 confirms that the image deployed is:\r\n\r\n<img width=\"556\" alt=\"image\" src=\"https://user-images.githubusercontent.com/9599/223841493-a104df28-5c9b-47d1-9c56-e7c7f6c61b00.png\">\r\n\r\nCompared to https://console.cloud.google.com/run/detail/us-central1/datasette-io/revisions?project=datasette-222320 which shows that `datasette.io` is running:\r\n\r\n<img width=\"539\" alt=\"image\" src=\"https://user-images.githubusercontent.com/9599/223841616-67a1bb99-b704-4c2c-9dcd-247fb8592766.png\">\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1615862295, "label": "`publish cloudrun` reuses image tags, which can lead to very surprising deploy problems"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/2036#issuecomment-1460816528", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2036", "id": 1460816528, "node_id": "IC_kwDOBm6k_c5XEkqQ", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T20:22:50Z", "updated_at": "2023-03-08T20:23:20Z", "author_association": "OWNER", "body": "Testing this manually:\r\n\r\n```\r\n% datasette publish cloudrun content.db --service new-service\r\nCreating temporary tarball archive of 2 file(s) totalling 13.8 MiB before compression.\r\nUploading tarball of [.] to [gs://datasette-222320_cloudbuild/source/1678306859.271661-805303f364144b6094cc9c8532ab5133.tgz]\r\nCreated [https://cloudbuild.googleapis.com/v1/projects/datasette-222320/locations/global/builds/290f41a4-e29a-443c-a1e5-c54513c6143d].\r\nLogs are available at [ https://console.cloud.google.com/cloud-build/builds/290f41a4-e29a-443c-a1e5-c54513c6143d?project=99025868001 ].\r\n---- REMOTE BUILD OUTPUT ----\r\nstarting build \"290f41a4-e29a-443c-a1e5-c54513c6143d\"\r\n\r\nFETCHSOURCE\r\nFetching storage object: gs://datasette-222320_cloudbuild/source/1678306859.271661-805303f364144b6094cc9c8532ab5133.tgz#1678306862810483\r\nCopying gs://datasette-222320_cloudbuild/source/1678306859.271661-805303f364144b6094cc9c8532ab5133.tgz#1678306862810483...\r\n/ [1 files][  3.9 MiB/  3.9 MiB]                                                \r\nOperation completed over 1 objects/3.9 MiB.\r\nBUILD\r\nAlready have image (with digest): gcr.io/cloud-builders/docker\r\nSending build context to Docker daemon  14.52MB\r\nStep 1/9 : FROM python:3.11.0-slim-bullseye\r\n...\r\nInstalling collected packages: rfc3986, typing-extensions, sniffio, PyYAML, python-multipart, pluggy, pint, mergedeep, MarkupSafe, itsdangerous, idna, hupper, h11, click, certifi, asgiref, aiofiles, uvicorn, Jinja2, janus, click-default-group-wheel, asgi-csrf, anyio, httpcore, httpx, datasette\r\nSuccessfully installed Jinja2-3.1.2 MarkupSafe-2.1.2 PyYAML-6.0 aiofiles-23.1.0 anyio-3.6.2 asgi-csrf-0.9 asgiref-3.6.0 certifi-2022.12.7 click-8.1.3 click-default-group-wheel-1.2.2 datasette-0.64.1 h11-0.14.0 httpcore-0.16.3 httpx-0.23.3 hupper-1.11 idna-3.4 itsdangerous-2.1.2 janus-1.0.0 mergedeep-1.3.4 pint-0.20.1 pluggy-1.0.0 python-multipart-0.0.6 rfc3986-1.5.0 sniffio-1.3.0 typing-extensions-4.5.0 uvicorn-0.20.0\r\nWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\r\n\r\n[notice] A new release of pip available: 22.3 -> 23.0.1\r\n[notice] To update, run: pip install --upgrade pip\r\nRemoving intermediate container 8ccebfebebc9\r\n ---> b972c85b38bb\r\n...\r\nSuccessfully built 606b7c286d7f\r\nSuccessfully tagged gcr.io/datasette-222320/datasette-new-service:latest\r\nPUSH\r\nPushing gcr.io/datasette-222320/datasette-new-service\r\nThe push refers to repository [gcr.io/datasette-222320/datasette-new-service]\r\n667b1dc69e5e: Preparing\r\n...\r\nd8ddfcff216f: Pushed\r\nlatest: digest: sha256:452daffb2d3d7a8579c2ab39854be285155252c9428b4c1c50caac6a3a269e3f size: 2004\r\nDONE\r\n---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\r\nID                                    CREATE_TIME                DURATION  SOURCE                                                                                          IMAGES                                                   STATUS\r\n290f41a4-e29a-443c-a1e5-c54513c6143d  2023-03-08T20:21:03+00:00  39S       gs://datasette-222320_cloudbuild/source/1678306859.271661-805303f364144b6094cc9c8532ab5133.tgz  gcr.io/datasette-222320/datasette-new-service (+1 more)  SUCCESS\r\nDeploying container to Cloud Run service [new-service] in project [datasette-222320] region [us-central1]\r\n\u2713 Deploying new service... Done.                                                                                                                                                      \r\n  \u2713 Creating Revision...                                                                                                                                                              \r\n  \u2713 Routing traffic...                                                                                                                                                                \r\n  \u2713 Setting IAM Policy...                                                                                                                                                             \r\nDone.                                                                                                                                                                                 \r\nService [new-service] revision [new-service-00001-zon] has been deployed and is serving 100 percent of traffic.\r\nService URL: https://new-service-j7hipcg4aq-uc.a.run.app\r\n```\r\nhttps://new-service-j7hipcg4aq-uc.a.run.app/ was deployed successfully.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1615862295, "label": "`publish cloudrun` reuses image tags, which can lead to very surprising deploy problems"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/2036#issuecomment-1460810523", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2036", "id": 1460810523, "node_id": "IC_kwDOBm6k_c5XEjMb", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T20:17:01Z", "updated_at": "2023-03-08T20:17:01Z", "author_association": "OWNER", "body": "I'm going to solve this by using the service name in that `image_id` instead:\r\n\r\n```python\r\nimage_id = f\"gcr.io/{project}/{service_name}\"\r\n```\r\nThis is a nasty bug, so I'm going to backport it to a `0.64.2` release as well.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1615862295, "label": "`publish cloudrun` reuses image tags, which can lead to very surprising deploy problems"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/2036#issuecomment-1460809643", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2036", "id": 1460809643, "node_id": "IC_kwDOBm6k_c5XEi-r", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T20:16:10Z", "updated_at": "2023-03-08T20:16:10Z", "author_association": "OWNER", "body": "I think the code at fault is here:\r\n\r\nhttps://github.com/simonw/datasette/blob/1ad92a1d87d79084ebe524ed186c900ff042328c/datasette/publish/cloudrun.py#L176-L182\r\n\r\nThat name ends up defaulting to `datasette` - so multiple different projects may end up deploying to the same `image_id`.\r\n\r\nWhat I think happened in the `datasette.io` bug is that this workflow: https://github.com/simonw/simonwillisonblog-backup/blob/bfb573e96d8622ab52b22fdcd54724fe6e59fd24/.github/workflows/backup.yml and this workflow: https://github.com/simonw/datasette.io/blob/4676db5bf4a3fc9f792ee270ec0c59eb902cd2c3/.github/workflows/deploy.yml both happened to run at the exact same time.\r\n\r\nAnd so the image that was pushed to `gcr.io/datasette-222320/datasette:latest` by the `simonw/simonwillisonblog-backup` action was then deployed by the `simonw/datasette.io/` action, which broke the site.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1615862295, "label": "`publish cloudrun` reuses image tags, which can lead to very surprising deploy problems"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1460760116", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1460760116, "node_id": "IC_kwDOBm6k_c5XEW40", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T19:48:52Z", "updated_at": "2023-03-08T19:48:52Z", "author_association": "OWNER", "body": "I'm trying to get `http://127.0.0.1:8001/fixtures/compound_three_primary_keys?_next=a,d,v` to return the correct results.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/2035#issuecomment-1460682625", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2035", "id": 1460682625, "node_id": "IC_kwDOBm6k_c5XED-B", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T18:40:57Z", "updated_at": "2023-03-08T18:40:57Z", "author_association": "OWNER", "body": "Pushed that prototype to a branch: https://github.com/simonw/datasette/commit/0fe844e9adb006a0138e83102ced1329d9155c59 / https://github.com/simonw/datasette/compare/sql-list-parameters", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1615692818, "label": "Potential feature: special support for `?a=1&a=2` on the query page"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/2035#issuecomment-1460679434", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2035", "id": 1460679434, "node_id": "IC_kwDOBm6k_c5XEDMK", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T18:39:35Z", "updated_at": "2023-03-08T18:39:35Z", "author_association": "OWNER", "body": "I should consider the existing design of magic parameters here: https://docs.datasette.io/en/stable/sql_queries.html#magic-parameters\r\n\r\n- `_actor_*`\r\n- `_header_*`\r\n- `_cookie_`\r\n- `_now_epoch`\r\n- `_now_date_utc`\r\n- `_now_datetime_utc`\r\n- `_random_chars_*`\r\n\r\nShould this new `id__list` syntax look more like those magic parameters, or is it OK to use `name__magic` syntax here instead?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1615692818, "label": "Potential feature: special support for `?a=1&a=2` on the query page"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/2035#issuecomment-1460668431", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2035", "id": 1460668431, "node_id": "IC_kwDOBm6k_c5XEAgP", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T18:35:34Z", "updated_at": "2023-03-08T18:35:34Z", "author_association": "OWNER", "body": "To implement this properly need to do the following:\r\n- Get the page to display multiple `id: [ text input here ]` fields such that re-submission works\r\n- Figure out how this should work for canned queries and for writable canned queries\r\n- Tests that cover queries, canned queries, writable canned queries\r\n\r\nAnd a bonus feature: what if the Datasette UI layer spotted `:id__list` parameters and used them to add a bit of JavaScript that allowed users to click a `+` button next to an `id` form field to add another one?\r\n\r\nAlso, when a page is re-displayed for on of these queries it could potentially add an extra form field allowing people to add another value.\r\n\r\nThough this has an annoying problem: how to tell the difference between an additional `id` input field that the user chose not to populate, v.s. one that is supposed to represent an empty string?\r\n\r\nMaybe only support multiple `id` fields for users with JavaScript in order to avoid this problem.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1615692818, "label": "Potential feature: special support for `?a=1&a=2` on the query page"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/2035#issuecomment-1460664619", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2035", "id": 1460664619, "node_id": "IC_kwDOBm6k_c5XD_kr", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T18:32:29Z", "updated_at": "2023-03-08T18:32:29Z", "author_association": "OWNER", "body": "Got a prototype working:\r\n```diff\r\ndiff --git a/datasette/views/database.py b/datasette/views/database.py\r\nindex 8d289105..6f9d8a44 100644\r\n--- a/datasette/views/database.py\r\n+++ b/datasette/views/database.py\r\n@@ -226,6 +226,12 @@ class QueryView(DataView):\r\n     ):\r\n         db = await self.ds.resolve_database(request)\r\n         database = db.name\r\n+        # Disallow x__list query string parameters\r\n+        invalid_params = [k for k in request.args if k.endswith(\"__list\")]\r\n+        if invalid_params:\r\n+            raise DatasetteError(\r\n+                \"Invalid query string parameters: {}\".format(\", \".join(invalid_params))\r\n+            )\r\n         params = {key: request.args.get(key) for key in request.args}\r\n         if \"sql\" in params:\r\n             params.pop(\"sql\")\r\n@@ -258,6 +264,11 @@ class QueryView(DataView):\r\n             for named_parameter in named_parameters\r\n             if not named_parameter.startswith(\"_\")\r\n         }\r\n+        # Handle any __list parameters\r\n+        for named_parameter in named_parameters:\r\n+            if named_parameter.endswith(\"__list\"):\r\n+                list_values = request.args.getlist(named_parameter[:-6])\r\n+                params[named_parameter] = json.dumps(list_values)\r\n \r\n         # Set to blank string if missing from params\r\n         for named_parameter in named_parameters:\r\n```\r\nThis isn't yet doing the right thing on form re-submission: it breaks because it attempts to pass through the `?id__list=` invalid parameter. But I did manage to get it to do this through careful editing of the URL:\r\n\r\n<img width=\"1226\" alt=\"image\" src=\"https://user-images.githubusercontent.com/9599/223803758-2a1f90bd-bc20-4a77-bdce-4644afe235ba.png\">\r\n\r\nThat was this URL: `http://127.0.0.1:8034/content?sql=select+%3Aid__list%2C*+from+releases+where+id+in+(select+value+from+json_each(%3Aid__list))&id=62642726&id=18402901&id=38714866`", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1615692818, "label": "Potential feature: special support for `?a=1&a=2` on the query page"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/2035#issuecomment-1460659382", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2035", "id": 1460659382, "node_id": "IC_kwDOBm6k_c5XD-S2", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T18:28:00Z", "updated_at": "2023-03-08T18:28:00Z", "author_association": "OWNER", "body": "Also: `datasette-explain` may need to be updated to understand how to handle this:\r\n\r\n`ERROR: conn=<sqlite3.Connection object at 0x102834940>, sql = 'explain select * from releases where id in (select id from json_each(:id__list))', params = None: You did not supply a value for binding parameter :id__list.`\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1615692818, "label": "Potential feature: special support for `?a=1&a=2` on the query page"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/2035#issuecomment-1460654136", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2035", "id": 1460654136, "node_id": "IC_kwDOBm6k_c5XD9A4", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T18:25:46Z", "updated_at": "2023-03-08T18:25:46Z", "author_association": "OWNER", "body": "Trickiest part of the implementation here is that it needs to know to output three `id` HTML form fields on the page, such that their values are persisted when the form is submitted a second time.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1615692818, "label": "Potential feature: special support for `?a=1&a=2` on the query page"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/2035#issuecomment-1460639749", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2035", "id": 1460639749, "node_id": "IC_kwDOBm6k_c5XD5gF", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T18:17:31Z", "updated_at": "2023-03-08T18:17:31Z", "author_association": "OWNER", "body": "Since we are pre-1.0 it's still OK to implement a feature that disallows `?id__list=` in the URL, but allows `:id__list` in SQL queries to reference the JSON list of parameters.\r\n\r\nSo I'm going to prototype this as the `:id__list` feature and see how it feels.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1615692818, "label": "Potential feature: special support for `?a=1&a=2` on the query page"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/2035#issuecomment-1460637906", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2035", "id": 1460637906, "node_id": "IC_kwDOBm6k_c5XD5DS", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T18:16:31Z", "updated_at": "2023-03-08T18:16:31Z", "author_association": "OWNER", "body": "I'm pretty sold on this as a feature now. The main question I have is which of these options to implement:\r\n\r\n1. `?id=1&?id=2` results in `:id` in the query being `[\"1\", \"2\"]` - no additional syntax required\r\n2. `:id` in the query continues to reference just the first of those parameters - but `:id__list` (or some other custom syntax) instead gets `[\"1\", \"2\"]` - or, if the URL is `?id=1` - gets `[\"1\"]`\r\n\r\nActually on writing these out I realize that option 2 is the ONLY valid option. It's no good building a query that works against a JSON list if the user might pass just a single ID, `?id=1`, resulting in their query breaking.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1615692818, "label": "Potential feature: special support for `?a=1&a=2` on the query page"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/2035#issuecomment-1460632758", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2035", "id": 1460632758, "node_id": "IC_kwDOBm6k_c5XD3y2", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T18:13:49Z", "updated_at": "2023-03-08T18:13:49Z", "author_association": "OWNER", "body": "https://github.com/rclement/datasette-dashboards/issues/54 makes the excellent point that the `<select multiple>` default HTML widget produces this exact format of query string:\r\n\r\n```html\r\n<form action=\"https://www.example.com/\">\r\n<select multiple name=\"id\">\r\n  <option>21</option>\r\n  <option>32</option>\r\n  <option>15</option>\r\n  <option>63</option>\r\n</select>\r\n<input type=\"submit\">\r\n</form>\r\n```\r\nSubmitting that form with the middle two options selected navigates to: `https://www.example.com/?id=32&id=15`", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1615692818, "label": "Potential feature: special support for `?a=1&a=2` on the query page"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/2035#issuecomment-1460628199", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2035", "id": 1460628199, "node_id": "IC_kwDOBm6k_c5XD2rn", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T18:11:31Z", "updated_at": "2023-03-08T18:11:31Z", "author_association": "OWNER", "body": "One variant on this idea: maybe you have to specify in your query that you want it to be the JSON list version, not the single item (first `?id=` parameter version)? Maybe with syntax like this:\r\n\r\n    where id in (select value from json_each(:id__list))\r\n\r\nDatasette would automatically pass `{\"id\": \"11\", \"id__list\": '[\"11\", \"32\", \"62\"]'}` as arguments to the `db.execute()` method, if the page was called with `?id=11&id=32&id=62`.\r\n\r\nThis is more explicit, though the syntax is a bit uglier (maybe there's a nicer design for this?). I also worry about `?id__list=` conflicting with this, but I think that's a risk I can take - tell people not to do that, or even block `?id__list=` style parameters entirely.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1615692818, "label": "Potential feature: special support for `?a=1&a=2` on the query page"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/2035#issuecomment-1460621871", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2035", "id": 1460621871, "node_id": "IC_kwDOBm6k_c5XD1Iv", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T18:08:25Z", "updated_at": "2023-03-08T18:09:04Z", "author_association": "OWNER", "body": "My current preferred solution is to lean into SQLite's JSON support.\r\n\r\nWhat if the query page spotted `?id=11&id=32&id=62` and turned that into  a JSON string called `:id:` with a value of `[\"11\", \"32\", \"62\"]`?\r\n\r\nNote that this is still a string, not a list. This avoids a nasty problem that occurred in PHP world, where `?id[]=1&id[]=2` would result in an actual PHP array object, which often broke underlying code that had expected `$_GET[\"id\"]` to be a string, not an array.\r\n\r\nSo in a query you'd be able to do this:\r\n\r\n    where id in (select value from json_each(:id))\r\n\r\nAnd then call it with `?id=11&id=32&id=62`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1615692818, "label": "Potential feature: special support for `?a=1&a=2` on the query page"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/2035#issuecomment-1460618433", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2035", "id": 1460618433, "node_id": "IC_kwDOBm6k_c5XD0TB", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T18:06:34Z", "updated_at": "2023-03-08T18:06:34Z", "author_association": "OWNER", "body": "One way to do this would be to dynamically generate the `where id in (?, ?, ?)` with the correct number of question marks, then feed in a list from `request.args.getlist(\"id\")` - but that would require rewriting the SQL query text to add those question marks.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1615692818, "label": "Potential feature: special support for `?a=1&a=2` on the query page"}, "performed_via_github_app": null}