{"html_url": "https://github.com/simonw/datasette/issues/2035#issuecomment-1492206593", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2035", "id": 1492206593, "node_id": "IC_kwDOBm6k_c5Y8UQB", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-31T16:09:08Z", "updated_at": "2023-03-31T16:09:08Z", "author_association": "OWNER", "body": "I could ship this as part of:\n- #2049 ", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1615692818, "label": "Potential feature: special support for `?a=1&a=2` on the query page"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2035#issuecomment-1460808028", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2035", "id": 1460808028, "node_id": "IC_kwDOBm6k_c5XEilc", "user": {"value": 1176293, "label": "ar-jan"}, "created_at": "2023-03-08T20:14:47Z", "updated_at": "2023-03-08T20:14:47Z", "author_association": "NONE", "body": "+1, I have been wishing for this feature (also for use with template-sql). It was requested before here #1304.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1615692818, "label": "Potential feature: special support for `?a=1&a=2` on the query page"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2035#issuecomment-1460682625", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2035", "id": 1460682625, "node_id": "IC_kwDOBm6k_c5XED-B", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T18:40:57Z", "updated_at": "2023-03-08T18:40:57Z", "author_association": "OWNER", "body": "Pushed that prototype to a branch: https://github.com/simonw/datasette/commit/0fe844e9adb006a0138e83102ced1329d9155c59 / https://github.com/simonw/datasette/compare/sql-list-parameters", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1615692818, "label": "Potential feature: special support for `?a=1&a=2` on the query page"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2035#issuecomment-1460679434", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2035", "id": 1460679434, "node_id": "IC_kwDOBm6k_c5XEDMK", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T18:39:35Z", "updated_at": "2023-03-08T18:39:35Z", "author_association": "OWNER", "body": "I should consider the existing design of magic parameters here: https://docs.datasette.io/en/stable/sql_queries.html#magic-parameters\r\n\r\n- `_actor_*`\r\n- `_header_*`\r\n- `_cookie_`\r\n- `_now_epoch`\r\n- `_now_date_utc`\r\n- `_now_datetime_utc`\r\n- `_random_chars_*`\r\n\r\nShould this new `id__list` syntax look more like those magic parameters, or is it OK to use `name__magic` syntax here instead?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1615692818, "label": "Potential feature: special support for `?a=1&a=2` on the query page"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2035#issuecomment-1460668431", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2035", "id": 1460668431, "node_id": "IC_kwDOBm6k_c5XEAgP", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T18:35:34Z", "updated_at": "2023-03-08T18:35:34Z", "author_association": "OWNER", "body": "To implement this properly need to do the following:\r\n- Get the page to display multiple `id: [ text input here ]` fields such that re-submission works\r\n- Figure out how this should work for canned queries and for writable canned queries\r\n- Tests that cover queries, canned queries, writable canned queries\r\n\r\nAnd a bonus feature: what if the Datasette UI layer spotted `:id__list` parameters and used them to add a bit of JavaScript that allowed users to click a `+` button next to an `id` form field to add another one?\r\n\r\nAlso, when a page is re-displayed for on of these queries it could potentially add an extra form field allowing people to add another value.\r\n\r\nThough this has an annoying problem: how to tell the difference between an additional `id` input field that the user chose not to populate, v.s. one that is supposed to represent an empty string?\r\n\r\nMaybe only support multiple `id` fields for users with JavaScript in order to avoid this problem.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1615692818, "label": "Potential feature: special support for `?a=1&a=2` on the query page"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2035#issuecomment-1460664619", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2035", "id": 1460664619, "node_id": "IC_kwDOBm6k_c5XD_kr", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T18:32:29Z", "updated_at": "2023-03-08T18:32:29Z", "author_association": "OWNER", "body": "Got a prototype working:\r\n```diff\r\ndiff --git a/datasette/views/database.py b/datasette/views/database.py\r\nindex 8d289105..6f9d8a44 100644\r\n--- a/datasette/views/database.py\r\n+++ b/datasette/views/database.py\r\n@@ -226,6 +226,12 @@ class QueryView(DataView):\r\n ):\r\n db = await self.ds.resolve_database(request)\r\n database = db.name\r\n+ # Disallow x__list query string parameters\r\n+ invalid_params = [k for k in request.args if k.endswith(\"__list\")]\r\n+ if invalid_params:\r\n+ raise DatasetteError(\r\n+ \"Invalid query string parameters: {}\".format(\", \".join(invalid_params))\r\n+ )\r\n params = {key: request.args.get(key) for key in request.args}\r\n if \"sql\" in params:\r\n params.pop(\"sql\")\r\n@@ -258,6 +264,11 @@ class QueryView(DataView):\r\n for named_parameter in named_parameters\r\n if not named_parameter.startswith(\"_\")\r\n }\r\n+ # Handle any __list parameters\r\n+ for named_parameter in named_parameters:\r\n+ if named_parameter.endswith(\"__list\"):\r\n+ list_values = request.args.getlist(named_parameter[:-6])\r\n+ params[named_parameter] = json.dumps(list_values)\r\n \r\n # Set to blank string if missing from params\r\n for named_parameter in named_parameters:\r\n```\r\nThis isn't yet doing the right thing on form re-submission: it breaks because it attempts to pass through the `?id__list=` invalid parameter. But I did manage to get it to do this through careful editing of the URL:\r\n\r\n\"image\"\r\n\r\nThat was this URL: `http://127.0.0.1:8034/content?sql=select+%3Aid__list%2C*+from+releases+where+id+in+(select+value+from+json_each(%3Aid__list))&id=62642726&id=18402901&id=38714866`", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1615692818, "label": "Potential feature: special support for `?a=1&a=2` on the query page"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2035#issuecomment-1460659382", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2035", "id": 1460659382, "node_id": "IC_kwDOBm6k_c5XD-S2", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T18:28:00Z", "updated_at": "2023-03-08T18:28:00Z", "author_association": "OWNER", "body": "Also: `datasette-explain` may need to be updated to understand how to handle this:\r\n\r\n`ERROR: conn=, sql = 'explain select * from releases where id in (select id from json_each(:id__list))', params = None: You did not supply a value for binding parameter :id__list.`\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1615692818, "label": "Potential feature: special support for `?a=1&a=2` on the query page"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2035#issuecomment-1460654136", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2035", "id": 1460654136, "node_id": "IC_kwDOBm6k_c5XD9A4", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T18:25:46Z", "updated_at": "2023-03-08T18:25:46Z", "author_association": "OWNER", "body": "Trickiest part of the implementation here is that it needs to know to output three `id` HTML form fields on the page, such that their values are persisted when the form is submitted a second time.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1615692818, "label": "Potential feature: special support for `?a=1&a=2` on the query page"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2035#issuecomment-1460639749", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2035", "id": 1460639749, "node_id": "IC_kwDOBm6k_c5XD5gF", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T18:17:31Z", "updated_at": "2023-03-08T18:17:31Z", "author_association": "OWNER", "body": "Since we are pre-1.0 it's still OK to implement a feature that disallows `?id__list=` in the URL, but allows `:id__list` in SQL queries to reference the JSON list of parameters.\r\n\r\nSo I'm going to prototype this as the `:id__list` feature and see how it feels.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1615692818, "label": "Potential feature: special support for `?a=1&a=2` on the query page"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2035#issuecomment-1460637906", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2035", "id": 1460637906, "node_id": "IC_kwDOBm6k_c5XD5DS", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T18:16:31Z", "updated_at": "2023-03-08T18:16:31Z", "author_association": "OWNER", "body": "I'm pretty sold on this as a feature now. The main question I have is which of these options to implement:\r\n\r\n1. `?id=1&?id=2` results in `:id` in the query being `[\"1\", \"2\"]` - no additional syntax required\r\n2. `:id` in the query continues to reference just the first of those parameters - but `:id__list` (or some other custom syntax) instead gets `[\"1\", \"2\"]` - or, if the URL is `?id=1` - gets `[\"1\"]`\r\n\r\nActually on writing these out I realize that option 2 is the ONLY valid option. It's no good building a query that works against a JSON list if the user might pass just a single ID, `?id=1`, resulting in their query breaking.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1615692818, "label": "Potential feature: special support for `?a=1&a=2` on the query page"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2035#issuecomment-1460632758", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2035", "id": 1460632758, "node_id": "IC_kwDOBm6k_c5XD3y2", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T18:13:49Z", "updated_at": "2023-03-08T18:13:49Z", "author_association": "OWNER", "body": "https://github.com/rclement/datasette-dashboards/issues/54 makes the excellent point that the `\r\n \r\n \r\n \r\n \r\n\r\n\r\n\r\n```\r\nSubmitting that form with the middle two options selected navigates to: `https://www.example.com/?id=32&id=15`", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1615692818, "label": "Potential feature: special support for `?a=1&a=2` on the query page"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2035#issuecomment-1460628199", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2035", "id": 1460628199, "node_id": "IC_kwDOBm6k_c5XD2rn", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T18:11:31Z", "updated_at": "2023-03-08T18:11:31Z", "author_association": "OWNER", "body": "One variant on this idea: maybe you have to specify in your query that you want it to be the JSON list version, not the single item (first `?id=` parameter version)? Maybe with syntax like this:\r\n\r\n where id in (select value from json_each(:id__list))\r\n\r\nDatasette would automatically pass `{\"id\": \"11\", \"id__list\": '[\"11\", \"32\", \"62\"]'}` as arguments to the `db.execute()` method, if the page was called with `?id=11&id=32&id=62`.\r\n\r\nThis is more explicit, though the syntax is a bit uglier (maybe there's a nicer design for this?). I also worry about `?id__list=` conflicting with this, but I think that's a risk I can take - tell people not to do that, or even block `?id__list=` style parameters entirely.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1615692818, "label": "Potential feature: special support for `?a=1&a=2` on the query page"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2035#issuecomment-1460621871", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2035", "id": 1460621871, "node_id": "IC_kwDOBm6k_c5XD1Iv", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T18:08:25Z", "updated_at": "2023-03-08T18:09:04Z", "author_association": "OWNER", "body": "My current preferred solution is to lean into SQLite's JSON support.\r\n\r\nWhat if the query page spotted `?id=11&id=32&id=62` and turned that into a JSON string called `:id:` with a value of `[\"11\", \"32\", \"62\"]`?\r\n\r\nNote that this is still a string, not a list. This avoids a nasty problem that occurred in PHP world, where `?id[]=1&id[]=2` would result in an actual PHP array object, which often broke underlying code that had expected `$_GET[\"id\"]` to be a string, not an array.\r\n\r\nSo in a query you'd be able to do this:\r\n\r\n where id in (select value from json_each(:id))\r\n\r\nAnd then call it with `?id=11&id=32&id=62`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1615692818, "label": "Potential feature: special support for `?a=1&a=2` on the query page"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2035#issuecomment-1460618433", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2035", "id": 1460618433, "node_id": "IC_kwDOBm6k_c5XD0TB", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T18:06:34Z", "updated_at": "2023-03-08T18:06:34Z", "author_association": "OWNER", "body": "One way to do this would be to dynamically generate the `where id in (?, ?, ?)` with the correct number of question marks, then feed in a list from `request.args.getlist(\"id\")` - but that would require rewriting the SQL query text to add those question marks.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1615692818, "label": "Potential feature: special support for `?a=1&a=2` on the query page"}, "performed_via_github_app": null}