(impact)`, `\u00f8 = not affected`, `? = missing data`\n> Powered by [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/247?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Last update [1fe73c8...af989af](https://codecov.io/gh/simonw/sqlite-utils/pull/247?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 832687563, "label": "FTS quote functionality from datasette"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/pull/247#issuecomment-901338988", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/247", "id": 901338988, "node_id": "IC_kwDOCGYnMM41uVds", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-18T18:33:39Z", "updated_at": "2021-08-18T18:33:39Z", "author_association": "OWNER", "body": "This was also requested in #296.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 832687563, "label": "FTS quote functionality from datasette"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/296#issuecomment-901338356", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/296", "id": 901338356, "node_id": "IC_kwDOCGYnMM41uVT0", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-18T18:32:39Z", "updated_at": "2021-08-18T18:32:39Z", "author_association": "OWNER", "body": "This is a good call. I have a fix for this in Datasette but it's not in `sqlite-utils` yet: https://github.com/simonw/datasette/blob/adb5b70de5cec3c3dd37184defe606a082c232cf/datasette/utils/__init__.py#L824-L835", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 944326512, "label": "`table.search(..., quote=True)` parameter and `sqlite-utils search --quote` option"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/317#issuecomment-901337305", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/317", "id": 901337305, "node_id": "IC_kwDOCGYnMM41uVDZ", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-18T18:30:59Z", "updated_at": "2021-08-18T18:30:59Z", "author_association": "OWNER", "body": "I'm just going to remove this - I added it when the library was mostly undocumented, but it has comprehensive documentation now.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 972827346, "label": "Link to a better example on docs index"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1439#issuecomment-900715375", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1439", "id": 900715375, "node_id": "IC_kwDOBm6k_c41r9Nv", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-18T00:15:28Z", "updated_at": "2021-08-18T00:15:28Z", "author_association": "OWNER", "body": "Maybe I should use `-/` to encode forward slashes too, to defend against any ASGI servers that might not implement `raw_path` correctly.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 973139047, "label": "Rethink how .ext formats (v.s. ?_format=) works before 1.0"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1439#issuecomment-900714630", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1439", "id": 900714630, "node_id": "IC_kwDOBm6k_c41r9CG", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-18T00:13:33Z", "updated_at": "2021-08-18T00:13:33Z", "author_association": "OWNER", "body": "The documentation should definitely cover how table names become URLs, in case any third party code needs to be able to calculate this themselves.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 973139047, "label": "Rethink how .ext formats (v.s. ?_format=) works before 1.0"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1439#issuecomment-900712981", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1439", "id": 900712981, "node_id": "IC_kwDOBm6k_c41r8oV", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-18T00:09:59Z", "updated_at": "2021-08-18T00:12:32Z", "author_association": "OWNER", "body": "So given the original examples, a table called `table.csv` would have the following URLs:\r\n\r\n- `/db/table-.csv` - the HTML version\r\n- `/db/table-.csv.csv` - the CSV version\r\n- `/db/table-.csv.json` - the JSON version\r\n\r\nAnd if for some horific reason you had a table with the name `/db/table-.csv.csv` (so `/db/` was the first part of the actual table name in SQLite) the URLs would look like this:\r\n\r\n- `/db/%2Fdb%2Ftable---.csv-.csv` - the HTML version\r\n- `/db/%2Fdb%2Ftable---.csv-.csv.csv` - the CSV version\r\n- `/db/%2Fdb%2Ftable---.csv-.csv.json` - the JSON version", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 973139047, "label": "Rethink how .ext formats (v.s. ?_format=) works before 1.0"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1439#issuecomment-900711967", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1439", "id": 900711967, "node_id": "IC_kwDOBm6k_c41r8Yf", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-18T00:08:09Z", "updated_at": "2021-08-18T00:08:09Z", "author_association": "OWNER", "body": "Here's an alternative I just made up which I'm calling \"dot dash\" encoding:\r\n\r\n```python\r\ndef dot_dash_encode(s):\r\n return s.replace(\"-\", \"--\").replace(\".\", \"-.\")\r\n\r\ndef dot_dash_decode(s):\r\n return s.replace(\"-.\", \".\").replace(\"--\", \"-\")\r\n```\r\nAnd some examples:\r\n```python\r\nfor example in (\r\n \"hello\",\r\n \"hello.csv\",\r\n \"hello-and-so-on.csv\",\r\n \"hello-.csv\",\r\n \"hello--and--so--on-.csv\",\r\n \"hello.csv.\",\r\n \"hello.csv.-\",\r\n \"hello.csv.--\",\r\n):\r\n print(example)\r\n print(dot_dash_encode(example))\r\n print(example == dot_dash_decode(dot_dash_encode(example)))\r\n print()\r\n```\r\nOutputs:\r\n```\r\nhello\r\nhello\r\nTrue\r\n\r\nhello.csv\r\nhello-.csv\r\nTrue\r\n\r\nhello-and-so-on.csv\r\nhello--and--so--on-.csv\r\nTrue\r\n\r\nhello-.csv\r\nhello---.csv\r\nTrue\r\n\r\nhello--and--so--on-.csv\r\nhello----and----so----on---.csv\r\nTrue\r\n\r\nhello.csv.\r\nhello-.csv-.\r\nTrue\r\n\r\nhello.csv.-\r\nhello-.csv-.--\r\nTrue\r\n\r\nhello.csv.--\r\nhello-.csv-.----\r\nTrue\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 973139047, "label": "Rethink how .ext formats (v.s. ?_format=) works before 1.0"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1439#issuecomment-900709703", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1439", "id": 900709703, "node_id": "IC_kwDOBm6k_c41r71H", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-18T00:03:09Z", "updated_at": "2021-08-18T00:03:09Z", "author_association": "OWNER", "body": "But... what if I invent my own escaping scheme?\r\n\r\nI actually did this once before, in https://github.com/simonw/datasette/commit/9fdb47ca952b93b7b60adddb965ea6642b1ff523 - while I was working on porting Datasette to ASGI in https://github.com/simonw/datasette/issues/272#issuecomment-494192779 because ASGI didn't yet have the `raw_path` mechanism.\r\n\r\nI could bring that back - it looked like this:\r\n\r\n```\r\n \"table/and/slashes\" => \"tableU+002FandU+002Fslashes\"\r\n \"~table\" => \"U+007Etable\"\r\n \"+bobcats!\" => \"U+002Bbobcats!\"\r\n \"U+007Etable\" => \"UU+002B007Etable\"\r\n```\r\nBut I didn't particularly like it - it was quite verbose.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 973139047, "label": "Rethink how .ext formats (v.s. ?_format=) works before 1.0"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1439#issuecomment-900705226", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1439", "id": 900705226, "node_id": "IC_kwDOBm6k_c41r6vK", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-17T23:50:32Z", "updated_at": "2021-08-17T23:50:47Z", "author_association": "OWNER", "body": "An alternative solution would be to use some form of escaping for the characters that form the name of the table.\r\n\r\nThe obvious way to do this would be URL-encoding - but it doesn't hold for `.` characters. The hex for that is `%2E` but watch what happens with that in a URL:\r\n\r\n```\r\n# Against Cloud Run:\r\ncurl -s 'https://datasette.io/-/asgi-scope/foo/bar%2Fbaz%2E' | rg path\r\n 'path': '/-/asgi-scope/foo/bar/baz.',\r\n 'raw_path': b'/-/asgi-scope/foo/bar%2Fbaz.',\r\n 'root_path': '',\r\n# Against Vercel:\r\ncurl -s 'https://til.simonwillison.net/-/asgi-scope/foo/bar%2Fbaz%2E' | rg path\r\n 'path': '/-/asgi-scope/foo/bar%2Fbaz%2E',\r\n 'raw_path': b'/-/asgi-scope/foo/bar%2Fbaz%2E',\r\n 'root_path': '',\r\n```\r\nSurprisingly in this case Vercel DOES keep it intact, but Cloud Run does not.\r\n\r\nIt's still no good though: I need a solution that works on Vercel, Cloud Run and every other potential hosting provider too.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 973139047, "label": "Rethink how .ext formats (v.s. ?_format=) works before 1.0"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1439#issuecomment-900699670", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1439", "id": 900699670, "node_id": "IC_kwDOBm6k_c41r5YW", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-17T23:34:23Z", "updated_at": "2021-08-17T23:34:23Z", "author_association": "OWNER", "body": "The challenge comes down to telling the difference between the following:\r\n\r\n- `/db/table` - an HTML table page\r\n- `/db/table.csv` - the CSV version of `/db/table`\r\n- `/db/table.csv` - no this one is actually a database table called `table.csv`\r\n- `/db/table.csv.csv` - the CSV version of `/db/table.csv`\r\n- `/db/table.csv.csv.csv` and so on...", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 973139047, "label": "Rethink how .ext formats (v.s. ?_format=) works before 1.0"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1438#issuecomment-900690998", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1438", "id": 900690998, "node_id": "IC_kwDOBm6k_c41r3Q2", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-17T23:11:16Z", "updated_at": "2021-08-17T23:12:25Z", "author_association": "OWNER", "body": "I have completely failed to replicate this initial bug - but it's still there on the `thesession.vercel.app` deployment (even though my own deployments to Vercel do not exhibit it). Here's a one-liner to replicate it against that deployment:\r\n\r\n`curl -s 'https://thesession.vercel.app/thesession?sql=select+*+from+tunes+where+name+like+%22%25wise+maid%25%22' | rg '.csv'`\r\n\r\nWhit outputs this:\r\n\r\n`This data as json, CSV
`\r\n\r\nIt looks like, rather than being URL-encoded, the original query string is somehow making it through to Jinja and then being auto-escaped there.\r\n\r\nThe weird thing is that the equivalent query executed against my `til.simonwillison.net` Vercel instance does this:\r\n\r\n`curl -s 'https://til.simonwillison.net/fixtures?sql=select+*+from+searchable+where+text1+like+%22%25a%25%22' | rg '.csv'`\r\n\r\n`This data as json, CSV
`", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 972918533, "label": "Query page .csv and .json links are not correctly URL-encoded on Vercel under unknown specific conditions"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1438#issuecomment-900681413", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1438", "id": 900681413, "node_id": "IC_kwDOBm6k_c41r07F", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-17T22:47:44Z", "updated_at": "2021-08-17T22:47:44Z", "author_association": "OWNER", "body": "I deployed another copy of `fixtures.db` on Vercel at https://til.simonwillison.net/fixtures so I can compare it with `fixtures.db` on Cloud Run at https://latest.datasette.io/fixtures", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 972918533, "label": "Query page .csv and .json links are not correctly URL-encoded on Vercel under unknown specific conditions"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1438#issuecomment-900518343", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1438", "id": 900518343, "node_id": "IC_kwDOBm6k_c41rNHH", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-17T18:04:42Z", "updated_at": "2021-08-17T18:04:42Z", "author_association": "OWNER", "body": "Here's how `request.query_string` works: https://github.com/simonw/datasette/blob/adb5b70de5cec3c3dd37184defe606a082c232cf/datasette/utils/asgi.py#L86-L88", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 972918533, "label": "Query page .csv and .json links are not correctly URL-encoded on Vercel under unknown specific conditions"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1438#issuecomment-900516826", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1438", "id": 900516826, "node_id": "IC_kwDOBm6k_c41rMva", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-17T18:02:27Z", "updated_at": "2021-08-17T18:02:27Z", "author_association": "OWNER", "body": "The key difference I can spot between Vercel and Cloud Run is that `+` in a query string gets converted to `%20` by Vercel before it gets to my app, but does not for Cloud Run:\r\n```\r\n# Vercel\r\n~ % curl -s 'https://til.simonwillison.net/-/asgi-scope?sql=select+*+from+tunes+where+name+like+%22%25wise+maid%25%22%0D%0A' | rg 'query_string' -C 2\r\n 'method': 'GET',\r\n 'path': '/-/asgi-scope',\r\n 'query_string': b'sql=select%20*%20from%20tunes%20where%20name%20like%20%22%25'\r\n b'wise%20maid%25%22%0D%0A',\r\n 'raw_path': b'/-/asgi-scope',\r\n\r\n# Cloud Run\r\n~ % curl -s 'https://latest-with-plugins.datasette.io/-/asgi-scope?sql=select+*+from+tunes+where+name+like+%22%25wise+maid%25%22%0D%0A' | rg 'query_string' -C 2\r\n 'method': 'GET',\r\n 'path': '/-/asgi-scope',\r\n 'query_string': b'sql=select+*+from+tunes+where+name+like+%22%25wise+maid%25%2'\r\n b'2%0D%0A',\r\n 'raw_path': b'/-/asgi-scope',\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 972918533, "label": "Query page .csv and .json links are not correctly URL-encoded on Vercel under unknown specific conditions"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1438#issuecomment-900513267", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1438", "id": 900513267, "node_id": "IC_kwDOBm6k_c41rL3z", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-17T17:57:05Z", "updated_at": "2021-08-17T17:57:05Z", "author_association": "OWNER", "body": "I'm having trouble replicating this bug outside of Vercel. Against Cloud Run: view-source:https://latest.datasette.io/fixtures?sql=select+*+from+searchable+where+text1+like+%22%25cat%25%22\r\n\r\nThe HTML here is:\r\n\r\n```html\r\nThis data as\r\n json, \r\n ...\r\n CSV\r\n
\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 972918533, "label": "Query page .csv and .json links are not correctly URL-encoded on Vercel under unknown specific conditions"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1438#issuecomment-900502364", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1438", "id": 900502364, "node_id": "IC_kwDOBm6k_c41rJNc", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-17T17:40:41Z", "updated_at": "2021-08-17T17:40:41Z", "author_association": "OWNER", "body": "Bug is likely in `path_with_format` itself: https://github.com/simonw/datasette/blob/adb5b70de5cec3c3dd37184defe606a082c232cf/datasette/utils/__init__.py#L710-L729", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 972918533, "label": "Query page .csv and .json links are not correctly URL-encoded on Vercel under unknown specific conditions"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1438#issuecomment-900500824", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1438", "id": 900500824, "node_id": "IC_kwDOBm6k_c41rI1Y", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-17T17:38:16Z", "updated_at": "2021-08-17T17:38:16Z", "author_association": "OWNER", "body": "Relevant template code: https://github.com/simonw/datasette/blob/adb5b70de5cec3c3dd37184defe606a082c232cf/datasette/templates/query.html#L71\r\n\r\n`renderers` comes from here: https://github.com/simonw/datasette/blob/2883098770fc66e50183b2b231edbde20848d4d6/datasette/views/base.py#L593-L608", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 972918533, "label": "Query page .csv and .json links are not correctly URL-encoded on Vercel under unknown specific conditions"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1293#issuecomment-899915829", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1293", "id": 899915829, "node_id": "IC_kwDOBm6k_c41o6A1", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-17T01:02:35Z", "updated_at": "2021-08-17T01:02:35Z", "author_association": "OWNER", "body": "New approach: this time I'm building a simplified executor for the bytecode operations themselves.\r\n```python\r\ndef execute_operations(operations, max_iterations = 100, trace=None):\r\n trace = trace or (lambda *args: None)\r\n registers: Dict[int, Any] = {}\r\n cursors: Dict[int, Tuple[str, Dict]] = {}\r\n instruction_pointer = 0\r\n iterations = 0\r\n result_row = None\r\n while True:\r\n iterations += 1\r\n if iterations > max_iterations:\r\n break\r\n operation = operations[instruction_pointer]\r\n trace(instruction_pointer, dict(operation))\r\n opcode = operation[\"opcode\"]\r\n if opcode == \"Init\":\r\n if operation[\"p2\"] != 0:\r\n instruction_pointer = operation[\"p2\"]\r\n continue\r\n else:\r\n instruction_pointer += 1\r\n continue\r\n elif opcode == \"Goto\":\r\n instruction_pointer = operation[\"p2\"]\r\n continue\r\n elif opcode == \"Halt\":\r\n break\r\n elif opcode == \"OpenRead\":\r\n cursors[operation[\"p1\"]] = (\"database_table\", {\r\n \"rootpage\": operation[\"p2\"],\r\n \"connection\": operation[\"p3\"],\r\n })\r\n elif opcode == \"OpenEphemeral\":\r\n cursors[operation[\"p1\"]] = (\"ephemeral\", {\r\n \"num_columns\": operation[\"p2\"],\r\n \"index_keys\": [],\r\n })\r\n elif opcode == \"MakeRecord\":\r\n registers[operation[\"p3\"]] = (\"MakeRecord\", {\r\n \"registers\": list(range(operation[\"p1\"] + operation[\"p2\"]))\r\n })\r\n elif opcode == \"IdxInsert\":\r\n record = registers[operation[\"p2\"]]\r\n cursors[operation[\"p1\"]][1][\"index_keys\"].append(record)\r\n elif opcode == \"Rowid\":\r\n registers[operation[\"p2\"]] = (\"rowid\", {\r\n \"table\": operation[\"p1\"]\r\n })\r\n elif opcode == \"Sequence\":\r\n registers[operation[\"p2\"]] = (\"sequence\", {\r\n \"next_from_cursor\": operation[\"p1\"]\r\n })\r\n elif opcode == \"Column\":\r\n registers[operation[\"p3\"]] = (\"column\", {\r\n \"cursor\": operation[\"p1\"],\r\n \"column_offset\": operation[\"p2\"]\r\n })\r\n elif opcode == \"ResultRow\":\r\n p1 = operation[\"p1\"]\r\n p2 = operation[\"p2\"]\r\n trace(\"ResultRow: \", list(range(p1, p1 + p2)), registers)\r\n result_row = [registers.get(i) for i in range(p1, p1 + p2)]\r\n elif opcode == \"Integer\":\r\n registers[operation[\"p2\"]] = (\"Integer\", operation[\"p1\"])\r\n elif opcode == \"String8\":\r\n registers[operation[\"p2\"]] = (\"String\", operation[\"p4\"])\r\n instruction_pointer += 1\r\n return {\"registers\": registers, \"cursors\": cursors, \"result_row\": result_row}\r\n```\r\nResults are promising!\r\n```\r\nexecute_operations(db.execute(\"explain select 'hello', 55, rowid, * from searchable\").fetchall())\r\n\r\n{'registers': {1: ('String', 'hello'),\r\n 2: ('Integer', 55),\r\n 3: ('rowid', {'table': 0}),\r\n 4: ('rowid', {'table': 0}),\r\n 5: ('column', {'cursor': 0, 'column_offset': 1}),\r\n 6: ('column', {'cursor': 0, 'column_offset': 2}),\r\n 7: ('column', {'cursor': 0, 'column_offset': 3})},\r\n 'cursors': {0: ('database_table', {'rootpage': 32, 'connection': 0})},\r\n 'result_row': [('String', 'hello'),\r\n ('Integer', 55),\r\n ('rowid', {'table': 0}),\r\n ('rowid', {'table': 0}),\r\n ('column', {'cursor': 0, 'column_offset': 1}),\r\n ('column', {'cursor': 0, 'column_offset': 2}),\r\n ('column', {'cursor': 0, 'column_offset': 3})]}\r\n```\r\nHere's what happens with a union across three tables:\r\n```\r\nexecute_operations(db.execute(f\"\"\"\r\nexplain select data as content from binary_data\r\nunion\r\nselect pk as content from complex_foreign_keys\r\nunion\r\nselect name as content from facet_cities\r\n\"\"\"}).fetchall())\r\n\r\n{'registers': {1: ('column', {'cursor': 4, 'column_offset': 0}),\r\n 2: ('MakeRecord', {'registers': [0, 1, 2, 3]}),\r\n 3: ('column', {'cursor': 0, 'column_offset': 1}),\r\n 4: ('column', {'cursor': 3, 'column_offset': 0})},\r\n 'cursors': {3: ('ephemeral',\r\n {'num_columns': 1,\r\n 'index_keys': [('MakeRecord', {'registers': [0, 1]}),\r\n ('MakeRecord', {'registers': [0, 1]}),\r\n ('MakeRecord', {'registers': [0, 1, 2, 3]})]}),\r\n 2: ('database_table', {'rootpage': 44, 'connection': 0}),\r\n 4: ('database_table', {'rootpage': 24, 'connection': 0}),\r\n 0: ('database_table', {'rootpage': 42, 'connection': 0})},\r\n 'result_row': [('column', {'cursor': 3, 'column_offset': 0})]}\r\n```\r\nNote how the result_row refers to cursor 3, which is an ephemeral table which had three different sets of `MakeRecord` index keys assigned to it - indicating that the output column is NOT from the same underlying table source.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 849978964, "label": "Show column metadata plus links for foreign keys on arbitrary query results"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1423#issuecomment-899749881", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1423", "id": 899749881, "node_id": "IC_kwDOBm6k_c41oRf5", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-16T19:07:02Z", "updated_at": "2021-08-16T19:07:02Z", "author_association": "OWNER", "body": "Demo: https://latest.datasette.io/fixtures/compound_three_primary_keys?_facet=content&_facet_size=max&_facet=pk1&_facet=pk2\r\n\r\n\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 962391325, "label": "Show count of facet values if ?_facet_size=max"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1423#issuecomment-899744109", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1423", "id": 899744109, "node_id": "IC_kwDOBm6k_c41oQFt", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-16T18:58:29Z", "updated_at": "2021-08-16T18:58:29Z", "author_association": "OWNER", "body": "I didn't bother with the tooltip, just the visible display if `?_facet_size=max`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 962391325, "label": "Show count of facet values if ?_facet_size=max"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1293#issuecomment-898961535", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1293", "id": 898961535, "node_id": "IC_kwDOBm6k_c41lRB_", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-14T21:37:24Z", "updated_at": "2021-08-14T21:37:24Z", "author_association": "OWNER", "body": "Did some more research into building SQLite custom versions via `pysqlite3` - here's what I figured out for macOS (which should hopefully work for Linux too): https://til.simonwillison.net/sqlite/build-specific-sqlite-pysqlite-macos", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 849978964, "label": "Show column metadata plus links for foreign keys on arbitrary query results"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1293#issuecomment-898936068", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1293", "id": 898936068, "node_id": "IC_kwDOBm6k_c41lK0E", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-14T17:44:54Z", "updated_at": "2021-08-14T17:44:54Z", "author_association": "OWNER", "body": "Another interesting query to consider: https://latest.datasette.io/fixtures?sql=explain+select+*+from++pragma_table_info%28+%27123_starts_with_digits%27%29\r\n\r\nThat one shows `VColumn` instead of `Column`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 849978964, "label": "Show column metadata plus links for foreign keys on arbitrary query results"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1293#issuecomment-898933865", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1293", "id": 898933865, "node_id": "IC_kwDOBm6k_c41lKRp", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-14T17:27:16Z", "updated_at": "2021-08-14T17:28:29Z", "author_association": "OWNER", "body": "Maybe I split this out into a separate Python library that gets tested against *every* SQLite release I can possibly try it against, and then bakes out the supported release versions into the library code itself?\r\n\r\nDatasette could depend on that library. The library could be released independently of Datasette any time a new SQLite version comes out.\r\n\r\nI could even run a separate git scraper repo that checks for new SQLite releases and submits PRs against the library when a new release comes out.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 849978964, "label": "Show column metadata plus links for foreign keys on arbitrary query results"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1293#issuecomment-898913629", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1293", "id": 898913629, "node_id": "IC_kwDOBm6k_c41lFVd", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-14T16:14:12Z", "updated_at": "2021-08-14T16:14:12Z", "author_association": "OWNER", "body": "I would feel a lot more comfortable about all of this if I had a robust mechanism for running the Datasette test suite against multiple versions of SQLite itself.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 849978964, "label": "Show column metadata plus links for foreign keys on arbitrary query results"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1293#issuecomment-898913554", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1293", "id": 898913554, "node_id": "IC_kwDOBm6k_c41lFUS", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-14T16:13:40Z", "updated_at": "2021-08-14T16:13:40Z", "author_association": "OWNER", "body": "I think I need to care about the following:\r\n\r\n- `ResultRow` and `Column` for the final result\r\n- `OpenRead` for opening tables\r\n- `OpenEphemeral` then `MakeRecord` and `IdxInsert` for writing records into ephemeral tables\r\n\r\n`Column` may reference either a table (from `OpenRead`) or an ephemeral table (from `OpenEphemeral`).\r\n\r\nThat *might* be enough.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 849978964, "label": "Show column metadata plus links for foreign keys on arbitrary query results"}, "performed_via_github_app": null}