{"html_url": "https://github.com/simonw/datasette/pull/1893#issuecomment-1316253186", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1893", "id": 1316253186, "node_id": "IC_kwDOBm6k_c5OdG4C", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-16T03:16:36Z", "updated_at": "2022-11-16T03:16:36Z", "author_association": "OWNER", "body": "Yeah I haven't written this down anywhere but Datasette definitely has an undocumented preference for lower-case SQL.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1450363982, "label": "Upgrade to CodeMirror 6, add SQL autocomplete"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1890#issuecomment-1316242752", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1890", "id": 1316242752, "node_id": "IC_kwDOBm6k_c5OdEVA", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-16T03:10:52Z", "updated_at": "2022-11-16T03:12:47Z", "author_association": "OWNER", "body": "https://bugs.webkit.org/show_bug.cgi?id=201768 - \" Datalist option's label not used\" - marked as RESOLVED FIXED on March 31st 2020.\r\n\r\nThe commit: https://trac.webkit.org/changeset/259330/webkit\r\n\r\nAnd here's the test mirrored on GitHub: https://cs.github.com/qtwebkit/webkit-mirror/blob/cc3fcd0b4bad1f7cf77c26e34aa01d16618d6d5e/LayoutTests/fast/forms/datalist/datalist-option-labels.html?q=datalist-option-labels.html", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1448143294, "label": "Autocomplete text entry for filter values that correspond to facets"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1890#issuecomment-1316240839", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1890", "id": 1316240839, "node_id": "IC_kwDOBm6k_c5OdD3H", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-16T03:09:11Z", "updated_at": "2022-11-16T03:09:11Z", "author_association": "OWNER", "body": "Here's a polyfill for ``: https://github.com/mfranzke/datalist-polyfill\r\n\r\nIt shouldn't be necessary now that Safari has shipped support (apparently added in https://developer.apple.com/documentation/safari-release-notes/safari-12_1-release-notes#3130314 Safari 12.1 in March 2019).\r\n\r\nBut it does look like Safari doesn't support differing `label` and `value` attributes, though documentation about this is hard to come by.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1448143294, "label": "Autocomplete text entry for filter values that correspond to facets"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1893#issuecomment-1316236448", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1893", "id": 1316236448, "node_id": "IC_kwDOBm6k_c5OdCyg", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-16T03:04:57Z", "updated_at": "2022-11-16T03:04:57Z", "author_association": "OWNER", "body": "If you rebase from `main` you should get the fix for that test failure.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1450363982, "label": "Upgrade to CodeMirror 6, add SQL autocomplete"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1890#issuecomment-1316233532", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1890", "id": 1316233532, "node_id": "IC_kwDOBm6k_c5OdCE8", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-16T03:00:58Z", "updated_at": "2022-11-16T03:00:58Z", "author_association": "OWNER", "body": "Oops, introduced a test failure: \r\n\r\n```\r\n def test_table_html_foreign_key_facets(app_client):\r\n response = app_client.get(\r\n \"/fixtures/foreign_key_references?_facet=foreign_key_with_blank_label\"\r\n )\r\n assert response.status == 200\r\n> assert (\r\n '
  • '\r\n \"- 1
    • \"\r\n ) in response.text\r\nE assert '
  • - 1
    • ' in '\\n\\n\\n fixtures: foreign_key_references: 2 rows\\n \\n\\n\\n\\n\\n'\r\nE + where '\\n\\n\\n fixtures: foreign_key_references: 2 rows\\n \\n\\n\\n\\n\\n' = .text\r\n```\r\nNeed to fix this test:\r\n\r\nhttps://github.com/simonw/datasette/blob/eac028d3f77aa5473a5fcf59240635a1bca80f7d/tests/test_table_html.py#L616-L624", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1448143294, "label": "Autocomplete text entry for filter values that correspond to facets"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1893#issuecomment-1316232588", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1893", "id": 1316232588, "node_id": "IC_kwDOBm6k_c5OdB2M", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-16T03:00:04Z", "updated_at": "2022-11-16T03:00:04Z", "author_association": "OWNER", "body": "Oops, the tests are failing because of a test failure I introduced here:\r\n- #1890", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1450363982, "label": "Upgrade to CodeMirror 6, add SQL autocomplete"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1893#issuecomment-1316231560", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1893", "id": 1316231560, "node_id": "IC_kwDOBm6k_c5OdBmI", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-16T02:59:00Z", "updated_at": "2022-11-16T02:59:00Z", "author_association": "OWNER", "body": "The resize handle doesn't appear on Mobile Safari on iPhone - I don't think that particularly matters though.\r\n\r\nThe textarea does get a weird border around it when focused on iPhone though.\r\n\r\nFocused:\r\n\r\n![BF34E8FB-E35C-4CAB-9BFB-8EEF7E29B16C_1_201_a](https://user-images.githubusercontent.com/9599/202072748-c85bab94-a039-4ed6-8185-3cac25c78ed3.jpeg)\r\n\r\nNot focused:\r\n\r\n![31A5CF38-D540-4A1A-8A7D-E29453D150F4_1_201_a](https://user-images.githubusercontent.com/9599/202072744-d9f0ea62-13b7-46ff-afe1-6d88d7fb8b53.jpeg)\r\n\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1450363982, "label": "Upgrade to CodeMirror 6, add SQL autocomplete"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1893#issuecomment-1316227073", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1893", "id": 1316227073, "node_id": "IC_kwDOBm6k_c5OdAgB", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-16T02:54:22Z", "updated_at": "2022-11-16T02:54:32Z", "author_association": "OWNER", "body": "If you can get a version of this working with table and column autocompletion just using a static JavaScript object in the source code with the right tables and columns, I'm happy to take on the work of turning that static object into something that Datasette includes in the page itself with all of the correct values.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1450363982, "label": "Upgrade to CodeMirror 6, add SQL autocomplete"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1893#issuecomment-1316141764", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1893", "id": 1316141764, "node_id": "IC_kwDOBm6k_c5OcrrE", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-16T01:26:59Z", "updated_at": "2022-11-16T01:26:59Z", "author_association": "OWNER", "body": "Resizing works great for me - and the page automatically sizes the editor to fit an existing query, e.g. on https://datasette-pr-1893.vercel.app/fixtures?sql=select+id%2C+content%2C+content2%0D%0A++from+primary_key_multiple_columns_explicit_label%0D%0A++order+by+id%0D%0A++limit+101", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1450363982, "label": "Upgrade to CodeMirror 6, add SQL autocomplete"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1893#issuecomment-1316137982", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1893", "id": 1316137982, "node_id": "IC_kwDOBm6k_c5Ocqv-", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-16T01:23:47Z", "updated_at": "2022-11-16T01:23:47Z", "author_association": "OWNER", "body": "Autocomplete here looks promising (I've wanted that to work for years!), but it does currently show a whole bunch of suggestions which aren't part of the SQLite SQL dialect:\r\n\r\n![autocomplete](https://user-images.githubusercontent.com/9599/202060211-51ec9f45-bc52-459a-a729-27fc2faadff9.gif)\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1450363982, "label": "Upgrade to CodeMirror 6, add SQL autocomplete"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1893#issuecomment-1316135244", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1893", "id": 1316135244, "node_id": "IC_kwDOBm6k_c5OcqFM", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-16T01:21:41Z", "updated_at": "2022-11-16T01:21:41Z", "author_association": "OWNER", "body": "I just deployed a demo instance like this (using the commit hash from this PR):\r\n\r\n```bash\r\ndatasette publish vercel fixtures.db \\\r\n --branch 544f7025900b78f63c34b9985522271ba5fd9c0f \\\r\n --project datasette-pr-1893 \\\r\n --scope datasette \\\r\n --about 'PR 1893' \\\r\n --about_url https://github.com/simonw/datasette/pull/1893\r\n```\r\nHere's the result: https://datasette-pr-1893.vercel.app/fixtures", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1450363982, "label": "Upgrade to CodeMirror 6, add SQL autocomplete"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1863#issuecomment-1315812212", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1863", "id": 1315812212, "node_id": "IC_kwDOBm6k_c5ObbN0", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-15T20:12:02Z", "updated_at": "2022-11-15T20:12:02Z", "author_association": "OWNER", "body": "If the update succeeds it will return `{\"ok\": true}`.\r\n\r\nFor consistency with `/db/table/-/insert` you can pass `\"return\": true` and it will return a `\"row\"` key with the now-updated full row.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1425029242, "label": "Update a single record in an existing table"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1863#issuecomment-1315809867", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1863", "id": 1315809867, "node_id": "IC_kwDOBm6k_c5ObapL", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-15T20:09:44Z", "updated_at": "2022-11-15T20:09:44Z", "author_association": "OWNER", "body": "I'm also not going to implement `\"alter\": true` yet (which would add any missing columns based on the update) - I'll hold that off for a later feature.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1425029242, "label": "Update a single record in an existing table"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1863#issuecomment-1315809260", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1863", "id": 1315809260, "node_id": "IC_kwDOBm6k_c5Obafs", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-15T20:09:11Z", "updated_at": "2022-11-15T20:09:11Z", "author_association": "OWNER", "body": "I'm going to use the error format I've been experimenting with here:\r\n- #1875\r\n\r\n```json\r\n{\r\n \"type\": \"https://example.net/validation-error\",\r\n \"title\": \"Your request is not valid.\",\r\n \"errors\": [\r\n {\r\n \"detail\": \"must be a positive integer\",\r\n \"pointer\": \"#/age\"\r\n },\r\n {\r\n \"detail\": \"must be 'green', 'red' or 'blue'\",\r\n \"pointer\": \"#/profile/color\"\r\n }\r\n ]\r\n}\r\n```\r\nI'm not quite ready to commit to a `type` URL though, so I'll leave that to be solved later should I fully embrace that RFC.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1425029242, "label": "Update a single record in an existing table"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1863#issuecomment-1315808062", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1863", "id": 1315808062, "node_id": "IC_kwDOBm6k_c5ObaM-", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-15T20:08:04Z", "updated_at": "2022-11-15T20:08:04Z", "author_association": "OWNER", "body": "The initial design I'm going to implement will look like this:\r\n\r\n```\r\nPOST /db/table/1/-/update\r\nAuthorization: Bearer xxx\r\nContent-Type: application/json\r\n```\r\n```json\r\n{\r\n \"update\": {\r\n \"name\": \"New name\"\r\n }\r\n}\r\n```\r\nAny fields that are not yet columns will return an error.\r\n\r\nShould it enforce types, in as much as an integer column should have a JSON integer passed to it, or should it allow strings containing valid integers?\r\n\r\nI'm going to allow strings, mainly as a workaround for the fact that JavaScript integers have a maximum size.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1425029242, "label": "Update a single record in an existing table"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1892#issuecomment-1315805498", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1892", "id": 1315805498, "node_id": "IC_kwDOBm6k_c5ObZk6", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-15T20:05:30Z", "updated_at": "2022-11-15T20:05:30Z", "author_association": "OWNER", "body": "One slight concern: https://latest.datasette.io/ will increasingly reflect a version that isn't the most recent production release.\r\n\r\nI might setup https://stable.datasette.io/ as a demo instance of the most recent non-alpha release to compensate for that.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1450312343, "label": "Merge 1.0-dev branch back to main"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1892#issuecomment-1315804535", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1892", "id": 1315804535, "node_id": "IC_kwDOBm6k_c5ObZV3", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-15T20:04:38Z", "updated_at": "2022-11-15T20:04:38Z", "author_association": "OWNER", "body": "I'll do this after the 1.0a0 release:\r\n- #1708", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1450312343, "label": "Merge 1.0-dev branch back to main"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1708#issuecomment-1095675839", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1708", "id": 1095675839, "node_id": "IC_kwDOBm6k_c5BTq-_", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-04-11T23:06:30Z", "updated_at": "2022-11-15T19:57:53Z", "author_association": "OWNER", "body": "# Datasette 1.0 alpha 1\r\n\r\nThis alpha release is a preview of Datasette 1.0.\r\n\r\nDatasette 1.0 marks a significant milestone in the project: it is the point from which various aspects of Datasette can be considered \"stable\", in that code developed against them should expect not to be broken by future releases in the 1.x series.\r\n\r\nThis will hold true until the next major version release, Datasette 2.0 - which we hope to hold off releasing for as long as possible.\r\n\r\nThe following Datasette components should be be considered stable after 1.0:\r\n\r\n- The plugin API. Plugins developed against 1.0 should continue to work unmodified throughout the 1.x series.\r\n- The JSON API. Code written that interacts with Datasette's default JSON web API should continue to work.\r\n- The template context. If you build custom templates against Datasette your custom pages should continue to work.\r\n\r\nNote that none of these components will cease to introduce new features. New plugin hooks, new JSON APIs and new template context variables can be introduced without breaking existing code.\r\n\r\nSince this alpha release previews features that will be frozen for 1.0, please test this thoroughly against your existing Datasette projects.\r\n\r\nYou can install the alpha using:\r\n\r\n pip install datasette==1.0a0\r\n\r\n## JSON API changes\r\n\r\nThe most significant changes introduced in this new alpha concern Datasette's JSON API.\r\n\r\nThe default JSON returned by the `/database/table.json` endpoint has changed. It now returns an object with two keys: `rows` - which contains a list of objects representing the rows in the table or query, and `more` containing a `boolean` that shows if there are more rows or if this object contains them all.\r\n\r\n```json\r\n{\r\n \"rows\": [{\r\n \"id\": 1,\r\n \"name\": \"Name 1\"\r\n }, {\r\n \"id\": 2,\r\n \"name\": \"Name 2\"\r\n }],\r\n \"more\": false\r\n}\r\n```\r\n[ Initially I thought about going with `next_url`, which would be `null` if you have reached the last page of records. Maybe that would be better? But since `next_url` cannot be provided on query pages, should this be part of the default format at all? ]\r\n\r\n## Use ?_extra= to retrieve extra fields\r\n\r\nThe default format can be expanded using one or more `?_extra=` parameters. This takes names of extra keys you would like to include. These can be comma-separated or `?_extra=` can be applied multiple times.\r\n\r\nFor example:\r\n\r\n /database/table.json?_extra=total\r\n\r\nThis adds a `\"total\": 124` field to the returned JSON.\r\n\r\n[ Question: if you do `?_facet=foo` then do you still need to do `?_extra=facets` - I think not? ]", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1200649124, "label": "Datasette 1.0 alpha upcoming release notes"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1890#issuecomment-1314891228", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1890", "id": 1314891228, "node_id": "IC_kwDOBm6k_c5OX6Xc", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-15T07:23:01Z", "updated_at": "2022-11-15T07:23:01Z", "author_association": "OWNER", "body": "Annoying: Mobile Safari doesn't seem to support separate labels and values. I should probably disable this feature on that browser, at least for foreign key facets (for the moment).", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1448143294, "label": "Autocomplete text entry for filter values that correspond to facets"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1890#issuecomment-1314856513", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1890", "id": 1314856513, "node_id": "IC_kwDOBm6k_c5OXx5B", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-15T06:56:29Z", "updated_at": "2022-11-15T06:56:29Z", "author_association": "OWNER", "body": "Looks like I can fix that like so:\r\n```html\r\n\r\n \r\n \r\n \r\n \r\n\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1448143294, "label": "Autocomplete text entry for filter values that correspond to facets"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1890#issuecomment-1314850524", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1890", "id": 1314850524, "node_id": "IC_kwDOBm6k_c5OXwbc", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-15T06:48:37Z", "updated_at": "2022-11-15T06:48:37Z", "author_association": "OWNER", "body": "Spotted a bug with this on https://latest.datasette.io/fixtures/facetable?_facet=_city_id - the `_city_id` column is a foreign key, so you need to type `1` or `2` - but the autocomplete list shows the full text names for the cities.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1448143294, "label": "Autocomplete text entry for filter values that correspond to facets"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1890#issuecomment-1314849867", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1890", "id": 1314849867, "node_id": "IC_kwDOBm6k_c5OXwRL", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-15T06:47:51Z", "updated_at": "2022-11-15T06:47:51Z", "author_association": "OWNER", "body": "Demo now live here: https://congress-legislators.datasettes.com/legislators/legislator_terms?_facet=party - select `party` and start typing.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1448143294, "label": "Autocomplete text entry for filter values that correspond to facets"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1890#issuecomment-1314848432", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1890", "id": 1314848432, "node_id": "IC_kwDOBm6k_c5OXv6w", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-15T06:46:08Z", "updated_at": "2022-11-15T06:46:08Z", "author_association": "OWNER", "body": "Wrote a TIL about ``: https://til.simonwillison.net/html/datalist", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1448143294, "label": "Autocomplete text entry for filter values that correspond to facets"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1862#issuecomment-1314845667", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1862", "id": 1314845667, "node_id": "IC_kwDOBm6k_c5OXvPj", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-15T06:42:09Z", "updated_at": "2022-11-15T06:42:32Z", "author_association": "OWNER", "body": "I implemented this as part of `/db/-/create`.\r\n\r\nhttps://docs.datasette.io/en/1.0-dev/json_api.html#creating-a-table-from-example-data", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1425011030, "label": "Create a new table from one or more records, `sqlite-utils` style"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1890#issuecomment-1314835740", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1890", "id": 1314835740, "node_id": "IC_kwDOBm6k_c5OXs0c", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-15T06:30:26Z", "updated_at": "2022-11-15T06:30:26Z", "author_association": "OWNER", "body": "That prototype actually works really well! I'm going to add that to `table.js`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1448143294, "label": "Autocomplete text entry for filter values that correspond to facets"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1890#issuecomment-1314833881", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1890", "id": 1314833881, "node_id": "IC_kwDOBm6k_c5OXsXZ", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-15T06:27:21Z", "updated_at": "2022-11-15T06:27:21Z", "author_association": "OWNER", "body": "Here's a prototype:\r\n```javascript\r\nfunction createDataLists() {\r\n var facetResults = document.querySelectorAll(\".facet-results [data-column]\");\r\n Array.from(facetResults).forEach(function (facetResult) {\r\n // Use link text from all links in the facet result\r\n var linkTexts = Array.from(\r\n facetResult.querySelectorAll(\"li:not(.facet-truncated) a\")\r\n ).map(function (link) {\r\n return link.textContent;\r\n });\r\n // Create a datalist element\r\n var datalist = document.createElement(\"datalist\");\r\n datalist.id = \"datalist-\" + facetResult.dataset.column;\r\n // Create an option element for each link text\r\n linkTexts.forEach(function (linkText) {\r\n var option = document.createElement(\"option\");\r\n option.value = linkText;\r\n datalist.appendChild(option);\r\n });\r\n // Add the datalist to the facet result\r\n facetResult.appendChild(datalist);\r\n });\r\n}\r\ncreateDataLists();\r\n\r\n// When any select with name=_filter_column changes, update the datalist\r\ndocument.body.addEventListener(\"change\", function (event) {\r\n if (event.target.name === \"_filter_column\") {\r\n event.target\r\n .closest(\".filter-row\")\r\n .querySelector(\".filter-value\")\r\n .setAttribute(\"list\", \"datalist-\" + event.target.value);\r\n }\r\n});\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1448143294, "label": "Autocomplete text entry for filter values that correspond to facets"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1890#issuecomment-1314829751", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1890", "id": 1314829751, "node_id": "IC_kwDOBm6k_c5OXrW3", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-15T06:20:50Z", "updated_at": "2022-11-15T06:20:50Z", "author_association": "OWNER", "body": "This finds the right links on the page:\r\n\r\n document.querySelectorAll('.facet-results [data-column] li:not(.facet-truncated) a')", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1448143294, "label": "Autocomplete text entry for filter values that correspond to facets"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1890#issuecomment-1314825019", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1890", "id": 1314825019, "node_id": "IC_kwDOBm6k_c5OXqM7", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-15T06:13:36Z", "updated_at": "2022-11-15T06:13:36Z", "author_association": "OWNER", "body": "This could start out as a purely JavaScript enhancement for pages that already figured out the available values through faceting, like you suggested.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1448143294, "label": "Autocomplete text entry for filter values that correspond to facets"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1890#issuecomment-1314823752", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1890", "id": 1314823752, "node_id": "IC_kwDOBm6k_c5OXp5I", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-15T06:11:49Z", "updated_at": "2022-11-15T06:11:49Z", "author_association": "OWNER", "body": "I tried this out on https://congress-legislators.datasettes.com/legislators/legislator_terms for the `party` column - here's the demo:\r\n\r\n![datalist](https://user-images.githubusercontent.com/9599/201839812-db887ce0-c4b9-432c-8620-5ac73f222a63.gif)\r\n\r\nI made this work by dropping the following HTML into the page in the browser DevTools:\r\n```html\r\n\r\n\r\n```\r\nAnd then adding `list=\"party\"` to the input element in the filter form.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1448143294, "label": "Autocomplete text entry for filter values that correspond to facets"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1890#issuecomment-1314821337", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1890", "id": 1314821337, "node_id": "IC_kwDOBm6k_c5OXpTZ", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-15T06:08:19Z", "updated_at": "2022-11-15T06:08:19Z", "author_association": "OWNER", "body": "Oh interesting... this doesn't even need to be attached to the visible faceting feature, necessarily: Datasette could try to detect when a column has a limited number of options (which the faceting code handles already) and could turn those into an auto-complete interface.\r\n\r\nThere's actually a native HTML element for this these days: the `` https://developer.mozilla.org/en-US/docs/Web/HTML/Element/datalist", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1448143294, "label": "Autocomplete text entry for filter values that correspond to facets"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1882#issuecomment-1314813205", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1882", "id": 1314813205, "node_id": "IC_kwDOBm6k_c5OXnUV", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-15T06:00:41Z", "updated_at": "2022-11-15T06:00:41Z", "author_association": "OWNER", "body": "Documentation:\r\n\r\n- https://docs.datasette.io/en/1.0-dev/json_api.html#creating-a-table\r\n- https://docs.datasette.io/en/1.0-dev/json_api.html#creating-a-table-from-example-data\r\n\r\nWrote a TIL about how I wrote some of those tests with Copilot: https://til.simonwillison.net/gpt3/writing-test-with-copilot", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1435294468, "label": "`/db/-/create` API for creating tables"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1875#issuecomment-1314620086", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1875", "id": 1314620086, "node_id": "IC_kwDOBm6k_c5OW4K2", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-15T01:09:56Z", "updated_at": "2022-11-15T01:09:56Z", "author_association": "OWNER", "body": "Rough initial prototype:\r\n```diff\r\ndiff --git a/datasette/views/table.py b/datasette/views/table.py\r\nindex 8b987221..518ac578 100644\r\n--- a/datasette/views/table.py\r\n+++ b/datasette/views/table.py\r\n@@ -1103,19 +1103,30 @@ class TableInsertView(BaseView):\r\n except json.JSONDecodeError as e:\r\n return _errors([\"Invalid JSON: {}\".format(e)])\r\n if not isinstance(data, dict):\r\n- return _errors([\"JSON must be a dictionary\"])\r\n+ return _errors([{\"detail\": \"JSON must be a dictionary\", \"pointer\": \"#/\"}])\r\n keys = data.keys()\r\n \r\n # keys must contain \"row\" or \"rows\"\r\n if \"row\" not in keys and \"rows\" not in keys:\r\n return _errors(['JSON must have one or other of \"row\" or \"rows\"'])\r\n rows = []\r\n+ was_single_row = False\r\n if \"row\" in keys:\r\n if \"rows\" in keys:\r\n- return _errors(['Cannot use \"row\" and \"rows\" at the same time'])\r\n+ return _errors(\r\n+ [\r\n+ {\r\n+ \"detail\": 'Cannot use \"row\" and \"rows\" at the same time',\r\n+ \"pointer\": \"#/row\",\r\n+ }\r\n+ ]\r\n+ )\r\n+ was_single_row = True\r\n row = data[\"row\"]\r\n if not isinstance(row, dict):\r\n- return _errors(['\"row\" must be a dictionary'])\r\n+ return _errors(\r\n+ [{\"detail\": '\"row\" must be a dictionary', \"pointer\": \"#/row\"}]\r\n+ )\r\n rows = [row]\r\n data[\"return\"] = True\r\n else:\r\n@@ -1152,9 +1163,12 @@ class TableInsertView(BaseView):\r\n invalid_columns = set(row.keys()) - columns\r\n if invalid_columns:\r\n errors.append(\r\n- \"Row {} has invalid columns: {}\".format(\r\n- i, \", \".join(sorted(invalid_columns))\r\n- )\r\n+ {\r\n+ \"detail\": \"Invalid columns: {}\".format(\r\n+ \", \".join(sorted(invalid_columns))\r\n+ ),\r\n+ \"pointer\": \"#/blah/\",\r\n+ }\r\n )\r\n if errors:\r\n return _errors(errors)\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1430797211, "label": "Figure out design for JSON errors (consider RFC 7807)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1875#issuecomment-1314615592", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1875", "id": 1314615592, "node_id": "IC_kwDOBm6k_c5OW3Eo", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-15T01:04:28Z", "updated_at": "2022-11-15T01:04:28Z", "author_association": "OWNER", "body": "Worth noting this bit in RFC 7807:\r\n\r\n> The fictional problem type here defines the \"errors\" extension, an\r\n> array that describes the details of each validation error. Each\r\n> member is an object containing \"detail\" to describe the issue, and\r\n> \"pointer\" to locate the problem within the request's content using a\r\n> JSON Pointer [JSON-POINTER].\r\n\r\nSo the list of `\"errors\"` with JSON Pointer isn't technically part of the spec, it's an imaginary extension.\r\n\r\nIt fits what I need to do though, so I'm inclined to stick with it anyway.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1430797211, "label": "Figure out design for JSON errors (consider RFC 7807)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1875#issuecomment-1314545407", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1875", "id": 1314545407, "node_id": "IC_kwDOBm6k_c5OWl7_", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-14T23:30:34Z", "updated_at": "2022-11-14T23:30:34Z", "author_association": "OWNER", "body": "TIL: https://til.simonwillison.net/json/json-pointer", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1430797211, "label": "Figure out design for JSON errors (consider RFC 7807)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1875#issuecomment-1314491884", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1875", "id": 1314491884, "node_id": "IC_kwDOBm6k_c5OWY3s", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-14T22:26:11Z", "updated_at": "2022-11-14T22:26:54Z", "author_association": "OWNER", "body": "Spec looks pretty simple:\r\n\r\n> A JSON Pointer is a Unicode string (see [RFC4627], Section 3)\r\n> containing a sequence of zero or more reference tokens, each prefixed\r\n> by a `/` (%x2F) character.\r\n> \r\n> Because the characters `~` (%x7E) and `/` (%x2F) have special\r\n> meanings in JSON Pointer, `~` needs to be encoded as `~0` and `/`\r\n> needs to be encoded as `~1` when these characters appear in a\r\n> reference token.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1430797211, "label": "Figure out design for JSON errors (consider RFC 7807)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1875#issuecomment-1314491150", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1875", "id": 1314491150, "node_id": "IC_kwDOBm6k_c5OWYsO", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-14T22:25:20Z", "updated_at": "2022-11-14T22:25:20Z", "author_association": "OWNER", "body": "That's using JSON Pointer: https://www.rfc-editor.org/rfc/rfc6901\r\n\r\nThere's a Python library for that here https://github.com/stefankoegl/python-json-pointer/blob/master/jsonpointer.py - which looks simple and clean and well maintained and documented, but it only handles the \"what is at this pointer within this JSON object\" case - I need to generate the correct JSON pointer to explain where my error is.\r\n\r\nSo I think I'll end up hand-rolling this.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1430797211, "label": "Figure out design for JSON errors (consider RFC 7807)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1875#issuecomment-1314488010", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1875", "id": 1314488010, "node_id": "IC_kwDOBm6k_c5OWX7K", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-14T22:21:43Z", "updated_at": "2022-11-14T22:21:43Z", "author_association": "OWNER", "body": "Here's the most relevant example from the RFC spec:\r\n```\r\n POST /details HTTP/1.1\r\n Host: account.example.com\r\n Accept: application/json\r\n```\r\n```json\r\n {\r\n \"age\": 42.3,\r\n \"profile\": {\r\n \"color\": \"yellow\"\r\n }\r\n }\r\n```\r\n```\r\n HTTP/1.1 400 Bad Request\r\n Content-Type: application/problem+json\r\n Content-Language: en\r\n```\r\n```json\r\n{\r\n \"type\": \"https://example.net/validation-error\",\r\n \"title\": \"Your request is not valid.\",\r\n \"errors\": [\r\n {\r\n \"detail\": \"must be a positive integer\",\r\n \"pointer\": \"#/age\"\r\n },\r\n {\r\n \"detail\": \"must be 'green', 'red' or 'blue'\",\r\n \"pointer\": \"#/profile/color\"\r\n }\r\n ]\r\n}\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1430797211, "label": "Figure out design for JSON errors (consider RFC 7807)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1884#issuecomment-1314054300", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1884", "id": 1314054300, "node_id": "IC_kwDOBm6k_c5OUuCc", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-14T16:40:06Z", "updated_at": "2022-11-14T16:40:06Z", "author_association": "OWNER", "body": "I wonder if there are any reasons that inspect SHOULD try to count virtual tables? Like are there any likely uses for a cirial table where the count is both interesting and likely to be accessed often enough that it's worth caching?\r\n\r\nI have an issue open to add a setting to disable table counts entirely:\r\n\r\n- #1818 \r\n\r\nMaybe that should be expanded to automatically disable row counts for virtual tables entirely? Which would mean no count would be shown for them in the UI.\r\n\r\nIf you desperately wanted a count you would then have to run a count(*) query against them explicitly.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1439009231, "label": "Exclude virtual tables from datasette inspect"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1850#issuecomment-1313156167", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1850", "id": 1313156167, "node_id": "IC_kwDOBm6k_c5ORSxH", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-14T06:23:39Z", "updated_at": "2022-11-14T06:23:39Z", "author_association": "OWNER", "body": "The API explorer is now live here: https://latest-1-0-dev.datasette.io/-/api", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1421529723, "label": "Write API in Datasette core"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1850#issuecomment-1313155712", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1850", "id": 1313155712, "node_id": "IC_kwDOBm6k_c5ORSqA", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-14T06:22:57Z", "updated_at": "2022-11-14T06:22:57Z", "author_association": "OWNER", "body": "I think the ability to create tokens should be protected by a `create-tokens` permission, not just a global setting.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1421529723, "label": "Write API in Datasette core"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1888#issuecomment-1313139657", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1888", "id": 1313139657, "node_id": "IC_kwDOBm6k_c5OROvJ", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-14T06:04:48Z", "updated_at": "2022-11-14T06:04:48Z", "author_association": "OWNER", "body": "Demo: https://latest-1-0-dev.datasette.io/-/api", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1447439985, "label": "API explorer should take immutability into account"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1866#issuecomment-1313128913", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1866", "id": 1313128913, "node_id": "IC_kwDOBm6k_c5ORMHR", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-14T05:48:22Z", "updated_at": "2022-11-14T05:48:22Z", "author_association": "OWNER", "body": "I changed my mind about the `\"return_rows\": true` option - I'm going to rename it to `\"return\": true`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1426001541, "label": "API for bulk inserting records into a table"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1874#issuecomment-1313127054", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1874", "id": 1313127054, "node_id": "IC_kwDOBm6k_c5ORLqO", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-14T05:45:00Z", "updated_at": "2022-11-14T05:45:00Z", "author_association": "OWNER", "body": "Demo: https://latest-1-0-dev.datasette.io/-/api#path=%2Ffixtures%2Ffacetable%2F-%2Fdrop&json=&method=POST\r\n\r\n\"image\"\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1429030341, "label": "API to drop a table"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1871#issuecomment-1313125870", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1871", "id": 1313125870, "node_id": "IC_kwDOBm6k_c5ORLXu", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-14T05:42:50Z", "updated_at": "2022-11-14T05:42:50Z", "author_association": "OWNER", "body": "Demo: https://latest-1-0-dev.datasette.io/-/api#path=%2Ffixtures%2Ffacetable%2F-%2Fdrop&json=%7B%22confirm%22%3A+true%7D&method=POST", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1427293909, "label": "API explorer tool"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1874#issuecomment-1313125123", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1874", "id": 1313125123, "node_id": "IC_kwDOBm6k_c5ORLMD", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-14T05:41:20Z", "updated_at": "2022-11-14T05:42:23Z", "author_association": "OWNER", "body": "I also changed the confirmation JSON returned by this endpoint to add the `database` and `table` like so:\r\n```json\r\n{\r\n \"ok\": true,\r\n \"database\": \"data\",\r\n \"table\": \"docs\",\r\n \"row_count\": 1,\r\n \"message\": \"Pass \\\"confirm\\\": true to confirm\"\r\n}\r\n```\r\nUpdated docs: https://docs.datasette.io/en/1.0-dev/json_api.html#dropping-tables", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1429030341, "label": "API to drop a table"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1874#issuecomment-1313119558", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1874", "id": 1313119558, "node_id": "IC_kwDOBm6k_c5ORJ1G", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-14T05:30:27Z", "updated_at": "2022-11-14T05:30:27Z", "author_association": "OWNER", "body": "Found a bug: you get a 500 error if you try this against an immutable database.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1429030341, "label": "API to drop a table"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1850#issuecomment-1313115059", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1850", "id": 1313115059, "node_id": "IC_kwDOBm6k_c5ORIuz", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-14T05:21:30Z", "updated_at": "2022-11-14T05:21:30Z", "author_association": "OWNER", "body": "New documentation for these features currently lives here: https://docs.datasette.io/en/1.0-dev/json_api.html#the-json-write-api", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1421529723, "label": "Write API in Datasette core"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1875#issuecomment-1313114283", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1875", "id": 1313114283, "node_id": "IC_kwDOBm6k_c5ORIir", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-14T05:20:00Z", "updated_at": "2022-11-14T05:20:00Z", "author_association": "OWNER", "body": "I started a conversation about JSON error standards on Mastodon here: https://fedi.simonwillison.net/web/@simon/109338725610487457\r\n\r\nQuite a few people pointed to this RFC independently.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1430797211, "label": "Figure out design for JSON errors (consider RFC 7807)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1887#issuecomment-1313113642", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1887", "id": 1313113642, "node_id": "IC_kwDOBm6k_c5ORIYq", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-14T05:18:51Z", "updated_at": "2022-11-14T05:18:51Z", "author_association": "OWNER", "body": "Updated docs: https://docs.datasette.io/en/1.0-dev/json_api.html#dropping-tables", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1447388809, "label": "Add a confirm step to the drop table API"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1887#issuecomment-1313097713", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1887", "id": 1313097713, "node_id": "IC_kwDOBm6k_c5OREfx", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-14T05:00:54Z", "updated_at": "2022-11-14T05:00:54Z", "author_association": "OWNER", "body": "I'm going to add a `\"confirm\": true` option to the API. Without that, it returns a note about how many rows will be deleted.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1447388809, "label": "Add a confirm step to the drop table API"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1871#issuecomment-1313097057", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1871", "id": 1313097057, "node_id": "IC_kwDOBm6k_c5OREVh", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-14T04:59:28Z", "updated_at": "2022-11-14T04:59:28Z", "author_association": "OWNER", "body": "In playing with the API explorer just now I realized it's way too easy to accidentally drop a table using it.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1427293909, "label": "API explorer tool"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1871#issuecomment-1313072900", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1871", "id": 1313072900, "node_id": "IC_kwDOBm6k_c5OQ-cE", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-14T04:15:50Z", "updated_at": "2022-11-14T04:15:50Z", "author_association": "OWNER", "body": "For the example links - I'm going to have these at the bottom of the page so you don't have to scroll past them.\r\n\r\nIdeally these would take the user's permissions into account. This could make the page expensive to load, but I'm going to risk it for the moment.\r\n\r\nSomething like this then:\r\n\r\n> - data\r\n> - /data/-/create - create table\r\n> - /data/table1/-/insert - insert into table1\r\n> - /data/table1/-/drop - drop table1\r\n\r\nI won't bother with per-row demo links (for update and delete) because there could be thousands of them for each table.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1427293909, "label": "API explorer tool"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1871#issuecomment-1313062699", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1871", "id": 1313062699, "node_id": "IC_kwDOBm6k_c5OQ78r", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-14T04:03:29Z", "updated_at": "2022-11-14T04:12:41Z", "author_association": "OWNER", "body": "Two things left before I close this issue:\r\n\r\n- [x] I want to preserve the state of the forms in the URL - probably after a `#`\r\n- [ ] Instead of hard-coding the current examples, I want to provide a list of links which populate the forms", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1427293909, "label": "API explorer tool"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1886#issuecomment-1313052863", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1886", "id": 1313052863, "node_id": "IC_kwDOBm6k_c5OQ5i_", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-14T03:40:50Z", "updated_at": "2022-11-14T03:40:50Z", "author_association": "OWNER", "body": "Tim Sherratt on Twitter: https://twitter.com/wragge/status/1591930345469153282\r\n\r\n> Where do I start? The [#GLAMWorkbench](https://twitter.com/hashtag/GLAMWorkbench?src=hashtag_click) now includes a number of examples where GLAM data is harvested, processed, and then made available for exploration via Datasette.\r\n>\r\n> https://glam-workbench.net/\r\n>\r\n> For example the GLAM Name Index Search brings together 10+ million entries from 240 indexes and provides an aggregated search using the Datasette search-all plugin:\r\n>\r\n> https://glam-workbench.net/name-search/\r\n>\r\n> Most recently I converted PDFs of the Tasmanian Postal Directories to a big Datasette instance: https://updates.timsherratt.org/2022/09/15/from-pdfs-to.html the process is documented and reusable.", "reactions": "{\"total_count\": 1, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 1, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1447050738, "label": "Call for birthday presents: if you're using Datasette, let us know how you're using it here"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1871#issuecomment-1312822353", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1871", "id": 1312822353, "node_id": "IC_kwDOBm6k_c5OQBRR", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-13T21:07:40Z", "updated_at": "2022-11-13T21:07:40Z", "author_association": "OWNER", "body": "I'm going to need extra code to toggle POST closed when GET opens and vice-versa.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1427293909, "label": "API explorer tool"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1871#issuecomment-1312821031", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1871", "id": 1312821031, "node_id": "IC_kwDOBm6k_c5OQA8n", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-13T21:02:06Z", "updated_at": "2022-11-13T21:03:11Z", "author_association": "OWNER", "body": "Actually no, I'm going to add a class of `details-menu` to the other details elements that SHOULD be closed. That way custom templates using `
    ` won't close in a surprising way.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1427293909, "label": "API explorer tool"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1871#issuecomment-1312816451", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1871", "id": 1312816451, "node_id": "IC_kwDOBm6k_c5OP_1D", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-13T20:39:26Z", "updated_at": "2022-11-13T20:39:34Z", "author_association": "OWNER", "body": "I'm going to add a special `no-auto-close` class to these and teach that code not to close them.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1427293909, "label": "API explorer tool"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1871#issuecomment-1312816292", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1871", "id": 1312816292, "node_id": "IC_kwDOBm6k_c5OP_yk", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-13T20:38:42Z", "updated_at": "2022-11-13T20:38:42Z", "author_association": "OWNER", "body": "The current API explorer uses details/summary elements for the GET and POST dialogs.\r\n\r\nI only want one of these to be open at a time, to reflect that you can make either a GET or a POST.\r\n\r\nI just noticed that clicking anywhere else on the page closes both elements, which isn't what I want to happen. Turns out that's because of this code I added as part of Datasette's menu implementation!\r\n\r\nhttps://github.com/simonw/datasette/blob/9f54f00a50a4d950cfd69a0ff3526ae82c858826/datasette/templates/_close_open_menus.html#L2-L15", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1427293909, "label": "API explorer tool"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1882#issuecomment-1312582512", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1882", "id": 1312582512, "node_id": "IC_kwDOBm6k_c5OPGtw", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-12T22:11:18Z", "updated_at": "2022-11-12T22:11:18Z", "author_association": "OWNER", "body": "I like this:\r\n\r\n\"image\"\r\n\r\n```json\r\n{\r\n \"ok\": true,\r\n \"database\": \"data\",\r\n \"table\": \"agai2n\",\r\n \"table_url\": \"http://127.0.0.1:8001/data/agai2n\",\r\n \"schema\": \"CREATE TABLE [agai2n] (\\n [hello] INTEGER\\n)\"\r\n}\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1435294468, "label": "`/db/-/create` API for creating tables"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1882#issuecomment-1312581121", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1882", "id": 1312581121, "node_id": "IC_kwDOBm6k_c5OPGYB", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-12T22:01:32Z", "updated_at": "2022-11-12T22:01:32Z", "author_association": "OWNER", "body": "I'm going to change it to `table` in the output AND the input.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1435294468, "label": "`/db/-/create` API for creating tables"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1882#issuecomment-1312581008", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1882", "id": 1312581008, "node_id": "IC_kwDOBm6k_c5OPGWQ", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-12T22:00:52Z", "updated_at": "2022-11-12T22:00:52Z", "author_association": "OWNER", "body": "Tried out my prototype in the API explorer:\r\n\r\n\"image\"\r\n\r\nThe `\"name\"` on the output is bothering me a bit - should it be `table_name` or `table` instead?\r\n\r\nProblem is I really like `name` for the input, so should it be consistent to have the same name on the output here, or should I aim for consistency with other endpoints that might return `table` rather than the ambiguous `name` elsewhere?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1435294468, "label": "`/db/-/create` API for creating tables"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1882#issuecomment-1312580348", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1882", "id": 1312580348, "node_id": "IC_kwDOBm6k_c5OPGL8", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-12T21:55:54Z", "updated_at": "2022-11-12T21:56:45Z", "author_association": "OWNER", "body": "What should this API return?\r\n\r\nI think the name of the table (`name`), the URL to that table (`table_url` - for consistency with how faceting API works already) and the schema of the table (`schema`).", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1435294468, "label": "`/db/-/create` API for creating tables"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1882#issuecomment-1312575048", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1882", "id": 1312575048, "node_id": "IC_kwDOBm6k_c5OPE5I", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-12T21:22:58Z", "updated_at": "2022-11-12T21:22:58Z", "author_association": "OWNER", "body": "Need to validate the table name. SQLite supports almost any table name - but they can't contain a newline character and cannot start with `sqlite_` - according to https://stackoverflow.com/questions/3694276/what-are-valid-table-names-in-sqlite/43049720#43049720", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1435294468, "label": "`/db/-/create` API for creating tables"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1882#issuecomment-1312556044", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1882", "id": 1312556044, "node_id": "IC_kwDOBm6k_c5OPAQM", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-12T19:29:11Z", "updated_at": "2022-11-12T19:29:11Z", "author_association": "OWNER", "body": "Thought of an edge-case: with `sqlite-utils` one feature I really like is that I can pipe data into it without caring if the table already exists or not:\r\n\r\n cat data.json | sqlite-utils insert my.db mytable -\r\n\r\nHow could this new API support that?\r\n\r\nI thought about adding a `\"create\": true` option to `/db/table/-/insert` which creates the table if it doesn't already exist, but if I do that I'll need to start adding other options to that endpoint - to set the primary key, add foreign keys and suchlike - which would be ignored except for the cases where the table was being created from scratch.\r\n\r\nThis doesn't feel right to me - I want to keep those options here, on `/db/-/create`.\r\n\r\nOne idea I had was to implement it such that you can call `/db/-/create` multiple times for the same table, but only if you are using the `\"rows\"` option. If so, and if the rows can be safely inserted, it would let you do that.\r\n\r\nBut instead, I'm going to outsource this to the CLI tool I plan to write that feeds data into this API. I'm already planning to use that tool for CSV inserts (so the API doesn't need to accept CSV directly). I think it's a good place for other usability enhancements like \"insert this, creating the table if it does not exist\" as well.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1435294468, "label": "`/db/-/create` API for creating tables"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1883#issuecomment-1311314981", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1883", "id": 1311314981, "node_id": "IC_kwDOBm6k_c5OKRQl", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-11T07:15:48Z", "updated_at": "2022-11-11T07:15:48Z", "author_association": "OWNER", "body": "I released that fix in Datasette 0.63.1: https://docs.datasette.io/en/stable/changelog.html#v0-63-1", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1435917503, "label": "Errors when using table filters behind a proxy"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1883#issuecomment-1311299535", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1883", "id": 1311299535, "node_id": "IC_kwDOBm6k_c5OKNfP", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-11T06:54:58Z", "updated_at": "2022-11-11T06:54:58Z", "author_association": "OWNER", "body": "This time deployed with:\r\n```\r\ncd demos/apache-proxy\r\nfly deploy --build-arg DATASETTE_REF=8d9a957c6329d26cc1e417b5d6911640d74765eb\r\n```\r\nTo ensure the exact commit with the fix.\r\n\r\nAnd that fixed it!\r\n```\r\n% curl -i 'https://datasette-apache-proxy-demo.datasette.io/prefix/fixtures/binary_data?_filter_column=rowid&_filter_op=exact&_filter_value=1&_sort=rowid'\r\nHTTP/2 302 \r\ndate: Fri, 11 Nov 2022 06:54:45 GMT\r\nserver: Fly/b1863e2e7 (2022-11-09)\r\nlocation: /prefix/fixtures/binary_data?_sort=rowid&rowid__exact=1\r\nlink: ; rel=preload\r\ncontent-type: text/plain\r\nx-proxied-by: Apache2 Debian\r\nvia: 2 fly.io\r\nfly-request-id: 01GHJQGBSXBR7E53TY0EKMQ9PA-sjc\r\n```\r\n\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1435917503, "label": "Errors when using table filters behind a proxy"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1883#issuecomment-1311292463", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1883", "id": 1311292463, "node_id": "IC_kwDOBm6k_c5OKLwv", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-11T06:44:24Z", "updated_at": "2022-11-11T06:44:24Z", "author_association": "OWNER", "body": "Modifying that test to the following does indeed cause a failure:\r\n```python\r\ndef test_base_url_affects_filter_redirects(app_client_base_url_prefix):\r\n response = app_client_base_url_prefix.get(\r\n \"/fixtures/binary_data?_filter_column=rowid&_filter_op=exact&_filter_value=1&_sort=rowid\"\r\n )\r\n assert response.status == 302\r\n assert (\r\n response.headers[\"location\"]\r\n == \"/prefix/fixtures/binary_data?_sort=rowid&rowid__exact=1\"\r\n )\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1435917503, "label": "Errors when using table filters behind a proxy"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1883#issuecomment-1311291632", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1883", "id": 1311291632, "node_id": "IC_kwDOBm6k_c5OKLjw", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-11T06:43:00Z", "updated_at": "2022-11-11T06:43:00Z", "author_association": "OWNER", "body": "https://datasette-apache-proxy-demo.datasette.io/prefix/-/asgi-scope is useful:\r\n\r\nIt confirms that `/prefix/` is nowhere to be seen in the incoming request data:\r\n\r\n```\r\n 'path': '/-/asgi-scope',\r\n 'query_string': b'',\r\n 'raw_path': b'/-/asgi-scope',\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1435917503, "label": "Errors when using table filters behind a proxy"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1883#issuecomment-1311290115", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1883", "id": 1311290115, "node_id": "IC_kwDOBm6k_c5OKLMD", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-11T06:40:14Z", "updated_at": "2022-11-11T06:41:56Z", "author_association": "OWNER", "body": "I modified that config file to have this line instead:\r\n```\r\n ProxyPass /prefix/ http://127.0.0.1:8001/ nocanon\r\n```\r\nAnd then deployed it by running:\r\n\r\n flyctl deploy --build-arg DATASETTE_REF=main\r\n\r\nThis does NOT seem to have fixed the bug:\r\n\r\n```\r\n~ % curl -i 'https://datasette-apache-proxy-demo.datasette.io/prefix/fixtures/binary_data?_filter_column=rowid&_filter_op=exact&_filter_value=1&_sort=rowid'\r\nHTTP/2 302 \r\ndate: Fri, 11 Nov 2022 06:40:01 GMT\r\nserver: Fly/b1863e2e7 (2022-11-09)\r\nlocation: /fixtures/binary_data?_sort=rowid&rowid__exact=1\r\nlink: ; rel=preload\r\ncontent-type: text/plain\r\nx-proxied-by: Apache2 Debian\r\nvia: 2 fly.io\r\nfly-request-id: 01GHJPNCF51CJ626EWZEHK2CH9-sjc\r\n```\r\n\r\nhttps://datasette-apache-proxy-demo.datasette.io/prefix/-/versions seems to confirm that this is the latest deployed version (0.63), so it looks like the deploy worked.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1435917503, "label": "Errors when using table filters behind a proxy"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1883#issuecomment-1311286593", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1883", "id": 1311286593, "node_id": "IC_kwDOBm6k_c5OKKVB", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-11T06:34:09Z", "updated_at": "2022-11-11T06:34:09Z", "author_association": "OWNER", "body": "https://httpd.apache.org/docs/2.4/mod/mod_proxy.html#proxypass includes this note:\r\n\r\n> Normally, mod_proxy will canonicalise ProxyPassed URLs. But this may be incompatible with some backends, particularly those that make use of *PATH_INFO*. The optional *nocanon* keyword suppresses this and passes the URL path \"raw\" to the backend.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1435917503, "label": "Errors when using table filters behind a proxy"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1883#issuecomment-1311284537", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1883", "id": 1311284537, "node_id": "IC_kwDOBm6k_c5OKJ05", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-11T06:30:38Z", "updated_at": "2022-11-11T06:30:38Z", "author_association": "OWNER", "body": "Is there a chance that it's Apache that's messing with that `location:` header here, not Datasette?\r\n\r\nhttps://github.com/simonw/datasette/blob/bbaab3b38ec2ce5944239ffbe2dd53328df40fff/demos/apache-proxy/000-default.conf#L7-L13", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1435917503, "label": "Errors when using table filters behind a proxy"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1883#issuecomment-1311283301", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1883", "id": 1311283301, "node_id": "IC_kwDOBm6k_c5OKJhl", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-11T06:28:38Z", "updated_at": "2022-11-11T06:29:33Z", "author_association": "OWNER", "body": "`path_with_added_args(request, redirect_params)` should be preserving the current path from the request.\r\n\r\nhttps://github.com/simonw/datasette/blob/bbaab3b38ec2ce5944239ffbe2dd53328df40fff/datasette/utils/__init__.py#L273-L286", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1435917503, "label": "Errors when using table filters behind a proxy"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1883#issuecomment-1311282970", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1883", "id": 1311282970, "node_id": "IC_kwDOBm6k_c5OKJca", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-11T06:28:05Z", "updated_at": "2022-11-11T06:28:05Z", "author_association": "OWNER", "body": "Relevant code: https://github.com/simonw/datasette/blob/bbaab3b38ec2ce5944239ffbe2dd53328df40fff/datasette/views/table.py#L227-L249", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1435917503, "label": "Errors when using table filters behind a proxy"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1883#issuecomment-1311280709", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1883", "id": 1311280709, "node_id": "IC_kwDOBm6k_c5OKI5F", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-11T06:25:27Z", "updated_at": "2022-11-11T06:25:27Z", "author_association": "OWNER", "body": "I tried adding this test but it passed! I expected it to fail:\r\n\r\n```python\r\ndef test_base_url_affects_filter_redirects(app_client_base_url_prefix):\r\n response = app_client_base_url_prefix.get(\r\n \"/prefix/fixtures/binary_data?_filter_column=rowid&_filter_op=exact&_filter_value=1&_sort=rowid\"\r\n )\r\n assert response.status == 302\r\n assert (\r\n response.headers[\"location\"]\r\n == \"/prefix/fixtures/binary_data?_sort=rowid&rowid__exact=1\"\r\n )\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1435917503, "label": "Errors when using table filters behind a proxy"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1883#issuecomment-1311278678", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1883", "id": 1311278678, "node_id": "IC_kwDOBm6k_c5OKIZW", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-11T06:22:37Z", "updated_at": "2022-11-11T06:22:37Z", "author_association": "OWNER", "body": "If you view source on that page the HTML looks correct:\r\n```html\r\n
    \r\n```\r\n(I just added a test that confirms this too.)\r\n\r\nBut... it looks like the bug is in the redirection code. \r\n\r\nhttps://datasette-apache-proxy-demo.datasette.io/prefix/fixtures/binary_data?_filter_column=rowid&_filter_op=exact&_filter_value=1&_sort=rowid returns the following:\r\n\r\n location: /fixtures/binary_data?_sort=rowid&rowid__exact=1\r\n\r\nWhich is incorrect.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1435917503, "label": "Errors when using table filters behind a proxy"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1883#issuecomment-1311273461", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1883", "id": 1311273461, "node_id": "IC_kwDOBm6k_c5OKHH1", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-11T06:16:08Z", "updated_at": "2022-11-11T06:16:08Z", "author_association": "OWNER", "body": "Great catch, thanks!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1435917503, "label": "Errors when using table filters behind a proxy"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1880#issuecomment-1311273063", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1880", "id": 1311273063, "node_id": "IC_kwDOBm6k_c5OKHBn", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-11T06:15:28Z", "updated_at": "2022-11-11T06:15:28Z", "author_association": "OWNER", "body": "The `_internal` database is intended to help Datasette handle much larger attached databases. Right now Datasette attempts to show every database on the https://latest.datasette.io/ index page and every table on the https://latest.datasette.io/fixtures database index page - but these are not paginated. If you had a database containing 1,000 tables the database index page would get pretty slow.\r\n\r\nSo I want to be able to paginate (and search) those. But to paginate them it's useful to have them in a database table itself, since then I can paginate using SQL.\r\n\r\nMy plan for `_internal` is to use it to implement those advanced browsing features. I've not completed this work yet though. See this issue for more details on that:\r\n\r\n- #417", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1433576351, "label": "Datasette with many and large databases > Memory use"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1880#issuecomment-1311271298", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1880", "id": 1311271298, "node_id": "IC_kwDOBm6k_c5OKGmC", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-11T06:12:29Z", "updated_at": "2022-11-11T06:12:29Z", "author_association": "OWNER", "body": "I think you may have misunderstood this feature. This is talking about the `_internal` in-memory database, which maintains a set of tables that list the databases and tables that are attached to Datasette.\r\n\r\nThey're not a copy of the data itself - just a list of table names, column names and database names.\r\n\r\nYou can see what that database looks like by signing in as root - running `datasette --root` and clicking the link. Or you can see an example here:\r\n\r\n- Click the button on https://latest.datasette.io/login-as-root\r\n- Now visit https://latest.datasette.io/_internal\r\n\r\nFor the example instance that looks like this:\r\n\r\n\"image\"\r\n\r\nThe two most interesting tables in there are these ones:\r\n\r\n\"image\"\r\n\r\n\"CleanShot\r\n\r\nAs you can see, it's just the table schema itself and the columns that make up the tables. Even if you have hundreds of databases connected each with hundreds of tables this should still only add up to a few MB of RAM.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1433576351, "label": "Datasette with many and large databases > Memory use"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1884#issuecomment-1311269045", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1884", "id": 1311269045, "node_id": "IC_kwDOBm6k_c5OKGC1", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-11T06:08:28Z", "updated_at": "2022-11-11T06:08:28Z", "author_association": "OWNER", "body": "Does that work if you add `--load-extension spatialite`?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1439009231, "label": "Exclude virtual tables from datasette inspect"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1882#issuecomment-1302818784", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1882", "id": 1302818784, "node_id": "IC_kwDOBm6k_c5Np2_g", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-04T00:25:18Z", "updated_at": "2022-11-04T16:12:39Z", "author_association": "OWNER", "body": "On that basis I think the core API design should change to this:\r\n```\r\nPOST /db/-/create\r\nAuthorization: Bearer xxx\r\nContent-Type: application/json\r\n{\r\n \"name\": \"my new table\",\r\n \"columns\": [\r\n {\r\n \"name\": \"id\",\r\n \"type\": \"integer\"\r\n },\r\n {\r\n \"name\": \"title\",\r\n \"type\": \"text\"\r\n }\r\n ]\r\n \"pk\": \"id\"\r\n}\r\n```\r\nThis leaves room for a `\"rows\": []` key at the root too. Having that as a child of `\"table\"` felt unintuitive to me, and I didn't like the way this looked either:\r\n\r\n```json\r\n{\r\n \"table\": {\r\n \"name\": \"my_new_table\"\r\n },\r\n \"rows\": [\r\n {\"id\": 1, \"title\": \"Title\"}\r\n ]\r\n}\r\n```\r\nWeird to have the table `name` nested inside `table` when `rows` wasn't.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1435294468, "label": "`/db/-/create` API for creating tables"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1882#issuecomment-1302818153", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1882", "id": 1302818153, "node_id": "IC_kwDOBm6k_c5Np21p", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-04T00:23:58Z", "updated_at": "2022-11-04T00:23:58Z", "author_association": "OWNER", "body": "I made a decision here that this endpoint should also accept an optional `\"rows\": [...]` list which is used to automatically create the table using a schema derived from those example rows (which then get inserted):\r\n\r\n- https://github.com/simonw/datasette/issues/1862#issuecomment-1302817807", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1435294468, "label": "`/db/-/create` API for creating tables"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1862#issuecomment-1302817807", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1862", "id": 1302817807, "node_id": "IC_kwDOBm6k_c5Np2wP", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-04T00:23:13Z", "updated_at": "2022-11-04T00:23:13Z", "author_association": "OWNER", "body": "I don't like this on `/db/table/-/insert` - I think it makes more sense to optionally pass a `\"rows\"` key to the `/db/-/create` endpoint instead.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1425011030, "label": "Create a new table from one or more records, `sqlite-utils` style"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1862#issuecomment-1302817500", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1862", "id": 1302817500, "node_id": "IC_kwDOBm6k_c5Np2rc", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-04T00:22:31Z", "updated_at": "2022-11-04T00:22:31Z", "author_association": "OWNER", "body": "Maybe this is a feature added to the existing `/db/table/-/insert` endpoint?\r\n\r\nBit weird that you can call that endpoint for a table that doesn't exist yet, but it fits the `sqlite-utils` way of creating tables which I've found very pleasant over the past few years.\r\n\r\nSo perhaps the API looks like this:\r\n\r\n```\r\nPOST ///-/insert\r\nContent-Type: application/json\r\nAuthorization: Bearer dstok_\r\n{\r\n \"create_table\": true,\r\n \"rows\": [\r\n {\r\n \"column1\": \"value1\",\r\n \"column2\": \"value2\"\r\n },\r\n {\r\n \"column1\": \"value3\",\r\n \"column2\": \"value4\"\r\n }\r\n ]\r\n}\r\n```\r\nThe `create_table` option will cause the table to be created if it doesn't already exist.\r\n\r\nThat means I probably also need a `\"pk\": \"...\"` column for setting a primary key if the table is being created ... and maybe other options that I invent for this other feature too?\r\n- #1882", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1425011030, "label": "Create a new table from one or more records, `sqlite-utils` style"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1871#issuecomment-1302815105", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1871", "id": 1302815105, "node_id": "IC_kwDOBm6k_c5Np2GB", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-04T00:17:23Z", "updated_at": "2022-11-04T00:17:23Z", "author_association": "OWNER", "body": "I'll probably enhance it a bit more though, I want to provide a UI that lists all the tables you can explore and lets you click to pre-fill the forms with them.\r\n\r\nThough at that point what should I do about the other endpoints? Probably list those too. Gets a bit complex, especially with the row-level update and delete endpoints.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1427293909, "label": "API explorer tool"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1871#issuecomment-1302814693", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1871", "id": 1302814693, "node_id": "IC_kwDOBm6k_c5Np1_l", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-04T00:16:36Z", "updated_at": "2022-11-04T00:16:36Z", "author_association": "OWNER", "body": "I can close this issue once I fix it so it no longer hard-codes a potentially invalid example endpoint:\r\n\r\nhttps://github.com/simonw/datasette/blob/bcc781f4c50a8870e3389c4e60acb625c34b0317/datasette/templates/api_explorer.html#L24-L26\r\n\r\nhttps://github.com/simonw/datasette/blob/bcc781f4c50a8870e3389c4e60acb625c34b0317/datasette/templates/api_explorer.html#L34-L35", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1427293909, "label": "API explorer tool"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1881#issuecomment-1302813449", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1881", "id": 1302813449, "node_id": "IC_kwDOBm6k_c5Np1sJ", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-04T00:14:07Z", "updated_at": "2022-11-04T00:14:07Z", "author_association": "OWNER", "body": "Tool is now live here: https://latest-1-0-dev.datasette.io/-/permissions\r\n\r\nNeeds root perms, so access this first: https://latest-1-0-dev.datasette.io/login-as-root", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1434094365, "label": "Tool for simulating permission checks against actors"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1881#issuecomment-1302812918", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1881", "id": 1302812918, "node_id": "IC_kwDOBm6k_c5Np1j2", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-04T00:13:05Z", "updated_at": "2022-11-04T00:13:05Z", "author_association": "OWNER", "body": "Has tests now.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1434094365, "label": "Tool for simulating permission checks against actors"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1863#issuecomment-1302790013", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1863", "id": 1302790013, "node_id": "IC_kwDOBm6k_c5Npv99", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-03T23:32:30Z", "updated_at": "2022-11-03T23:32:30Z", "author_association": "OWNER", "body": "I'm not going to allow updates to primary keys. If you need to do that, you can instead delete the record and then insert a new one with the new primary keys you wanted - or maybe use a custom SQL query.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1425029242, "label": "Update a single record in an existing table"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1851#issuecomment-1294224185", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1851", "id": 1294224185, "node_id": "IC_kwDOBm6k_c5NJEs5", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-10-27T23:18:24Z", "updated_at": "2022-11-03T23:26:05Z", "author_association": "OWNER", "body": "So new API design is:\r\n\r\n```\r\nPOST /db/table/-/insert\r\nAuthorization: Bearer xxx\r\nContent-Type: application/json\r\n{\r\n \"row\": {\r\n \"id\": 1,\r\n \"name\": \"New record\"\r\n }\r\n}\r\n```\r\nReturns:\r\n```\r\n201 Created\r\n{\r\n \"row\": [{\r\n \"id\": 1,\r\n \"name\": \"New record\"\r\n }]\r\n}\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1421544654, "label": "API to insert a single record into an existing table"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1863#issuecomment-1302785086", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1863", "id": 1302785086, "node_id": "IC_kwDOBm6k_c5Npuw-", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-03T23:24:33Z", "updated_at": "2022-11-03T23:24:56Z", "author_association": "OWNER", "body": "Thinking more about validation: I'm considering if this should validate that columns which are defined as SQLite foreign keys are being updated to values that exist in those other tables.\r\n\r\nI like the sound of this. It seems like a sensible default behaviour for Datasette. And it fits with the fact that Datasette treats foreign keys specially elsewhere in the interface.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1425029242, "label": "Update a single record in an existing table"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1863#issuecomment-1302760549", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1863", "id": 1302760549, "node_id": "IC_kwDOBm6k_c5Npoxl", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-03T22:43:04Z", "updated_at": "2022-11-03T23:21:31Z", "author_association": "OWNER", "body": "The `id=(int, ...)` thing is weird, but is apparently Pydantic syntax for a required field?\r\n\r\nhttps://cs.github.com/starlite-api/starlite/blob/28ddc847c4cb072f0d5d21a9ecd5259711f12ec9/docs/usage/11-data-transfer-objects.md#L161 confirms:\r\n\r\n> 1. For required fields use a tuple of type + ellipsis, for example `(str, ...)`.\r\n> 2. For optional fields use a tuple of type + `None`, for example `(str, None)`\r\n> 3. To set a default value use a tuple of type + default value, for example `(str, \"Hello World\")`", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1425029242, "label": "Update a single record in an existing table"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1863#issuecomment-1302760382", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1863", "id": 1302760382, "node_id": "IC_kwDOBm6k_c5Npou-", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-03T22:42:47Z", "updated_at": "2022-11-03T22:42:47Z", "author_association": "OWNER", "body": "```python\r\nprint(create_model('document', id=(int, ...), title=(str, None)).schema_json(indent=2))\r\n```\r\n```json\r\n{\r\n \"title\": \"document\",\r\n \"type\": \"object\",\r\n \"properties\": {\r\n \"id\": {\r\n \"title\": \"Id\",\r\n \"type\": \"integer\"\r\n },\r\n \"title\": {\r\n \"title\": \"Title\",\r\n \"type\": \"string\"\r\n }\r\n },\r\n \"required\": [\r\n \"id\"\r\n ]\r\n}\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1425029242, "label": "Update a single record in an existing table"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1863#issuecomment-1302759174", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1863", "id": 1302759174, "node_id": "IC_kwDOBm6k_c5NpocG", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-03T22:40:47Z", "updated_at": "2022-11-03T22:40:47Z", "author_association": "OWNER", "body": "I'm considering Pydantic for this, see:\r\n- https://github.com/simonw/datasette/issues/1882#issuecomment-1302716350\r\n\r\nIn particular the `create_model()` method: https://pydantic-docs.helpmanual.io/usage/models/#dynamic-model-creation\r\n\r\nThis would give me good validation. It would also, weirdly, give me the ability to output JSON schema. Maybe I could have this as the JSON schema for a row?\r\n\r\n`/db/table/-/json-schema`", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1425029242, "label": "Update a single record in an existing table"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1882#issuecomment-1302716350", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1882", "id": 1302716350, "node_id": "IC_kwDOBm6k_c5Npd--", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-03T21:51:14Z", "updated_at": "2022-11-03T22:35:54Z", "author_association": "OWNER", "body": "Validating this JSON object is getting a tiny bit complex. I'm tempted to adopt https://pydantic-docs.helpmanual.io/ at this point.\r\n\r\nThe `create_model` example on https://stackoverflow.com/questions/66168517/generate-dynamic-model-using-pydantic/66168682#66168682 is particularly relevant, especially when I work on this issue:\r\n\r\n- #1863\r\n\r\n```python\r\nfrom pydantic import create_model\r\n\r\nd = {\"strategy\": {\"name\": \"test_strat2\", \"periods\": 10}}\r\n\r\nStrategy = create_model(\"Strategy\", **d[\"strategy\"])\r\n\r\nprint(Strategy.schema_json(indent=2))\r\n```\r\n`create_model()`: https://pydantic-docs.helpmanual.io/usage/models/#dynamic-model-creation", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1435294468, "label": "`/db/-/create` API for creating tables"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1882#issuecomment-1302721916", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1882", "id": 1302721916, "node_id": "IC_kwDOBm6k_c5NpfV8", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-03T21:58:50Z", "updated_at": "2022-11-03T21:59:17Z", "author_association": "OWNER", "body": "Mocked up a quick HTML+JavaScript form for creating that JSON structure using some iteration against Copilot prompts:\r\n```html\r\n
    \r\n/* JSON format:\r\n{\r\n  \"table\": {\r\n      \"name\": \"my new table\",\r\n      \"columns\": [\r\n          {\r\n              \"name\": \"id\",\r\n              \"type\": \"integer\"\r\n          },\r\n          {\r\n              \"name\": \"title\",\r\n              \"type\": \"text\"\r\n          }\r\n      ]\r\n     \"pk\": \"id\"\r\n  }\r\n}\r\n\r\nHTML form with Javascript for creating this JSON:\r\n*/
    \r\n\r\n \r\n
    \r\n \r\n
    \r\n \r\n \r\n \r\n \r\n \r\n

    Current columns:

    \r\n
      \r\n \r\n\r\n\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1435294468, "label": "`/db/-/create` API for creating tables"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1882#issuecomment-1302715662", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1882", "id": 1302715662, "node_id": "IC_kwDOBm6k_c5Npd0O", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-03T21:50:27Z", "updated_at": "2022-11-03T21:50:27Z", "author_association": "OWNER", "body": "API design for this:\r\n```\r\nPOST /db/-/create\r\nAuthorization: Bearer xxx\r\nContent-Type: application/json\r\n{\r\n \"table\": {\r\n \"name\": \"my new table\",\r\n \"columns\": [\r\n {\r\n \"name\": \"id\",\r\n \"type\": \"integer\"\r\n },\r\n {\r\n \"name\": \"title\",\r\n \"type\": \"text\"\r\n }\r\n ]\r\n \"pk\": \"id\"\r\n }\r\n}\r\n```\r\nSupported column types are:\r\n\r\n- `integer`\r\n- `text`\r\n- `float` (even though SQLite calls it a \"real\")\r\n- `blob`\r\n\r\nThis matches my design for `sqlite-utils`: https://sqlite-utils.datasette.io/en/stable/cli.html#cli-create-table", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1435294468, "label": "`/db/-/create` API for creating tables"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1843#issuecomment-1302679026", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1843", "id": 1302679026, "node_id": "IC_kwDOBm6k_c5NpU3y", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-03T21:22:42Z", "updated_at": "2022-11-03T21:22:42Z", "author_association": "OWNER", "body": "Docs for the new `db.close()` method: https://docs.datasette.io/en/latest/internals.html#db-close", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1408757705, "label": "Intermittent \"Too many open files\" error running tests"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1843#issuecomment-1302678384", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1843", "id": 1302678384, "node_id": "IC_kwDOBm6k_c5NpUtw", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-03T21:21:59Z", "updated_at": "2022-11-03T21:21:59Z", "author_association": "OWNER", "body": "I added extra debug info to `/-/threads` to see this for myself:\r\n\r\n```diff\r\ndiff --git a/datasette/app.py b/datasette/app.py\r\nindex 02bd38f1..16579e28 100644\r\n--- a/datasette/app.py\r\n+++ b/datasette/app.py\r\n@@ -969,6 +969,13 @@ class Datasette:\r\n \"threads\": [\r\n {\"name\": t.name, \"ident\": t.ident, \"daemon\": t.daemon} for t in threads\r\n ],\r\n+ \"file_connections\": {\r\n+ db.name: [\r\n+ [dict(r) for r in conn.execute(\"pragma database_list\").fetchall()]\r\n+ for conn in db._all_file_connections\r\n+ ]\r\n+ for db in self.databases.values()\r\n+ },\r\n }\r\n # Only available in Python 3.7+\r\n if hasattr(asyncio, \"all_tasks\"):\r\n```\r\nOutput after hitting refresh on a few `/fixtures` tables to ensure more threads started:\r\n\r\n```\r\n \"file_connections\": {\r\n \"_internal\": [],\r\n \"fixtures\": [\r\n [\r\n {\r\n \"seq\": 0,\r\n \"name\": \"main\",\r\n \"file\": \"/Users/simon/Dropbox/Development/datasette/fixtures.db\"\r\n }\r\n ],\r\n [\r\n {\r\n \"seq\": 0,\r\n \"name\": \"main\",\r\n \"file\": \"/Users/simon/Dropbox/Development/datasette/fixtures.db\"\r\n }\r\n ],\r\n [\r\n {\r\n \"seq\": 0,\r\n \"name\": \"main\",\r\n \"file\": \"/Users/simon/Dropbox/Development/datasette/fixtures.db\"\r\n }\r\n ]\r\n ]\r\n },\r\n```\r\nI decided not to ship this feature though as it leaks the names of internal database files.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1408757705, "label": "Intermittent \"Too many open files\" error running tests"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1843#issuecomment-1302634332", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1843", "id": 1302634332, "node_id": "IC_kwDOBm6k_c5NpJ9c", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-03T20:34:56Z", "updated_at": "2022-11-03T20:34:56Z", "author_association": "OWNER", "body": "Confirmed that calling `conn.close()` on each SQLite file-based connection is the way to fix this problem.\r\n\r\nI'm adding a `db.close()` method (sync, not async - I tried async first but it was really hard to cause every thread in the pool to close its threadlocal database connection) which loops through all known open file-based connections and closes them.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1408757705, "label": "Intermittent \"Too many open files\" error running tests"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1843#issuecomment-1302574330", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1843", "id": 1302574330, "node_id": "IC_kwDOBm6k_c5No7T6", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-03T19:30:22Z", "updated_at": "2022-11-03T19:30:22Z", "author_association": "OWNER", "body": "This is affecting me a lot at the moment, on my laptop (runs fine in CI).\r\n\r\nHere's a change to `conftest.py` which highlights the problem - it cause a failure the moment there are more than 5 open files according to `psutil`:\r\n\r\n```diff\r\ndiff --git a/tests/conftest.py b/tests/conftest.py\r\nindex f4638a14..21d433c1 100644\r\n--- a/tests/conftest.py\r\n+++ b/tests/conftest.py\r\n@@ -1,6 +1,7 @@\r\n import httpx\r\n import os\r\n import pathlib\r\n+import psutil\r\n import pytest\r\n import re\r\n import subprocess\r\n@@ -192,3 +193,8 @@ def ds_unix_domain_socket_server(tmp_path_factory):\r\n yield ds_proc, uds\r\n # Shut it down at the end of the pytest session\r\n ds_proc.terminate()\r\n+\r\n+\r\n+def pytest_runtest_teardown(item: pytest.Item) -> None:\r\n+ open_files = psutil.Process().open_files()\r\n+ assert len(open_files) < 5\r\n```\r\nThe first error I get from this with `pytest --pdb -x` is here:\r\n\r\n```\r\ntests/test_api.py ............E\r\n>>>>> traceback >>>>>\r\n\r\nitem = \r\n\r\n def pytest_runtest_teardown(item: pytest.Item) -> None:\r\n open_files = psutil.Process().open_files()\r\n> assert len(open_files) < 5\r\nE AssertionError: assert 5 < 5\r\nE + where 5 = len([popenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmpfglrt4p2/fixtures.db', fd=14), popenfile(... fd=19), popenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmphdi5b250/fixtures.dot.db', fd=20)])\r\n\r\n/Users/simon/Dropbox/Development/datasette/tests/conftest.py:200: AssertionError\r\n>>>>> entering PDB >>>>>\r\n\r\n>>>>> PDB post_mortem (IO-capturing turned off) >>>>>\r\n> /Users/simon/Dropbox/Development/datasette/tests/conftest.py(200)pytest_runtest_teardown()\r\n-> assert len(open_files) < 5\r\n```\r\nThat's this test:\r\n\r\nhttps://github.com/simonw/datasette/blob/2ec5583629005b32cb0877786f9681c5d43ca33f/tests/test_api.py#L656-L673\r\n\r\nWhich uses this fixture:\r\n\r\nhttps://github.com/simonw/datasette/blob/2ec5583629005b32cb0877786f9681c5d43ca33f/tests/fixtures.py#L228-L231\r\n\r\nWhich calls this function:\r\n\r\nhttps://github.com/simonw/datasette/blob/2ec5583629005b32cb0877786f9681c5d43ca33f/tests/fixtures.py#L105-L122\r\n\r\nSo now I'm suspicious that, even though the fixture is meant to be session scoped, the way I'm using `with tempfile.TemporaryDirectory() as tmpdir:` is causing a whole load of files to be created and held open which are not later closed.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1408757705, "label": "Intermittent \"Too many open files\" error running tests"}, "performed_via_github_app": null}