{"html_url": "https://github.com/simonw/datasette/issues/1875#issuecomment-1314491150", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1875", "id": 1314491150, "node_id": "IC_kwDOBm6k_c5OWYsO", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-14T22:25:20Z", "updated_at": "2022-11-14T22:25:20Z", "author_association": "OWNER", "body": "That's using JSON Pointer: https://www.rfc-editor.org/rfc/rfc6901\r\n\r\nThere's a Python library for that here https://github.com/stefankoegl/python-json-pointer/blob/master/jsonpointer.py - which looks simple and clean and well maintained and documented, but it only handles the \"what is at this pointer within this JSON object\" case - I need to generate the correct JSON pointer to explain where my error is.\r\n\r\nSo I think I'll end up hand-rolling this.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1430797211, "label": "Figure out design for JSON errors (consider RFC 7807)"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1875#issuecomment-1314488010", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1875", "id": 1314488010, "node_id": "IC_kwDOBm6k_c5OWX7K", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-14T22:21:43Z", "updated_at": "2022-11-14T22:21:43Z", "author_association": "OWNER", "body": "Here's the most relevant example from the RFC spec:\r\n```\r\n   POST /details HTTP/1.1\r\n   Host: account.example.com\r\n   Accept: application/json\r\n```\r\n```json\r\n   {\r\n     \"age\": 42.3,\r\n     \"profile\": {\r\n       \"color\": \"yellow\"\r\n     }\r\n   }\r\n```\r\n```\r\n   HTTP/1.1 400 Bad Request\r\n   Content-Type: application/problem+json\r\n   Content-Language: en\r\n```\r\n```json\r\n{\r\n    \"type\": \"https://example.net/validation-error\",\r\n    \"title\": \"Your request is not valid.\",\r\n    \"errors\": [\r\n        {\r\n            \"detail\": \"must be a positive integer\",\r\n            \"pointer\": \"#/age\"\r\n        },\r\n        {\r\n            \"detail\": \"must be 'green', 'red' or 'blue'\",\r\n            \"pointer\": \"#/profile/color\"\r\n        }\r\n    ]\r\n}\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1430797211, "label": "Figure out design for JSON errors (consider RFC 7807)"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1884#issuecomment-1314054300", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1884", "id": 1314054300, "node_id": "IC_kwDOBm6k_c5OUuCc", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-14T16:40:06Z", "updated_at": "2022-11-14T16:40:06Z", "author_association": "OWNER", "body": "I wonder if there are any reasons that inspect SHOULD try to count virtual tables? Like are there any likely uses for a cirial table where the count is both interesting and likely to be accessed often enough that it's worth caching?\r\n\r\nI have an issue open to add a setting to disable table counts entirely:\r\n\r\n- #1818 \r\n\r\nMaybe that should be expanded to automatically disable row counts for virtual tables entirely? Which would mean no count would be shown for them in the UI.\r\n\r\nIf you desperately wanted a count you would  then have to run a count(*) query against them explicitly.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1439009231, "label": "Exclude virtual tables from datasette inspect"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1850#issuecomment-1313156167", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1850", "id": 1313156167, "node_id": "IC_kwDOBm6k_c5ORSxH", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-14T06:23:39Z", "updated_at": "2022-11-14T06:23:39Z", "author_association": "OWNER", "body": "The API explorer is now live here: https://latest-1-0-dev.datasette.io/-/api", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1421529723, "label": "Write API in Datasette core"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1850#issuecomment-1313155712", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1850", "id": 1313155712, "node_id": "IC_kwDOBm6k_c5ORSqA", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-14T06:22:57Z", "updated_at": "2022-11-14T06:22:57Z", "author_association": "OWNER", "body": "I think the ability to create tokens should be protected by a `create-tokens` permission, not just a global setting.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1421529723, "label": "Write API in Datasette core"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1888#issuecomment-1313139657", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1888", "id": 1313139657, "node_id": "IC_kwDOBm6k_c5OROvJ", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-14T06:04:48Z", "updated_at": "2022-11-14T06:04:48Z", "author_association": "OWNER", "body": "Demo: https://latest-1-0-dev.datasette.io/-/api", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1447439985, "label": "API explorer should take immutability into account"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1866#issuecomment-1313128913", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1866", "id": 1313128913, "node_id": "IC_kwDOBm6k_c5ORMHR", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-14T05:48:22Z", "updated_at": "2022-11-14T05:48:22Z", "author_association": "OWNER", "body": "I changed my mind about the `\"return_rows\": true` option - I'm going to rename it to `\"return\": true`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1426001541, "label": "API for bulk inserting records into a table"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1874#issuecomment-1313127054", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1874", "id": 1313127054, "node_id": "IC_kwDOBm6k_c5ORLqO", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-14T05:45:00Z", "updated_at": "2022-11-14T05:45:00Z", "author_association": "OWNER", "body": "Demo: https://latest-1-0-dev.datasette.io/-/api#path=%2Ffixtures%2Ffacetable%2F-%2Fdrop&json=&method=POST\r\n\r\n<img width=\"974\" alt=\"image\" src=\"https://user-images.githubusercontent.com/9599/201584446-fd9d3978-d37d-4e81-8179-cd587027df46.png\">\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1429030341, "label": "API to drop a table"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1871#issuecomment-1313125870", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1871", "id": 1313125870, "node_id": "IC_kwDOBm6k_c5ORLXu", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-14T05:42:50Z", "updated_at": "2022-11-14T05:42:50Z", "author_association": "OWNER", "body": "Demo: https://latest-1-0-dev.datasette.io/-/api#path=%2Ffixtures%2Ffacetable%2F-%2Fdrop&json=%7B%22confirm%22%3A+true%7D&method=POST", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1427293909, "label": "API explorer tool"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1874#issuecomment-1313125123", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1874", "id": 1313125123, "node_id": "IC_kwDOBm6k_c5ORLMD", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-14T05:41:20Z", "updated_at": "2022-11-14T05:42:23Z", "author_association": "OWNER", "body": "I also changed the confirmation JSON returned by this endpoint to add the `database` and `table` like so:\r\n```json\r\n{\r\n    \"ok\": true,\r\n    \"database\": \"data\",\r\n    \"table\": \"docs\",\r\n    \"row_count\": 1,\r\n    \"message\": \"Pass \\\"confirm\\\": true to confirm\"\r\n}\r\n```\r\nUpdated docs: https://docs.datasette.io/en/1.0-dev/json_api.html#dropping-tables", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1429030341, "label": "API to drop a table"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1874#issuecomment-1313119558", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1874", "id": 1313119558, "node_id": "IC_kwDOBm6k_c5ORJ1G", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-14T05:30:27Z", "updated_at": "2022-11-14T05:30:27Z", "author_association": "OWNER", "body": "Found a bug: you get a 500 error if you try this against an immutable database.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1429030341, "label": "API to drop a table"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1850#issuecomment-1313115059", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1850", "id": 1313115059, "node_id": "IC_kwDOBm6k_c5ORIuz", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-14T05:21:30Z", "updated_at": "2022-11-14T05:21:30Z", "author_association": "OWNER", "body": "New documentation for these features currently lives here: https://docs.datasette.io/en/1.0-dev/json_api.html#the-json-write-api", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1421529723, "label": "Write API in Datasette core"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1875#issuecomment-1313114283", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1875", "id": 1313114283, "node_id": "IC_kwDOBm6k_c5ORIir", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-14T05:20:00Z", "updated_at": "2022-11-14T05:20:00Z", "author_association": "OWNER", "body": "I started a conversation about JSON error standards on Mastodon here: https://fedi.simonwillison.net/web/@simon/109338725610487457\r\n\r\nQuite a few people pointed to this RFC independently.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1430797211, "label": "Figure out design for JSON errors (consider RFC 7807)"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1887#issuecomment-1313113642", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1887", "id": 1313113642, "node_id": "IC_kwDOBm6k_c5ORIYq", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-14T05:18:51Z", "updated_at": "2022-11-14T05:18:51Z", "author_association": "OWNER", "body": "Updated docs: https://docs.datasette.io/en/1.0-dev/json_api.html#dropping-tables", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1447388809, "label": "Add a confirm step to the drop table API"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1887#issuecomment-1313097713", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1887", "id": 1313097713, "node_id": "IC_kwDOBm6k_c5OREfx", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-14T05:00:54Z", "updated_at": "2022-11-14T05:00:54Z", "author_association": "OWNER", "body": "I'm going to add a `\"confirm\": true` option to the API. Without that, it returns a note about how many rows will be deleted.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1447388809, "label": "Add a confirm step to the drop table API"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1871#issuecomment-1313097057", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1871", "id": 1313097057, "node_id": "IC_kwDOBm6k_c5OREVh", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-14T04:59:28Z", "updated_at": "2022-11-14T04:59:28Z", "author_association": "OWNER", "body": "In playing with the API explorer just now I realized it's way too easy to accidentally drop a table using it.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1427293909, "label": "API explorer tool"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1871#issuecomment-1313072900", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1871", "id": 1313072900, "node_id": "IC_kwDOBm6k_c5OQ-cE", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-14T04:15:50Z", "updated_at": "2022-11-14T04:15:50Z", "author_association": "OWNER", "body": "For the example links - I'm going to have these at the bottom of the page so you don't have to scroll past them.\r\n\r\nIdeally these would take the user's permissions into account. This could make the page expensive to load, but I'm going to risk it for the moment.\r\n\r\nSomething like this then:\r\n\r\n> - data\r\n>   - /data/-/create - create table\r\n>   - /data/table1/-/insert - insert into table1\r\n>   - /data/table1/-/drop - drop table1\r\n\r\nI won't bother with per-row demo links (for update and delete) because there could be thousands of them for each table.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1427293909, "label": "API explorer tool"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1871#issuecomment-1313062699", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1871", "id": 1313062699, "node_id": "IC_kwDOBm6k_c5OQ78r", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-14T04:03:29Z", "updated_at": "2022-11-14T04:12:41Z", "author_association": "OWNER", "body": "Two things left before I close this issue:\r\n\r\n- [x] I want to preserve the state of the forms in the URL - probably after a `#`\r\n- [ ] Instead of hard-coding the current examples, I want to provide a list of links which populate the forms", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1427293909, "label": "API explorer tool"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1886#issuecomment-1313052863", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1886", "id": 1313052863, "node_id": "IC_kwDOBm6k_c5OQ5i_", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-14T03:40:50Z", "updated_at": "2022-11-14T03:40:50Z", "author_association": "OWNER", "body": "Tim Sherratt on Twitter: https://twitter.com/wragge/status/1591930345469153282\r\n\r\n> Where do I start? The [#GLAMWorkbench](https://twitter.com/hashtag/GLAMWorkbench?src=hashtag_click) now includes a number of examples where GLAM data is harvested, processed, and then made available for exploration via Datasette.\r\n>\r\n> https://glam-workbench.net/\r\n>\r\n> For example the GLAM Name Index Search brings together 10+ million entries from 240 indexes and provides an aggregated search using the Datasette search-all plugin:\r\n>\r\n> https://glam-workbench.net/name-search/\r\n>\r\n> Most recently I converted PDFs of the Tasmanian Postal Directories to a big Datasette instance:  https://updates.timsherratt.org/2022/09/15/from-pdfs-to.html the process is documented and reusable.", "reactions": "{\"total_count\": 1, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 1, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1447050738, "label": "Call for birthday presents: if you're using Datasette, let us know how you're using it here"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1871#issuecomment-1312822353", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1871", "id": 1312822353, "node_id": "IC_kwDOBm6k_c5OQBRR", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-13T21:07:40Z", "updated_at": "2022-11-13T21:07:40Z", "author_association": "OWNER", "body": "I'm going to need extra code to toggle POST closed when GET opens and vice-versa.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1427293909, "label": "API explorer tool"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1871#issuecomment-1312821031", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1871", "id": 1312821031, "node_id": "IC_kwDOBm6k_c5OQA8n", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-13T21:02:06Z", "updated_at": "2022-11-13T21:03:11Z", "author_association": "OWNER", "body": "Actually no, I'm going to add a class of `details-menu` to the other details elements that SHOULD be closed. That way custom templates using `<details>` won't close in a surprising way.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1427293909, "label": "API explorer tool"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1871#issuecomment-1312816451", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1871", "id": 1312816451, "node_id": "IC_kwDOBm6k_c5OP_1D", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-13T20:39:26Z", "updated_at": "2022-11-13T20:39:34Z", "author_association": "OWNER", "body": "I'm going to add a special `no-auto-close` class to these and teach that code not to close them.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1427293909, "label": "API explorer tool"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1871#issuecomment-1312816292", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1871", "id": 1312816292, "node_id": "IC_kwDOBm6k_c5OP_yk", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-13T20:38:42Z", "updated_at": "2022-11-13T20:38:42Z", "author_association": "OWNER", "body": "The current API explorer uses details/summary elements for the GET and POST dialogs.\r\n\r\nI only want one of these to be open at a time, to reflect that you can make either a GET or a POST.\r\n\r\nI just noticed that clicking anywhere else on the page closes both elements, which isn't what I want to happen. Turns out that's because of this code I added as part of Datasette's menu implementation!\r\n\r\nhttps://github.com/simonw/datasette/blob/9f54f00a50a4d950cfd69a0ff3526ae82c858826/datasette/templates/_close_open_menus.html#L2-L15", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1427293909, "label": "API explorer tool"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1882#issuecomment-1312582512", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1882", "id": 1312582512, "node_id": "IC_kwDOBm6k_c5OPGtw", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-12T22:11:18Z", "updated_at": "2022-11-12T22:11:18Z", "author_association": "OWNER", "body": "I like this:\r\n\r\n<img width=\"1080\" alt=\"image\" src=\"https://user-images.githubusercontent.com/9599/201496508-2fe102e8-f0de-4dd8-87c5-40b6aae97413.png\">\r\n\r\n```json\r\n{\r\n  \"ok\": true,\r\n  \"database\": \"data\",\r\n  \"table\": \"agai2n\",\r\n  \"table_url\": \"http://127.0.0.1:8001/data/agai2n\",\r\n  \"schema\": \"CREATE TABLE [agai2n] (\\n   [hello] INTEGER\\n)\"\r\n}\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1435294468, "label": "`/db/-/create` API for creating tables"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1882#issuecomment-1312581121", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1882", "id": 1312581121, "node_id": "IC_kwDOBm6k_c5OPGYB", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-12T22:01:32Z", "updated_at": "2022-11-12T22:01:32Z", "author_association": "OWNER", "body": "I'm going to change it to `table` in the output AND the input.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1435294468, "label": "`/db/-/create` API for creating tables"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1882#issuecomment-1312581008", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1882", "id": 1312581008, "node_id": "IC_kwDOBm6k_c5OPGWQ", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-12T22:00:52Z", "updated_at": "2022-11-12T22:00:52Z", "author_association": "OWNER", "body": "Tried out my prototype in the API explorer:\r\n\r\n<img width=\"1229\" alt=\"image\" src=\"https://user-images.githubusercontent.com/9599/201496194-a6ebcefe-2be8-47d7-8f15-8c6345cba7cf.png\">\r\n\r\nThe `\"name\"` on the output is bothering me a bit - should it be `table_name` or `table` instead?\r\n\r\nProblem is I really like `name` for the input, so should it be consistent to have the same name on the output here, or should I aim for consistency with other endpoints that might return `table` rather than the ambiguous `name` elsewhere?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1435294468, "label": "`/db/-/create` API for creating tables"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1882#issuecomment-1312580348", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1882", "id": 1312580348, "node_id": "IC_kwDOBm6k_c5OPGL8", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-12T21:55:54Z", "updated_at": "2022-11-12T21:56:45Z", "author_association": "OWNER", "body": "What should this API return?\r\n\r\nI think the name of the table (`name`), the URL to that table (`table_url` - for consistency with how faceting API works already) and the schema of the table (`schema`).", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1435294468, "label": "`/db/-/create` API for creating tables"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1882#issuecomment-1312575048", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1882", "id": 1312575048, "node_id": "IC_kwDOBm6k_c5OPE5I", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-12T21:22:58Z", "updated_at": "2022-11-12T21:22:58Z", "author_association": "OWNER", "body": "Need to validate the table name. SQLite supports almost any table name - but they can't contain a newline character and cannot start with `sqlite_` - according to https://stackoverflow.com/questions/3694276/what-are-valid-table-names-in-sqlite/43049720#43049720", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1435294468, "label": "`/db/-/create` API for creating tables"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1882#issuecomment-1312556044", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1882", "id": 1312556044, "node_id": "IC_kwDOBm6k_c5OPAQM", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-12T19:29:11Z", "updated_at": "2022-11-12T19:29:11Z", "author_association": "OWNER", "body": "Thought of an edge-case: with `sqlite-utils` one feature I really like is that I can pipe data into it without caring if the table already exists or not:\r\n\r\n    cat data.json | sqlite-utils insert my.db mytable -\r\n\r\nHow could this new API support that?\r\n\r\nI thought about adding a `\"create\": true` option to `/db/table/-/insert` which creates the table if it doesn't already exist, but if I do that I'll need to start adding other options to that endpoint - to set the primary key, add foreign keys and suchlike - which would be ignored except for the cases where the table was being created from scratch.\r\n\r\nThis doesn't feel right to me - I want to keep those options here, on `/db/-/create`.\r\n\r\nOne idea I had was to implement it such that you can call `/db/-/create` multiple times for the same table, but only if you are using the `\"rows\"` option. If so, and if the rows can be safely inserted, it would let you do that.\r\n\r\nBut instead, I'm going to outsource this to the CLI tool I plan to write that feeds data into this API. I'm already planning to use that tool for CSV inserts (so the API doesn't need to accept CSV directly). I think it's a good place for other usability enhancements like \"insert this, creating the table if it does not exist\" as well.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1435294468, "label": "`/db/-/create` API for creating tables"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1883#issuecomment-1311314981", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1883", "id": 1311314981, "node_id": "IC_kwDOBm6k_c5OKRQl", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-11T07:15:48Z", "updated_at": "2022-11-11T07:15:48Z", "author_association": "OWNER", "body": "I released that fix in Datasette 0.63.1: https://docs.datasette.io/en/stable/changelog.html#v0-63-1", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1435917503, "label": "Errors when using table filters behind a proxy"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1883#issuecomment-1311299535", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1883", "id": 1311299535, "node_id": "IC_kwDOBm6k_c5OKNfP", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-11T06:54:58Z", "updated_at": "2022-11-11T06:54:58Z", "author_association": "OWNER", "body": "This time deployed with:\r\n```\r\ncd demos/apache-proxy\r\nfly deploy --build-arg DATASETTE_REF=8d9a957c6329d26cc1e417b5d6911640d74765eb\r\n```\r\nTo ensure the exact commit with the fix.\r\n\r\nAnd that fixed it!\r\n```\r\n% curl -i 'https://datasette-apache-proxy-demo.datasette.io/prefix/fixtures/binary_data?_filter_column=rowid&_filter_op=exact&_filter_value=1&_sort=rowid'\r\nHTTP/2 302 \r\ndate: Fri, 11 Nov 2022 06:54:45 GMT\r\nserver: Fly/b1863e2e7 (2022-11-09)\r\nlocation: /prefix/fixtures/binary_data?_sort=rowid&rowid__exact=1\r\nlink: </prefix/fixtures/binary_data?_sort=rowid&rowid__exact=1>; rel=preload\r\ncontent-type: text/plain\r\nx-proxied-by: Apache2 Debian\r\nvia: 2 fly.io\r\nfly-request-id: 01GHJQGBSXBR7E53TY0EKMQ9PA-sjc\r\n```\r\n\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1435917503, "label": "Errors when using table filters behind a proxy"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1883#issuecomment-1311292463", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1883", "id": 1311292463, "node_id": "IC_kwDOBm6k_c5OKLwv", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-11T06:44:24Z", "updated_at": "2022-11-11T06:44:24Z", "author_association": "OWNER", "body": "Modifying that test to the following does indeed cause a failure:\r\n```python\r\ndef test_base_url_affects_filter_redirects(app_client_base_url_prefix):\r\n    response = app_client_base_url_prefix.get(\r\n        \"/fixtures/binary_data?_filter_column=rowid&_filter_op=exact&_filter_value=1&_sort=rowid\"\r\n    )\r\n    assert response.status == 302\r\n    assert (\r\n        response.headers[\"location\"]\r\n        == \"/prefix/fixtures/binary_data?_sort=rowid&rowid__exact=1\"\r\n    )\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1435917503, "label": "Errors when using table filters behind a proxy"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1883#issuecomment-1311291632", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1883", "id": 1311291632, "node_id": "IC_kwDOBm6k_c5OKLjw", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-11T06:43:00Z", "updated_at": "2022-11-11T06:43:00Z", "author_association": "OWNER", "body": "https://datasette-apache-proxy-demo.datasette.io/prefix/-/asgi-scope is useful:\r\n\r\nIt confirms that `/prefix/` is nowhere to be seen in the incoming request data:\r\n\r\n```\r\n 'path': '/-/asgi-scope',\r\n 'query_string': b'',\r\n 'raw_path': b'/-/asgi-scope',\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1435917503, "label": "Errors when using table filters behind a proxy"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1883#issuecomment-1311290115", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1883", "id": 1311290115, "node_id": "IC_kwDOBm6k_c5OKLMD", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-11T06:40:14Z", "updated_at": "2022-11-11T06:41:56Z", "author_association": "OWNER", "body": "I modified that config file to have this line instead:\r\n```\r\n    ProxyPass /prefix/ http://127.0.0.1:8001/ nocanon\r\n```\r\nAnd then deployed it by running:\r\n\r\n    flyctl deploy --build-arg DATASETTE_REF=main\r\n\r\nThis does NOT seem to have fixed the bug:\r\n\r\n```\r\n~ % curl -i 'https://datasette-apache-proxy-demo.datasette.io/prefix/fixtures/binary_data?_filter_column=rowid&_filter_op=exact&_filter_value=1&_sort=rowid'\r\nHTTP/2 302 \r\ndate: Fri, 11 Nov 2022 06:40:01 GMT\r\nserver: Fly/b1863e2e7 (2022-11-09)\r\nlocation: /fixtures/binary_data?_sort=rowid&rowid__exact=1\r\nlink: </fixtures/binary_data?_sort=rowid&rowid__exact=1>; rel=preload\r\ncontent-type: text/plain\r\nx-proxied-by: Apache2 Debian\r\nvia: 2 fly.io\r\nfly-request-id: 01GHJPNCF51CJ626EWZEHK2CH9-sjc\r\n```\r\n\r\nhttps://datasette-apache-proxy-demo.datasette.io/prefix/-/versions seems to confirm that this is the latest deployed version (0.63), so it looks like the deploy worked.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1435917503, "label": "Errors when using table filters behind a proxy"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1883#issuecomment-1311286593", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1883", "id": 1311286593, "node_id": "IC_kwDOBm6k_c5OKKVB", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-11T06:34:09Z", "updated_at": "2022-11-11T06:34:09Z", "author_association": "OWNER", "body": "https://httpd.apache.org/docs/2.4/mod/mod_proxy.html#proxypass includes this note:\r\n\r\n> Normally, mod_proxy will canonicalise ProxyPassed URLs. But this may be incompatible with some backends, particularly those that make use of *PATH_INFO*. The optional *nocanon* keyword suppresses this and passes the URL path \"raw\" to the backend.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1435917503, "label": "Errors when using table filters behind a proxy"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1883#issuecomment-1311284537", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1883", "id": 1311284537, "node_id": "IC_kwDOBm6k_c5OKJ05", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-11T06:30:38Z", "updated_at": "2022-11-11T06:30:38Z", "author_association": "OWNER", "body": "Is there a chance that it's Apache that's messing with that `location:` header here, not Datasette?\r\n\r\nhttps://github.com/simonw/datasette/blob/bbaab3b38ec2ce5944239ffbe2dd53328df40fff/demos/apache-proxy/000-default.conf#L7-L13", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1435917503, "label": "Errors when using table filters behind a proxy"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1883#issuecomment-1311283301", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1883", "id": 1311283301, "node_id": "IC_kwDOBm6k_c5OKJhl", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-11T06:28:38Z", "updated_at": "2022-11-11T06:29:33Z", "author_association": "OWNER", "body": "`path_with_added_args(request, redirect_params)` should be preserving the current path from the request.\r\n\r\nhttps://github.com/simonw/datasette/blob/bbaab3b38ec2ce5944239ffbe2dd53328df40fff/datasette/utils/__init__.py#L273-L286", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1435917503, "label": "Errors when using table filters behind a proxy"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1883#issuecomment-1311282970", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1883", "id": 1311282970, "node_id": "IC_kwDOBm6k_c5OKJca", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-11T06:28:05Z", "updated_at": "2022-11-11T06:28:05Z", "author_association": "OWNER", "body": "Relevant code: https://github.com/simonw/datasette/blob/bbaab3b38ec2ce5944239ffbe2dd53328df40fff/datasette/views/table.py#L227-L249", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1435917503, "label": "Errors when using table filters behind a proxy"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1883#issuecomment-1311280709", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1883", "id": 1311280709, "node_id": "IC_kwDOBm6k_c5OKI5F", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-11T06:25:27Z", "updated_at": "2022-11-11T06:25:27Z", "author_association": "OWNER", "body": "I tried adding this test but it passed! I expected it to fail:\r\n\r\n```python\r\ndef test_base_url_affects_filter_redirects(app_client_base_url_prefix):\r\n    response = app_client_base_url_prefix.get(\r\n        \"/prefix/fixtures/binary_data?_filter_column=rowid&_filter_op=exact&_filter_value=1&_sort=rowid\"\r\n    )\r\n    assert response.status == 302\r\n    assert (\r\n        response.headers[\"location\"]\r\n        == \"/prefix/fixtures/binary_data?_sort=rowid&rowid__exact=1\"\r\n    )\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1435917503, "label": "Errors when using table filters behind a proxy"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1883#issuecomment-1311278678", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1883", "id": 1311278678, "node_id": "IC_kwDOBm6k_c5OKIZW", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-11T06:22:37Z", "updated_at": "2022-11-11T06:22:37Z", "author_association": "OWNER", "body": "If you view source on that page the HTML looks correct:\r\n```html\r\n<form class=\"filters\" action=\"/prefix/fixtures/binary_data\" method=\"get\">\r\n```\r\n(I just added a test that confirms this too.)\r\n\r\nBut... it looks like the bug is in the redirection code. \r\n\r\nhttps://datasette-apache-proxy-demo.datasette.io/prefix/fixtures/binary_data?_filter_column=rowid&_filter_op=exact&_filter_value=1&_sort=rowid returns the following:\r\n\r\n    location: /fixtures/binary_data?_sort=rowid&rowid__exact=1\r\n\r\nWhich is incorrect.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1435917503, "label": "Errors when using table filters behind a proxy"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1883#issuecomment-1311273461", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1883", "id": 1311273461, "node_id": "IC_kwDOBm6k_c5OKHH1", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-11T06:16:08Z", "updated_at": "2022-11-11T06:16:08Z", "author_association": "OWNER", "body": "Great catch, thanks!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1435917503, "label": "Errors when using table filters behind a proxy"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1880#issuecomment-1311273063", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1880", "id": 1311273063, "node_id": "IC_kwDOBm6k_c5OKHBn", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-11T06:15:28Z", "updated_at": "2022-11-11T06:15:28Z", "author_association": "OWNER", "body": "The `_internal` database is intended to help Datasette handle much larger attached databases. Right now Datasette attempts to show every database on the https://latest.datasette.io/ index page and every table on the https://latest.datasette.io/fixtures database index page - but these are not paginated. If you had a database containing 1,000 tables the database index page would get pretty slow.\r\n\r\nSo I want to be able to paginate (and search) those. But to paginate them it's useful to have them in a database table itself, since then I can paginate using SQL.\r\n\r\nMy plan for `_internal` is to use it to implement those advanced browsing features. I've not completed this work yet though. See this issue for more details on that:\r\n\r\n- #417", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1433576351, "label": "Datasette with many and large databases > Memory use"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1880#issuecomment-1311271298", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1880", "id": 1311271298, "node_id": "IC_kwDOBm6k_c5OKGmC", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-11T06:12:29Z", "updated_at": "2022-11-11T06:12:29Z", "author_association": "OWNER", "body": "I think you may have misunderstood this feature. This is talking about the `_internal` in-memory database, which maintains a set of tables that list the databases and tables that are attached to Datasette.\r\n\r\nThey're not a copy of the data itself - just a list of table names, column names and database names.\r\n\r\nYou can see what that database looks like by signing in as root - running `datasette --root` and clicking the link. Or you can see an example here:\r\n\r\n- Click the button on https://latest.datasette.io/login-as-root\r\n- Now visit https://latest.datasette.io/_internal\r\n\r\nFor the example instance that looks like this:\r\n\r\n<img width=\"697\" alt=\"image\" src=\"https://user-images.githubusercontent.com/9599/201275230-b6375574-17f0-4cd8-b363-0c69a5907080.png\">\r\n\r\nThe two most interesting tables in there are these ones:\r\n\r\n<img width=\"1703\" alt=\"image\" src=\"https://user-images.githubusercontent.com/9599/201275343-589eb6d7-6a9f-403c-b84a-07455d936a85.png\">\r\n\r\n<img width=\"1670\" alt=\"CleanShot 2022-11-10 at 22 11 23@2x\" src=\"https://user-images.githubusercontent.com/9599/201275287-04e6c57d-3ef9-4987-9a8b-5d421a48c452.png\">\r\n\r\nAs you can see, it's just the table schema itself and the columns that make up the tables. Even if you have hundreds of databases connected each with hundreds of tables this should still only add up to a few MB of RAM.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1433576351, "label": "Datasette with many and large databases > Memory use"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1884#issuecomment-1311269045", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1884", "id": 1311269045, "node_id": "IC_kwDOBm6k_c5OKGC1", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-11T06:08:28Z", "updated_at": "2022-11-11T06:08:28Z", "author_association": "OWNER", "body": "Does that work if you add `--load-extension spatialite`?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1439009231, "label": "Exclude virtual tables from datasette inspect"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1882#issuecomment-1302818784", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1882", "id": 1302818784, "node_id": "IC_kwDOBm6k_c5Np2_g", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-04T00:25:18Z", "updated_at": "2022-11-04T16:12:39Z", "author_association": "OWNER", "body": "On that basis I think the core API design should change to this:\r\n```\r\nPOST /db/-/create\r\nAuthorization: Bearer xxx\r\nContent-Type: application/json\r\n{\r\n    \"name\": \"my new table\",\r\n    \"columns\": [\r\n        {\r\n            \"name\": \"id\",\r\n            \"type\": \"integer\"\r\n        },\r\n        {\r\n            \"name\": \"title\",\r\n            \"type\": \"text\"\r\n        }\r\n    ]\r\n   \"pk\": \"id\"\r\n}\r\n```\r\nThis leaves room for a `\"rows\": []` key at the root too. Having that as a child of `\"table\"` felt unintuitive to me, and I didn't like the way this looked either:\r\n\r\n```json\r\n{\r\n  \"table\": {\r\n    \"name\": \"my_new_table\"\r\n  },\r\n  \"rows\": [\r\n    {\"id\": 1, \"title\": \"Title\"}\r\n  ]\r\n}\r\n```\r\nWeird to have the table `name` nested inside `table` when `rows` wasn't.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1435294468, "label": "`/db/-/create` API for creating tables"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1882#issuecomment-1302818153", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1882", "id": 1302818153, "node_id": "IC_kwDOBm6k_c5Np21p", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-04T00:23:58Z", "updated_at": "2022-11-04T00:23:58Z", "author_association": "OWNER", "body": "I made a decision here that this endpoint should also accept an optional `\"rows\": [...]` list which is used to automatically create the table using a schema derived from those example rows (which then get inserted):\r\n\r\n- https://github.com/simonw/datasette/issues/1862#issuecomment-1302817807", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1435294468, "label": "`/db/-/create` API for creating tables"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1862#issuecomment-1302817807", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1862", "id": 1302817807, "node_id": "IC_kwDOBm6k_c5Np2wP", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-04T00:23:13Z", "updated_at": "2022-11-04T00:23:13Z", "author_association": "OWNER", "body": "I don't like this on `/db/table/-/insert` - I think it makes more sense to optionally pass a `\"rows\"` key to the `/db/-/create` endpoint instead.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1425011030, "label": "Create a new table from one or more records, `sqlite-utils` style"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1862#issuecomment-1302817500", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1862", "id": 1302817500, "node_id": "IC_kwDOBm6k_c5Np2rc", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-04T00:22:31Z", "updated_at": "2022-11-04T00:22:31Z", "author_association": "OWNER", "body": "Maybe this is a feature added to the existing `/db/table/-/insert` endpoint?\r\n\r\nBit weird that you can call that endpoint for a table that doesn't exist yet, but it fits the `sqlite-utils` way of creating tables which I've found very pleasant over the past few years.\r\n\r\nSo perhaps the API looks like this:\r\n\r\n```\r\nPOST /<database>/<table>/-/insert\r\nContent-Type: application/json\r\nAuthorization: Bearer dstok_<rest-of-token>\r\n{\r\n    \"create_table\": true,\r\n    \"rows\": [\r\n        {\r\n            \"column1\": \"value1\",\r\n            \"column2\": \"value2\"\r\n        },\r\n        {\r\n            \"column1\": \"value3\",\r\n            \"column2\": \"value4\"\r\n        }\r\n    ]\r\n}\r\n```\r\nThe `create_table` option will cause the table to be created if it doesn't already exist.\r\n\r\nThat means I probably also need a `\"pk\": \"...\"` column for setting a primary key if the table is being created ... and maybe other options that I invent for this other feature too?\r\n- #1882", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1425011030, "label": "Create a new table from one or more records, `sqlite-utils` style"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1871#issuecomment-1302815105", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1871", "id": 1302815105, "node_id": "IC_kwDOBm6k_c5Np2GB", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-04T00:17:23Z", "updated_at": "2022-11-04T00:17:23Z", "author_association": "OWNER", "body": "I'll probably enhance it a bit more though, I want to provide a UI that lists all the tables you can explore and lets you click to pre-fill the forms with them.\r\n\r\nThough at that point what should I do about the other endpoints? Probably list those too. Gets a bit complex, especially with the row-level update and delete endpoints.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1427293909, "label": "API explorer tool"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1871#issuecomment-1302814693", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1871", "id": 1302814693, "node_id": "IC_kwDOBm6k_c5Np1_l", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-04T00:16:36Z", "updated_at": "2022-11-04T00:16:36Z", "author_association": "OWNER", "body": "I can close this issue once I fix it so it no longer hard-codes a potentially invalid example endpoint:\r\n\r\nhttps://github.com/simonw/datasette/blob/bcc781f4c50a8870e3389c4e60acb625c34b0317/datasette/templates/api_explorer.html#L24-L26\r\n\r\nhttps://github.com/simonw/datasette/blob/bcc781f4c50a8870e3389c4e60acb625c34b0317/datasette/templates/api_explorer.html#L34-L35", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1427293909, "label": "API explorer tool"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1881#issuecomment-1302813449", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1881", "id": 1302813449, "node_id": "IC_kwDOBm6k_c5Np1sJ", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-04T00:14:07Z", "updated_at": "2022-11-04T00:14:07Z", "author_association": "OWNER", "body": "Tool is now live here: https://latest-1-0-dev.datasette.io/-/permissions\r\n\r\nNeeds root perms, so access this first: https://latest-1-0-dev.datasette.io/login-as-root", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1434094365, "label": "Tool for simulating permission checks against actors"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1881#issuecomment-1302812918", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1881", "id": 1302812918, "node_id": "IC_kwDOBm6k_c5Np1j2", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-04T00:13:05Z", "updated_at": "2022-11-04T00:13:05Z", "author_association": "OWNER", "body": "Has tests now.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1434094365, "label": "Tool for simulating permission checks against actors"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1863#issuecomment-1302790013", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1863", "id": 1302790013, "node_id": "IC_kwDOBm6k_c5Npv99", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-03T23:32:30Z", "updated_at": "2022-11-03T23:32:30Z", "author_association": "OWNER", "body": "I'm not going to allow updates to primary keys. If you need to do that, you can instead delete the record and then insert a new one with the new primary keys you wanted - or maybe use a custom SQL query.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1425029242, "label": "Update a single record in an existing table"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1851#issuecomment-1294224185", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1851", "id": 1294224185, "node_id": "IC_kwDOBm6k_c5NJEs5", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-10-27T23:18:24Z", "updated_at": "2022-11-03T23:26:05Z", "author_association": "OWNER", "body": "So new API design is:\r\n\r\n```\r\nPOST /db/table/-/insert\r\nAuthorization: Bearer xxx\r\nContent-Type: application/json\r\n{\r\n  \"row\": {\r\n    \"id\": 1,\r\n    \"name\": \"New record\"\r\n  }\r\n}\r\n```\r\nReturns:\r\n```\r\n201 Created\r\n{\r\n  \"row\": [{\r\n    \"id\": 1,\r\n    \"name\": \"New record\"\r\n  }]\r\n}\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1421544654, "label": "API to insert a single record into an existing table"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1863#issuecomment-1302785086", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1863", "id": 1302785086, "node_id": "IC_kwDOBm6k_c5Npuw-", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-03T23:24:33Z", "updated_at": "2022-11-03T23:24:56Z", "author_association": "OWNER", "body": "Thinking more about validation: I'm considering if this should validate that columns which are defined as SQLite foreign keys are being updated to values that exist in those other tables.\r\n\r\nI like the sound of this. It seems like a sensible default behaviour for Datasette. And it fits with the fact that Datasette treats foreign keys specially elsewhere in the interface.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1425029242, "label": "Update a single record in an existing table"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1863#issuecomment-1302760549", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1863", "id": 1302760549, "node_id": "IC_kwDOBm6k_c5Npoxl", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-03T22:43:04Z", "updated_at": "2022-11-03T23:21:31Z", "author_association": "OWNER", "body": "The `id=(int, ...)` thing is weird, but is apparently Pydantic syntax for a required field?\r\n\r\nhttps://cs.github.com/starlite-api/starlite/blob/28ddc847c4cb072f0d5d21a9ecd5259711f12ec9/docs/usage/11-data-transfer-objects.md#L161 confirms:\r\n\r\n> 1. For required fields use a tuple of type + ellipsis, for example `(str, ...)`.\r\n> 2. For optional fields use a tuple of type + `None`, for example `(str, None)`\r\n> 3. To set a default value use a tuple of type + default value, for example `(str, \"Hello World\")`", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1425029242, "label": "Update a single record in an existing table"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1863#issuecomment-1302760382", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1863", "id": 1302760382, "node_id": "IC_kwDOBm6k_c5Npou-", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-03T22:42:47Z", "updated_at": "2022-11-03T22:42:47Z", "author_association": "OWNER", "body": "```python\r\nprint(create_model('document', id=(int, ...), title=(str, None)).schema_json(indent=2))\r\n```\r\n```json\r\n{\r\n  \"title\": \"document\",\r\n  \"type\": \"object\",\r\n  \"properties\": {\r\n    \"id\": {\r\n      \"title\": \"Id\",\r\n      \"type\": \"integer\"\r\n    },\r\n    \"title\": {\r\n      \"title\": \"Title\",\r\n      \"type\": \"string\"\r\n    }\r\n  },\r\n  \"required\": [\r\n    \"id\"\r\n  ]\r\n}\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1425029242, "label": "Update a single record in an existing table"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1863#issuecomment-1302759174", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1863", "id": 1302759174, "node_id": "IC_kwDOBm6k_c5NpocG", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-03T22:40:47Z", "updated_at": "2022-11-03T22:40:47Z", "author_association": "OWNER", "body": "I'm considering Pydantic for this, see:\r\n- https://github.com/simonw/datasette/issues/1882#issuecomment-1302716350\r\n\r\nIn particular the `create_model()` method: https://pydantic-docs.helpmanual.io/usage/models/#dynamic-model-creation\r\n\r\nThis would give me good validation. It would also, weirdly, give me the ability to output JSON schema. Maybe I could have this as the JSON schema for a row?\r\n\r\n`/db/table/-/json-schema`", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1425029242, "label": "Update a single record in an existing table"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1882#issuecomment-1302716350", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1882", "id": 1302716350, "node_id": "IC_kwDOBm6k_c5Npd--", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-03T21:51:14Z", "updated_at": "2022-11-03T22:35:54Z", "author_association": "OWNER", "body": "Validating this JSON object is getting a tiny bit complex. I'm tempted to adopt https://pydantic-docs.helpmanual.io/ at this point.\r\n\r\nThe `create_model` example on https://stackoverflow.com/questions/66168517/generate-dynamic-model-using-pydantic/66168682#66168682 is particularly relevant, especially when I work on this issue:\r\n\r\n- #1863\r\n\r\n```python\r\nfrom pydantic import create_model\r\n\r\nd = {\"strategy\": {\"name\": \"test_strat2\", \"periods\": 10}}\r\n\r\nStrategy = create_model(\"Strategy\", **d[\"strategy\"])\r\n\r\nprint(Strategy.schema_json(indent=2))\r\n```\r\n`create_model()`: https://pydantic-docs.helpmanual.io/usage/models/#dynamic-model-creation", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1435294468, "label": "`/db/-/create` API for creating tables"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1882#issuecomment-1302721916", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1882", "id": 1302721916, "node_id": "IC_kwDOBm6k_c5NpfV8", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-03T21:58:50Z", "updated_at": "2022-11-03T21:59:17Z", "author_association": "OWNER", "body": "Mocked up a quick HTML+JavaScript form for creating that JSON structure using some iteration against Copilot prompts:\r\n```html\r\n<pre>\r\n/* JSON format:\r\n{\r\n  \"table\": {\r\n      \"name\": \"my new table\",\r\n      \"columns\": [\r\n          {\r\n              \"name\": \"id\",\r\n              \"type\": \"integer\"\r\n          },\r\n          {\r\n              \"name\": \"title\",\r\n              \"type\": \"text\"\r\n          }\r\n      ]\r\n     \"pk\": \"id\"\r\n  }\r\n}\r\n\r\nHTML form with Javascript for creating this JSON:\r\n*/</pre>\r\n<form id=\"create-table-form\">\r\n  <label for=\"table-name\">Table name</label>\r\n  <input type=\"text\" id=\"table-name\" name=\"table-name\" required><br>\r\n  <label for=\"table-pk\">Primary key</label>\r\n  <input type=\"text\" id=\"table-pk\" name=\"table-pk\" required><br>\r\n  <label for=\"column-name\">Column name</label>\r\n  <input type=\"text\" id=\"column-name\" name=\"column-name\" required>\r\n  <label for=\"column-type\">Column type</label>\r\n  <input type=\"text\" id=\"column-type\" name=\"column-type\" required>\r\n  <button type=\"button\" id=\"add-column\">Add column</button>\r\n  <p>Current columns:</p>\r\n  <ul id=\"columns\"></ul>\r\n  <button type=\"button\" id=\"create-table\">Create table</button>\r\n</form>\r\n<script>\r\nvar form = document.getElementById('create-table-form');\r\nvar tableName = document.getElementById('table-name');\r\nvar tablePk = document.getElementById('table-pk');\r\nvar columnName = document.getElementById('column-name');\r\nvar columnType = document.getElementById('column-type');\r\nvar addColumn = document.getElementById('add-column');\r\nvar createTable = document.getElementById('create-table');\r\nvar columns = [];\r\naddColumn.addEventListener('click', () => {\r\n  columns.push({\r\n    name: columnName.value,\r\n    type: columnType.value\r\n  });\r\n  var li = document.createElement('li');\r\n  li.textContent = columnName.value + ' (' + columnType.value + ')';\r\n  // Add a delete button to each column\r\n  var deleteButton = document.createElement('button');\r\n  deleteButton.textContent = 'Delete';\r\n  deleteButton.addEventListener('click', () => {\r\n    columns.splice(columns.indexOf(li), 1);\r\n    li.remove();\r\n  });\r\n  li.appendChild(deleteButton);\r\n  document.getElementById('columns').appendChild(li);\r\n});\r\ncreateTable.addEventListener('click', () => {\r\n  var table = {\r\n    name: tableName.value,\r\n    pk: tablePk.value,\r\n    columns: columns\r\n  };\r\n  console.log(JSON.stringify(table, null, 2));\r\n});\r\n</script>\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1435294468, "label": "`/db/-/create` API for creating tables"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1882#issuecomment-1302715662", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1882", "id": 1302715662, "node_id": "IC_kwDOBm6k_c5Npd0O", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-03T21:50:27Z", "updated_at": "2022-11-03T21:50:27Z", "author_association": "OWNER", "body": "API design for this:\r\n```\r\nPOST /db/-/create\r\nAuthorization: Bearer xxx\r\nContent-Type: application/json\r\n{\r\n    \"table\": {\r\n        \"name\": \"my new table\",\r\n        \"columns\": [\r\n            {\r\n                \"name\": \"id\",\r\n                \"type\": \"integer\"\r\n            },\r\n            {\r\n                \"name\": \"title\",\r\n                \"type\": \"text\"\r\n            }\r\n        ]\r\n       \"pk\": \"id\"\r\n    }\r\n}\r\n```\r\nSupported column types are:\r\n\r\n- `integer`\r\n- `text`\r\n- `float` (even though SQLite calls it a \"real\")\r\n- `blob`\r\n\r\nThis matches my design for `sqlite-utils`: https://sqlite-utils.datasette.io/en/stable/cli.html#cli-create-table", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1435294468, "label": "`/db/-/create` API for creating tables"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1843#issuecomment-1302679026", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1843", "id": 1302679026, "node_id": "IC_kwDOBm6k_c5NpU3y", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-03T21:22:42Z", "updated_at": "2022-11-03T21:22:42Z", "author_association": "OWNER", "body": "Docs for the new `db.close()` method: https://docs.datasette.io/en/latest/internals.html#db-close", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1408757705, "label": "Intermittent \"Too many open files\" error running tests"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1843#issuecomment-1302678384", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1843", "id": 1302678384, "node_id": "IC_kwDOBm6k_c5NpUtw", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-03T21:21:59Z", "updated_at": "2022-11-03T21:21:59Z", "author_association": "OWNER", "body": "I added extra debug info to `/-/threads` to see this for myself:\r\n\r\n```diff\r\ndiff --git a/datasette/app.py b/datasette/app.py\r\nindex 02bd38f1..16579e28 100644\r\n--- a/datasette/app.py\r\n+++ b/datasette/app.py\r\n@@ -969,6 +969,13 @@ class Datasette:\r\n             \"threads\": [\r\n                 {\"name\": t.name, \"ident\": t.ident, \"daemon\": t.daemon} for t in threads\r\n             ],\r\n+            \"file_connections\": {\r\n+                db.name: [\r\n+                    [dict(r) for r in conn.execute(\"pragma database_list\").fetchall()]\r\n+                    for conn in db._all_file_connections\r\n+                ]\r\n+                for db in self.databases.values()\r\n+            },\r\n         }\r\n         # Only available in Python 3.7+\r\n         if hasattr(asyncio, \"all_tasks\"):\r\n```\r\nOutput after hitting refresh on a few `/fixtures` tables to ensure more threads started:\r\n\r\n```\r\n    \"file_connections\": {\r\n        \"_internal\": [],\r\n        \"fixtures\": [\r\n            [\r\n                {\r\n                    \"seq\": 0,\r\n                    \"name\": \"main\",\r\n                    \"file\": \"/Users/simon/Dropbox/Development/datasette/fixtures.db\"\r\n                }\r\n            ],\r\n            [\r\n                {\r\n                    \"seq\": 0,\r\n                    \"name\": \"main\",\r\n                    \"file\": \"/Users/simon/Dropbox/Development/datasette/fixtures.db\"\r\n                }\r\n            ],\r\n            [\r\n                {\r\n                    \"seq\": 0,\r\n                    \"name\": \"main\",\r\n                    \"file\": \"/Users/simon/Dropbox/Development/datasette/fixtures.db\"\r\n                }\r\n            ]\r\n        ]\r\n    },\r\n```\r\nI decided not to ship this feature though as it leaks the names of internal database files.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1408757705, "label": "Intermittent \"Too many open files\" error running tests"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1843#issuecomment-1302634332", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1843", "id": 1302634332, "node_id": "IC_kwDOBm6k_c5NpJ9c", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-03T20:34:56Z", "updated_at": "2022-11-03T20:34:56Z", "author_association": "OWNER", "body": "Confirmed that calling `conn.close()` on each SQLite file-based connection is the way to fix this problem.\r\n\r\nI'm adding a `db.close()` method (sync, not async - I tried async first but it was really hard to cause every thread in the pool to close its threadlocal database connection) which loops through all known open file-based connections and closes them.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1408757705, "label": "Intermittent \"Too many open files\" error running tests"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1843#issuecomment-1302574330", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1843", "id": 1302574330, "node_id": "IC_kwDOBm6k_c5No7T6", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-03T19:30:22Z", "updated_at": "2022-11-03T19:30:22Z", "author_association": "OWNER", "body": "This is affecting me a lot at the moment, on my laptop (runs fine in CI).\r\n\r\nHere's a change to `conftest.py` which highlights the problem - it cause a failure the moment there are more than 5 open files according to `psutil`:\r\n\r\n```diff\r\ndiff --git a/tests/conftest.py b/tests/conftest.py\r\nindex f4638a14..21d433c1 100644\r\n--- a/tests/conftest.py\r\n+++ b/tests/conftest.py\r\n@@ -1,6 +1,7 @@\r\n import httpx\r\n import os\r\n import pathlib\r\n+import psutil\r\n import pytest\r\n import re\r\n import subprocess\r\n@@ -192,3 +193,8 @@ def ds_unix_domain_socket_server(tmp_path_factory):\r\n     yield ds_proc, uds\r\n     # Shut it down at the end of the pytest session\r\n     ds_proc.terminate()\r\n+\r\n+\r\n+def pytest_runtest_teardown(item: pytest.Item) -> None:\r\n+    open_files = psutil.Process().open_files()\r\n+    assert len(open_files) < 5\r\n```\r\nThe first error I get from this with `pytest --pdb -x` is here:\r\n\r\n```\r\ntests/test_api.py ............E\r\n>>>>> traceback >>>>>\r\n\r\nitem = <Function test_sql_time_limit>\r\n\r\n    def pytest_runtest_teardown(item: pytest.Item) -> None:\r\n        open_files = psutil.Process().open_files()\r\n>       assert len(open_files) < 5\r\nE       AssertionError: assert 5 < 5\r\nE        +  where 5 = len([popenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmpfglrt4p2/fixtures.db', fd=14), popenfile(... fd=19), popenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmphdi5b250/fixtures.dot.db', fd=20)])\r\n\r\n/Users/simon/Dropbox/Development/datasette/tests/conftest.py:200: AssertionError\r\n>>>>> entering PDB >>>>>\r\n\r\n>>>>> PDB post_mortem (IO-capturing turned off) >>>>>\r\n> /Users/simon/Dropbox/Development/datasette/tests/conftest.py(200)pytest_runtest_teardown()\r\n-> assert len(open_files) < 5\r\n```\r\nThat's this test:\r\n\r\nhttps://github.com/simonw/datasette/blob/2ec5583629005b32cb0877786f9681c5d43ca33f/tests/test_api.py#L656-L673\r\n\r\nWhich uses this fixture:\r\n\r\nhttps://github.com/simonw/datasette/blob/2ec5583629005b32cb0877786f9681c5d43ca33f/tests/fixtures.py#L228-L231\r\n\r\nWhich calls this function:\r\n\r\nhttps://github.com/simonw/datasette/blob/2ec5583629005b32cb0877786f9681c5d43ca33f/tests/fixtures.py#L105-L122\r\n\r\nSo now I'm suspicious that, even though the fixture is meant to be session scoped, the way I'm using `with tempfile.TemporaryDirectory() as tmpdir:` is causing a whole load of files to be created and held open which are not later closed.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1408757705, "label": "Intermittent \"Too many open files\" error running tests"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1855#issuecomment-1301646670", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1855", "id": 1301646670, "node_id": "IC_kwDOBm6k_c5NlY1O", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-03T05:11:26Z", "updated_at": "2022-11-03T05:11:26Z", "author_association": "OWNER", "body": "That still needs comprehensive tests before I land it.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1423336089, "label": "`datasette create-token` ability to create tokens with a reduced set of permissions"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1855#issuecomment-1301646493", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1855", "id": 1301646493, "node_id": "IC_kwDOBm6k_c5NlYyd", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-03T05:11:06Z", "updated_at": "2022-11-03T05:11:06Z", "author_association": "OWNER", "body": "Built a prototype of the above:\r\n\r\n```diff\r\ndiff --git a/datasette/default_permissions.py b/datasette/default_permissions.py\r\nindex 32b0c758..f68aa38f 100644\r\n--- a/datasette/default_permissions.py\r\n+++ b/datasette/default_permissions.py\r\n@@ -6,8 +6,8 @@ import json\r\n import time\r\n \r\n \r\n-@hookimpl(tryfirst=True)\r\n-def permission_allowed(datasette, actor, action, resource):\r\n+@hookimpl(tryfirst=True, specname=\"permission_allowed\")\r\n+def permission_allowed_default(datasette, actor, action, resource):\r\n     async def inner():\r\n         if action in (\r\n             \"permissions-debug\",\r\n@@ -57,6 +57,44 @@ def permission_allowed(datasette, actor, action, resource):\r\n     return inner\r\n \r\n \r\n+@hookimpl(specname=\"permission_allowed\")\r\n+def permission_allowed_actor_restrictions(actor, action, resource):\r\n+    if actor is None:\r\n+        return None\r\n+    _r = actor.get(\"_r\")\r\n+    if not _r:\r\n+        # No restrictions, so we have no opinion\r\n+        return None\r\n+    action_initials = \"\".join([word[0] for word in action.split(\"-\")])\r\n+    # If _r is defined then we use those to further restrict the actor\r\n+    # Crucially, we only use this to say NO (return False) - we never\r\n+    # use it to return YES (True) because that might over-ride other\r\n+    # restrictions placed on this actor\r\n+    all_allowed = _r.get(\"a\")\r\n+    if all_allowed is not None:\r\n+        assert isinstance(all_allowed, list)\r\n+        if action_initials in all_allowed:\r\n+            return None\r\n+    # How about for the current database?\r\n+    if action in (\"view-database\", \"view-database-download\", \"execute-sql\"):\r\n+        database_allowed = _r.get(\"d\", {}).get(resource)\r\n+        if database_allowed is not None:\r\n+            assert isinstance(database_allowed, list)\r\n+            if action_initials in database_allowed:\r\n+                return None\r\n+    # Or the current table? That's any time the resource is (database, table)\r\n+    if not isinstance(resource, str) and len(resource) == 2:\r\n+        database, table = resource\r\n+        table_allowed = _r.get(\"t\", {}).get(database, {}).get(table)\r\n+        # TODO: What should this do for canned queries?\r\n+        if table_allowed is not None:\r\n+            assert isinstance(table_allowed, list)\r\n+            if action_initials in table_allowed:\r\n+                return None\r\n+    # This action is not specifically allowed, so reject it\r\n+    return False\r\n+\r\n+\r\n @hookimpl\r\n def actor_from_request(datasette, request):\r\n     prefix = \"dstok_\"\r\n\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1423336089, "label": "`datasette create-token` ability to create tokens with a reduced set of permissions"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1881#issuecomment-1301639741", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1881", "id": 1301639741, "node_id": "IC_kwDOBm6k_c5NlXI9", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-03T04:58:21Z", "updated_at": "2022-11-03T04:58:21Z", "author_association": "OWNER", "body": "The whole `database_name` or `(database_name, table_name)` tuple for resource is a bit of a code smell. Maybe this is a chance to tidy that up too?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1434094365, "label": "Tool for simulating permission checks against actors"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1881#issuecomment-1301639370", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1881", "id": 1301639370, "node_id": "IC_kwDOBm6k_c5NlXDK", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-03T04:57:21Z", "updated_at": "2022-11-03T04:57:21Z", "author_association": "OWNER", "body": "The plugin hook would be called `register_permissions()`, for consistency with `register_routes()` and `register_commands()`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1434094365, "label": "Tool for simulating permission checks against actors"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1881#issuecomment-1301638918", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1881", "id": 1301638918, "node_id": "IC_kwDOBm6k_c5NlW8G", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-03T04:56:06Z", "updated_at": "2022-11-03T04:56:06Z", "author_association": "OWNER", "body": "I've also introduced a new concept of a permission abbreviation, which like the permission name needs to be globally unique.\r\n\r\nThat's a problem for plugins - they might just be able to guarantee that their permission long-form name is unique among other plugins (through sensible naming conventions) but the thing where they declare a initial-letters-only abbreviation is far more risky.\r\n\r\nI think abbreviations are optional - they are provided for core permissions but plugins are advised not to use them.\r\n\r\nAlso Datasette could check that the installed plugins do not provide conflicting permissions on startup and refuse to start if they do.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1434094365, "label": "Tool for simulating permission checks against actors"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1881#issuecomment-1301638156", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1881", "id": 1301638156, "node_id": "IC_kwDOBm6k_c5NlWwM", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-03T04:54:00Z", "updated_at": "2022-11-03T04:54:00Z", "author_association": "OWNER", "body": "If I have the permissions defined like this:\r\n```python\r\nPERMISSIONS = (\r\n    Permission(\"view-instance\", \"vi\", False, False, True),\r\n    Permission(\"view-database\", \"vd\", True, False, True),\r\n    Permission(\"view-database-download\", \"vdd\", True, False, True),\r\n    Permission(\"view-table\", \"vt\", True, True, True),\r\n    Permission(\"view-query\", \"vq\", True, True, True),\r\n    Permission(\"insert-row\", \"ir\", True, True, False),\r\n    Permission(\"delete-row\", \"dr\", True, True, False),\r\n    Permission(\"drop-table\", \"dt\", True, True, False),\r\n    Permission(\"execute-sql\", \"es\", True, False, True),\r\n    Permission(\"permissions-debug\", \"pd\", False, False, False),\r\n    Permission(\"debug-menu\", \"dm\", False, False, False),\r\n)\r\n```\r\nInstead of just calling them by their undeclared names in places like this:\r\n```python\r\nawait self.ds.permission_allowed(\r\n    request.actor, \"execute-sql\", database, default=True\r\n)\r\n```\r\nOn the one hand I can ditch that confusing `default=True` option - whether a permission is on by default becomes a characteristic of that `Permission()` itself, which feels much neater.\r\n\r\nOn the other hand though, plugins that introduce their own permissions - like https://datasette.io/plugins/datasette-edit-schema - will need a way to register those permissions with Datasette core. Probably another plugin hook.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1434094365, "label": "Tool for simulating permission checks against actors"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1881#issuecomment-1301635906", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1881", "id": 1301635906, "node_id": "IC_kwDOBm6k_c5NlWNC", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-03T04:48:09Z", "updated_at": "2022-11-03T04:48:09Z", "author_association": "OWNER", "body": "I built this prototype on the http://127.0.0.1:8001/-/allow-debug page, which is open to anyone to visit.\r\n\r\nBut... I just realized that using this tool can leak information - you can use it to guess the names of invisible databases and tables and run theoretical permission checks against them.\r\n\r\nUsing the tool also pollutes the list of permission checks that show up on the root-anlo `/-/permissions` page.\r\n\r\nSo.... I'm going to restrict the usage of this tool to users with access to `/-/permissions` and put it on that page instead.\r\n\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1434094365, "label": "Tool for simulating permission checks against actors"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1881#issuecomment-1301635340", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1881", "id": 1301635340, "node_id": "IC_kwDOBm6k_c5NlWEM", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-03T04:46:41Z", "updated_at": "2022-11-03T04:46:41Z", "author_association": "OWNER", "body": "Built this prototype:\r\n\r\n![prototype](https://user-images.githubusercontent.com/9599/199649219-f146e43b-bfb5-45e6-9777-956f21a79887.gif)\r\n\r\nIn building it I realized I needed to know which permissions took a table, a database, both or neither. So I had to bake that into the code.\r\n\r\nHere's the prototype so far (which includes a prototype of the logic for the `_r` field on actor, see #1855):\r\n\r\n```diff\r\ndiff --git a/datasette/default_permissions.py b/datasette/default_permissions.py\r\nindex 32b0c758..f68aa38f 100644\r\n--- a/datasette/default_permissions.py\r\n+++ b/datasette/default_permissions.py\r\n@@ -6,8 +6,8 @@ import json\r\n import time\r\n \r\n \r\n-@hookimpl(tryfirst=True)\r\n-def permission_allowed(datasette, actor, action, resource):\r\n+@hookimpl(tryfirst=True, specname=\"permission_allowed\")\r\n+def permission_allowed_default(datasette, actor, action, resource):\r\n     async def inner():\r\n         if action in (\r\n             \"permissions-debug\",\r\n@@ -57,6 +57,44 @@ def permission_allowed(datasette, actor, action, resource):\r\n     return inner\r\n \r\n \r\n+@hookimpl(specname=\"permission_allowed\")\r\n+def permission_allowed_actor_restrictions(actor, action, resource):\r\n+    if actor is None:\r\n+        return None\r\n+    _r = actor.get(\"_r\")\r\n+    if not _r:\r\n+        # No restrictions, so we have no opinion\r\n+        return None\r\n+    action_initials = \"\".join([word[0] for word in action.split(\"-\")])\r\n+    # If _r is defined then we use those to further restrict the actor\r\n+    # Crucially, we only use this to say NO (return False) - we never\r\n+    # use it to return YES (True) because that might over-ride other\r\n+    # restrictions placed on this actor\r\n+    all_allowed = _r.get(\"a\")\r\n+    if all_allowed is not None:\r\n+        assert isinstance(all_allowed, list)\r\n+        if action_initials in all_allowed:\r\n+            return None\r\n+    # How about for the current database?\r\n+    if action in (\"view-database\", \"view-database-download\", \"execute-sql\"):\r\n+        database_allowed = _r.get(\"d\", {}).get(resource)\r\n+        if database_allowed is not None:\r\n+            assert isinstance(database_allowed, list)\r\n+            if action_initials in database_allowed:\r\n+                return None\r\n+    # Or the current table? That's any time the resource is (database, table)\r\n+    if not isinstance(resource, str) and len(resource) == 2:\r\n+        database, table = resource\r\n+        table_allowed = _r.get(\"t\", {}).get(database, {}).get(table)\r\n+        # TODO: What should this do for canned queries?\r\n+        if table_allowed is not None:\r\n+            assert isinstance(table_allowed, list)\r\n+            if action_initials in table_allowed:\r\n+                return None\r\n+    # This action is not specifically allowed, so reject it\r\n+    return False\r\n+\r\n+\r\n @hookimpl\r\n def actor_from_request(datasette, request):\r\n     prefix = \"dstok_\"\r\ndiff --git a/datasette/templates/allow_debug.html b/datasette/templates/allow_debug.html\r\nindex 0f1b30f0..ae43f0f5 100644\r\n--- a/datasette/templates/allow_debug.html\r\n+++ b/datasette/templates/allow_debug.html\r\n@@ -35,7 +35,7 @@ p.message-warning {\r\n \r\n <p>Use this tool to try out different actor and allow combinations. See <a href=\"https://docs.datasette.io/en/stable/authentication.html#defining-permissions-with-allow-blocks\">Defining permissions with \"allow\" blocks</a> for documentation.</p>\r\n \r\n-<form action=\"{{ urls.path('-/allow-debug') }}\" method=\"get\">\r\n+<form action=\"{{ urls.path('-/allow-debug') }}\" method=\"get\" style=\"margin-bottom: 1em\">\r\n     <div class=\"two-col\">\r\n         <p><label>Allow block</label></p>\r\n         <textarea name=\"allow\">{{ allow_input }}</textarea>\r\n@@ -55,4 +55,82 @@ p.message-warning {\r\n \r\n {% if result == \"False\" %}<p class=\"message-error\">Result: deny</p>{% endif %}\r\n \r\n+<h2>Test permission check</h2>\r\n+\r\n+<p>This tool lets you simulate an actor and a permission check for that actor.</p>\r\n+\r\n+<form action=\"{{ urls.path('-/allow-debug') }}\" id=\"debug-post\" method=\"post\" style=\"margin-bottom: 1em\">\r\n+    <input type=\"hidden\" name=\"csrftoken\" value=\"{{ csrftoken() }}\">\r\n+    <div class=\"two-col\">\r\n+        <p><label>Actor</label></p>\r\n+        <textarea name=\"actor\">{% if actor_input %}{{ actor_input }}{% else %}{\"id\": \"root\"}{% endif %}</textarea>\r\n+    </div>\r\n+    <div class=\"two-col\" style=\"vertical-align: top\">\r\n+        <p><label>Permission check</label></p>\r\n+        <p><label for=\"permission\" style=\"display:block\">Permission</label>\r\n+        <select name=\"permission\" id=\"permission\">\r\n+            {% for permission in [\r\n+                \"view-instance\",\r\n+                \"view-database\",\r\n+                \"view-database-download\",\r\n+                \"view-table\",\r\n+                \"view-query\",\r\n+                \"insert-row\",\r\n+                \"delete-row\",\r\n+                \"drop-table\",\r\n+                \"execute-sql\",\r\n+                \"permissions-debug\",\r\n+                \"debug-menu\"] %}\r\n+            <option value=\"{{ permission }}\">{{ permission }}</option>\r\n+            {% endfor %}\r\n+        </select>\r\n+        <p><label for=\"resource_1\">Database name</label><input type=\"text\" id=\"resource_1\" name=\"resource_1\"></p>\r\n+        <p><label for=\"resource_2\">Table or query name</label><input type=\"text\" id=\"resource_2\" name=\"resource_2\"></p>\r\n+    </div>\r\n+    <div style=\"margin-top: 1em;\">\r\n+        <input type=\"submit\" value=\"Simulate permission check\">\r\n+    </div>\r\n+</form>\r\n+\r\n+<script>\r\n+var rawPerms = {{ permissions|tojson }};\r\n+var permissions = Object.fromEntries(rawPerms.map(([label, abbr, needs_resource_1, needs_resource_2, def]) => [label, {needs_resource_1, needs_resource_2, def}]))\r\n+var permissionSelect = document.getElementById('permission');\r\n+var resource1 = document.getElementById('resource_1');\r\n+var resource2 = document.getElementById('resource_2');\r\n+function updateResourceVisibility() {\r\n+    var permission = permissionSelect.value;\r\n+    var {needs_resource_1, needs_resource_2} = permissions[permission];\r\n+    if (needs_resource_1) {\r\n+        resource1.closest('p').style.display = 'block';\r\n+    } else {\r\n+        resource1.closest('p').style.display = 'none';\r\n+    }\r\n+    if (needs_resource_2) {\r\n+        resource2.closest('p').style.display = 'block';\r\n+    } else {\r\n+        resource2.closest('p').style.display = 'none';\r\n+    }\r\n+}\r\n+permissionSelect.addEventListener('change', updateResourceVisibility);\r\n+updateResourceVisibility();\r\n+\r\n+// When #debug-post form is submitted, use fetch() to POST data\r\n+var debugPost = document.getElementById('debug-post');\r\n+debugPost.addEventListener('submit', function(ev) {\r\n+    ev.preventDefault();\r\n+    var formData = new FormData(debugPost);\r\n+    console.log(formData);\r\n+    fetch(debugPost.action, {\r\n+        method: 'POST',\r\n+        body: new URLSearchParams(formData),\r\n+    }).then(function(response) {\r\n+        return response.json();\r\n+    }).then(function(data) {\r\n+        alert(JSON.stringify(data, null, 4));\r\n+    });\r\n+});\r\n+</script>\r\n+    \r\n+\r\n {% endblock %}\r\ndiff --git a/datasette/views/special.py b/datasette/views/special.py\r\nindex 9922a621..d46fc280 100644\r\n--- a/datasette/views/special.py\r\n+++ b/datasette/views/special.py\r\n@@ -1,6 +1,8 @@\r\n import json\r\n+from datasette.permissions import PERMISSIONS\r\n from datasette.utils.asgi import Response, Forbidden\r\n from datasette.utils import actor_matches_allow, add_cors_headers\r\n+from datasette.permissions import PERMISSIONS\r\n from .base import BaseView\r\n import secrets\r\n import time\r\n@@ -138,9 +140,34 @@ class AllowDebugView(BaseView):\r\n                 \"error\": \"\\n\\n\".join(errors) if errors else \"\",\r\n                 \"actor_input\": actor_input,\r\n                 \"allow_input\": allow_input,\r\n+                \"permissions\": PERMISSIONS,\r\n             },\r\n         )\r\n \r\n+    async def post(self, request):\r\n+        vars = await request.post_vars()\r\n+        actor = json.loads(vars[\"actor\"])\r\n+        permission = vars[\"permission\"]\r\n+        resource_1 = vars[\"resource_1\"]\r\n+        resource_2 = vars[\"resource_2\"]\r\n+        resource = []\r\n+        if resource_1:\r\n+            resource.append(resource_1)\r\n+        if resource_2:\r\n+            resource.append(resource_2)\r\n+        resource = tuple(resource)\r\n+        result = await self.ds.permission_allowed(\r\n+            actor, permission, resource, default=\"USE_DEFAULT\"\r\n+        )\r\n+        return Response.json(\r\n+            {\r\n+                \"actor\": actor,\r\n+                \"permission\": permission,\r\n+                \"resource\": resource,\r\n+                \"result\": result,\r\n+            }\r\n+        )\r\n+\r\n \r\n class MessagesDebugView(BaseView):\r\n     name = \"messages_debug\"\r\n```\r\n\r\n\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1434094365, "label": "Tool for simulating permission checks against actors"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1855#issuecomment-1301594495", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1855", "id": 1301594495, "node_id": "IC_kwDOBm6k_c5NlMF_", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-03T03:11:17Z", "updated_at": "2022-11-03T03:11:17Z", "author_association": "OWNER", "body": "Maybe the way to do this is through a new standard mechanism on the actor: a set of additional restrictions, e.g.:\r\n\r\n```\r\n{\r\n  \"id\": \"root\",\r\n  \"_r\": {\r\n    \"a\": [\"ir\", \"ur\", \"dr\"],\r\n    \"d\": {\r\n        \"fixtures\": [\"ir\", \"ur\", \"dr\"]\r\n    },\r\n    \"t\": {\r\n        \"fixtures\": {\r\n            \"searchable\": [\"ir\"]\r\n        }\r\n    }\r\n}\r\n```\r\n`\"a\"` is \"all permissions\" - these apply to everything.\r\n`\"d\"` permissions only apply to the specified database\r\n`\"t\"` permissions only apply to the specified table\r\n\r\nThe way this works is there's a default [permission_allowed(datasette, actor, action, resource)](https://docs.datasette.io/en/stable/plugin_hooks.html#id25) hook which only consults these, and crucially just says NO if those rules do not match.\r\n\r\nIn this way it would apply as an extra layer of permission rules over the defaults (which for this `root` instance would all return yes).", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1423336089, "label": "`datasette create-token` ability to create tokens with a reduced set of permissions"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1871#issuecomment-1299607082", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1871", "id": 1299607082, "node_id": "IC_kwDOBm6k_c5Ndm4q", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-02T05:45:31Z", "updated_at": "2022-11-02T05:45:31Z", "author_association": "OWNER", "body": "I'm going to add a link to the Datasette API docs for the current running version of Datasette, e.g. to https://docs.datasette.io/en/0.63/json_api.html", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1427293909, "label": "API explorer tool"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1871#issuecomment-1299600257", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1871", "id": 1299600257, "node_id": "IC_kwDOBm6k_c5NdlOB", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-02T05:36:40Z", "updated_at": "2022-11-02T05:36:40Z", "author_association": "OWNER", "body": "The API Explorer should definitely link to the `/-/create-token` page for users who have permission though.\r\n\r\nAnd it should probably go in the Datasette application menu?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1427293909, "label": "API explorer tool"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1871#issuecomment-1299599461", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1871", "id": 1299599461, "node_id": "IC_kwDOBm6k_c5NdlBl", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-02T05:35:36Z", "updated_at": "2022-11-02T05:36:15Z", "author_association": "OWNER", "body": "Here's a slightly wild idea: what if there was a button on `/-/api` that you could click to turn on \"API explorer mode\" for the rest of the Datasette interface - which sets a cookie, and that cookie means you then see \"API explorer\" links in all sorts of other relevant places in the Datasette UI (maybe tucked away in cog menus).\r\n\r\nOnly reason I don't want to show these to everyone is that I don't think this is a very user-friendly feature: if you don't know what an API is I don't want to expose you to it unnecessarily.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1427293909, "label": "API explorer tool"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1871#issuecomment-1299598570", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1871", "id": 1299598570, "node_id": "IC_kwDOBm6k_c5Ndkzq", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-02T05:34:28Z", "updated_at": "2022-11-02T05:34:28Z", "author_association": "OWNER", "body": "This is pretty useful now. Two features I still want to add:\r\n\r\n- The ability to link to the API explorer such that the form is pre-filled with material from the URL. Need to guard against clickjacking first though, so no-one can link to it in an invisible iframe and trick the user into hitting POST.\r\n- Some kind of list of endpoints so people can click links to start using the API explorer. A list of every table the user can write to with each of their `/db/table/-/insert` endpoints for example.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1427293909, "label": "API explorer tool"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1871#issuecomment-1299597066", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1871", "id": 1299597066, "node_id": "IC_kwDOBm6k_c5NdkcK", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-02T05:32:22Z", "updated_at": "2022-11-02T05:32:22Z", "author_association": "OWNER", "body": "Demo of the latest API explorer:\r\n\r\n![explorer](https://user-images.githubusercontent.com/9599/199406184-1292df42-25ea-4daf-8b54-ca26170ec1ea.gif)\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1427293909, "label": "API explorer tool"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1871#issuecomment-1299388341", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1871", "id": 1299388341, "node_id": "IC_kwDOBm6k_c5Ncxe1", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-02T00:24:28Z", "updated_at": "2022-11-02T00:25:00Z", "author_association": "OWNER", "body": "I want JSON syntax highlighting.\r\n\r\nhttps://github.com/luyilin/json-format-highlight is an MIT licensed tiny highlighter that looks decent for this.\r\n\r\nhttps://unpkg.com/json-format-highlight@1.0.1/dist/json-format-highlight.js", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1427293909, "label": "API explorer tool"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1871#issuecomment-1299349741", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1871", "id": 1299349741, "node_id": "IC_kwDOBm6k_c5NcoDt", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-01T23:22:55Z", "updated_at": "2022-11-01T23:22:55Z", "author_association": "OWNER", "body": "It's weird that the API explorer only lets you explore POST APIs. It should probably also let you explore GET APIs, or be renamed.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1427293909, "label": "API explorer tool"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1879#issuecomment-1299098458", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1879", "id": 1299098458, "node_id": "IC_kwDOBm6k_c5Nbqta", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-01T20:27:40Z", "updated_at": "2022-11-01T20:33:52Z", "author_association": "OWNER", "body": "https://github.com/simonw/datasette-x-forwarded-host/blob/main/datasette_x_forwarded_host/__init__.py could happen in core controlled by:\r\n\r\n`--setting trust_forwarded_host 1`", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1432037325, "label": "Make it easier to fix URL proxy problems"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1879#issuecomment-1299102108", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1879", "id": 1299102108, "node_id": "IC_kwDOBm6k_c5Nbrmc", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-01T20:30:54Z", "updated_at": "2022-11-01T20:33:06Z", "author_association": "OWNER", "body": "One idea: add a `/-/debug` page (or `/-/tips` or `/-/checks`) which shows the incoming requests headers and could even detect if there's an `x-forwarded-host` header that isn't being repeated and show a tip on how to fix that.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1432037325, "label": "Make it easier to fix URL proxy problems"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1879#issuecomment-1299102755", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1879", "id": 1299102755, "node_id": "IC_kwDOBm6k_c5Nbrwj", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-01T20:31:37Z", "updated_at": "2022-11-01T20:31:37Z", "author_association": "OWNER", "body": "And some JavaScript that can spot if Datasette thinks it is being served over HTTP when it's actually being served over HTTPS.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1432037325, "label": "Make it easier to fix URL proxy problems"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1879#issuecomment-1299096850", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1879", "id": 1299096850, "node_id": "IC_kwDOBm6k_c5NbqUS", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-01T20:26:12Z", "updated_at": "2022-11-01T20:26:12Z", "author_association": "OWNER", "body": "The other relevant plugin here is https://datasette.io/plugins/datasette-x-forwarded-host\r\n\r\nMaybe that should be rolled into core too?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1432037325, "label": "Make it easier to fix URL proxy problems"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1879#issuecomment-1299090678", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1879", "id": 1299090678, "node_id": "IC_kwDOBm6k_c5Nboz2", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-01T20:20:28Z", "updated_at": "2022-11-01T20:20:28Z", "author_association": "OWNER", "body": "My first step in debugging these is to install https://datasette.io/plugins/datasette-debug-asgi - but now I'm thinking maybe something like that should be part of core.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1432037325, "label": "Make it easier to fix URL proxy problems"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1862#issuecomment-1299073433", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1862", "id": 1299073433, "node_id": "IC_kwDOBm6k_c5NbkmZ", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-01T20:04:31Z", "updated_at": "2022-11-01T20:04:31Z", "author_association": "OWNER", "body": "It really feels like this should be accompanied by a `/db/-/create` API for creating tables. I had to add that to `sqlite-utils` eventually (initially it only supported creating by passing in an example document):\r\n\r\nhttps://sqlite-utils.datasette.io/en/stable/cli.html#cli-create-table", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1425011030, "label": "Create a new table from one or more records, `sqlite-utils` style"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1878#issuecomment-1299071456", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1878", "id": 1299071456, "node_id": "IC_kwDOBm6k_c5NbkHg", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-01T20:02:43Z", "updated_at": "2022-11-01T20:02:43Z", "author_association": "OWNER", "body": "Note that \"update\" is partially covered by the `replace` option to `/-/insert`, added here:\r\n- https://github.com/simonw/datasette/issues/1873#issuecomment-1298885451\r\n\r\n\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1432013704, "label": "/db/table/-/upsert API"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1873#issuecomment-1298919552", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1873", "id": 1298919552, "node_id": "IC_kwDOBm6k_c5Na_CA", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-01T18:11:27Z", "updated_at": "2022-11-01T18:11:27Z", "author_association": "OWNER", "body": "I forgot to document `ignore` and `replace`. Also I need to add tests that cover:\r\n\r\n- Forgetting to include a primary key on a non-autoincrement table\r\n- Compound primary keys\r\n- Rowid only tables with and without rowid specified\r\n\r\nI think my validation logic here will get caught out by the fact that `rowid` does not show up as a valid column name: https://github.com/simonw/datasette/blob/9bec7c38eb93cde5afb16df9bdd96aea2a5b0459/datasette/views/table.py#L1151-L1160\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1428630253, "label": "Ensure insert API has good tests for rowid and compound primark key tables"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1873#issuecomment-1298905135", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1873", "id": 1298905135, "node_id": "IC_kwDOBm6k_c5Na7gv", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-01T17:59:59Z", "updated_at": "2022-11-01T17:59:59Z", "author_association": "OWNER", "body": "It's a bit surprising that you can send `\"ignore\": true, \"return_rows\": true` and the returned `\"inserted\"` key will list rows that were NOT inserted (since they were ignored).\r\n\r\nThree options:\r\n\r\n1. Ignore that and document it\r\n2. Fix it so `\"inserted\"` only returns rows that were actually inserted (bit tricky)\r\n3. Change the name of `\"inserted\"` to something else\r\n\r\nI'm picking 3 - I'm going to change it to be called `\"rows\"` instead.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1428630253, "label": "Ensure insert API has good tests for rowid and compound primark key tables"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1873#issuecomment-1298885451", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1873", "id": 1298885451, "node_id": "IC_kwDOBm6k_c5Na2tL", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-01T17:42:20Z", "updated_at": "2022-11-01T17:42:20Z", "author_association": "OWNER", "body": "Design decision:\r\n```json\r\n{\r\n  \"rows\": [{\"id\": 1, \"title\": \"The title\"}],\r\n  \"ignore\": true\r\n}\r\n```\r\nOr `\"replace\": true`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1428630253, "label": "Ensure insert API has good tests for rowid and compound primark key tables"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/506#issuecomment-1298879701", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/506", "id": 1298879701, "node_id": "IC_kwDOCGYnMM5Na1TV", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-01T17:37:13Z", "updated_at": "2022-11-01T17:37:13Z", "author_association": "OWNER", "body": "The question I was originally trying to answer here was this: how many rows were actually inserted by that call to `.insert_all()`?\r\n\r\nI don't know that `.rowcount` would ever be useful here, since the \"correct\" answer depends on other factors - had I determined to ignore or replace records with a primary key that matches an existing record for example?\r\n\r\nSo I think if people need `rowcount` they can get it by using a `cursor` directly.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1429029604, "label": "Make `cursor.rowcount` accessible (wontfix)"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/506#issuecomment-1298877872", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/506", "id": 1298877872, "node_id": "IC_kwDOCGYnMM5Na02w", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-01T17:35:30Z", "updated_at": "2022-11-01T17:35:30Z", "author_association": "OWNER", "body": "This may not make sense.\r\n\r\nFirst, `.last_rowid` is a property on table - but that doesn't make sense for `rowcount` since it should clearly be a property on the database itself (you can run a query directly using `db.execute()` without going through a `Table` object).\r\n\r\nSo I tried this prototype:\r\n\r\n```diff\r\ndiff --git a/docs/python-api.rst b/docs/python-api.rst\r\nindex 206e5e6..78d3a8d 100644\r\n--- a/docs/python-api.rst\r\n+++ b/docs/python-api.rst\r\n@@ -186,6 +186,15 @@ The ``db.query(sql)`` function executes a SQL query and returns an iterator over\r\n     # {'name': 'Cleo'}\r\n     # {'name': 'Pancakes'}\r\n \r\n+After executing a query the ``db.rowcount`` property on that database instance will reflect the number of rows affected by any insert, update or delete operations performed by that query:\r\n+\r\n+.. code-block:: python\r\n+\r\n+    db = Database(memory=True)\r\n+    db[\"dogs\"].insert_all([{\"name\": \"Cleo\"}, {\"name\": \"Pancakes\"}])\r\n+    print(db.rowcount)\r\n+    # Outputs: 2\r\n+\r\n .. _python_api_execute:\r\n \r\n db.execute(sql, params)\r\ndiff --git a/sqlite_utils/db.py b/sqlite_utils/db.py\r\nindex a06f4b7..c19c2dd 100644\r\n--- a/sqlite_utils/db.py\r\n+++ b/sqlite_utils/db.py\r\n@@ -294,6 +294,8 @@ class Database:\r\n \r\n     _counts_table_name = \"_counts\"\r\n     use_counts_table = False\r\n+    # Number of rows inserted, updated or deleted\r\n+    rowcount: Optional[int] = None\r\n \r\n     def __init__(\r\n         self,\r\n@@ -480,9 +482,11 @@ class Database:\r\n         if self._tracer:\r\n             self._tracer(sql, parameters)\r\n         if parameters is not None:\r\n-            return self.conn.execute(sql, parameters)\r\n+            cursor = self.conn.execute(sql, parameters)\r\n         else:\r\n-            return self.conn.execute(sql)\r\n+            cursor = self.conn.execute(sql)\r\n+        self.rowcount = cursor.rowcount\r\n+        return cursor\r\n \r\n     def executescript(self, sql: str) -> sqlite3.Cursor:\r\n         \"\"\"\r\n```\r\nBut this happens:\r\n```pycon\r\n>>> from sqlite_utils import Database\r\n>>> db = Database(memory=True)\r\n>>> db[\"dogs\"].insert_all([{\"name\": \"Cleo\"}, {\"name\": \"Pancakes\"}])\r\n<Table dogs (name)>\r\n>>> db.rowcount\r\n-1\r\n```\r\nTurning on query tracing demonstrates why:\r\n```pycon\r\n>>> db = Database(memory=True, tracer=print)\r\nPRAGMA recursive_triggers=on; None\r\n>>> db[\"dogs\"].insert_all([{\"name\": \"Cleo\"}, {\"name\": \"Pancakes\"}])\r\nselect name from sqlite_master where type = 'view' None\r\nselect name from sqlite_master where type = 'table' None\r\nselect name from sqlite_master where type = 'view' None\r\nCREATE TABLE [dogs] (\r\n   [name] TEXT\r\n);\r\n         None\r\nselect name from sqlite_master where type = 'view' None\r\nINSERT INTO [dogs] ([name]) VALUES (?), (?); ['Cleo', 'Pancakes']\r\nselect name from sqlite_master where type = 'table' None\r\nselect name from sqlite_master where type = 'table' None\r\nPRAGMA table_info([dogs]) None\r\n<Table dogs (name)>\r\n>>>\r\n```\r\nThe `.insert_all()` function does a bunch of other queries too, so `.rowcount` is quickly over-ridden by the same result from extra queries that it executed.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1429029604, "label": "Make `cursor.rowcount` accessible (wontfix)"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1876#issuecomment-1298856054", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1876", "id": 1298856054, "node_id": "IC_kwDOBm6k_c5Navh2", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-01T17:16:01Z", "updated_at": "2022-11-01T17:16:01Z", "author_association": "OWNER", "body": "`ta.style.height = ta.scrollHeight + 'px'` is an easy way to do that.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1431786951, "label": "SQL query should wrap on SQL interrupted screen"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1876#issuecomment-1298854321", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1876", "id": 1298854321, "node_id": "IC_kwDOBm6k_c5NavGx", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-01T17:14:33Z", "updated_at": "2022-11-01T17:14:33Z", "author_association": "OWNER", "body": "I could use a `textarea` here (would need to figure out a neat pattern to expand it to fit the query):\r\n\r\n<img width=\"426\" alt=\"image\" src=\"https://user-images.githubusercontent.com/9599/199295041-25abe0b9-f825-43a2-ae5a-face622e08bc.png\">\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1431786951, "label": "SQL query should wrap on SQL interrupted screen"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1864#issuecomment-1296403316", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1864", "id": 1296403316, "node_id": "IC_kwDOBm6k_c5NRYt0", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-10-31T00:39:43Z", "updated_at": "2022-10-31T00:39:43Z", "author_association": "OWNER", "body": "It looks like SQLite has features for this already: https://www.sqlite.org/foreignkeys.html#fk_actions\r\n\r\n> Foreign key ON DELETE and ON UPDATE clauses are used to configure actions that take place when deleting rows from the parent table (ON DELETE), or modifying the parent key values of existing rows (ON UPDATE). A single foreign key constraint may have different actions configured for ON DELETE and ON UPDATE. Foreign key actions are similar to triggers in many ways. \r\n\r\nOn that basis, I'm not going to implement anything additional in the `.../-/delete` endpoint relating to foreign keys. Developers who want special treatment of them can do that with a combination of a plugin (maybe I'll build a `datasette-enable-foreign-keys` plugin) and tables created using those `ON DELETE` clauses.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1425029275, "label": "Delete a single record from an existing table"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1864#issuecomment-1296402071", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1864", "id": 1296402071, "node_id": "IC_kwDOBm6k_c5NRYaX", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-10-31T00:37:09Z", "updated_at": "2022-10-31T00:37:09Z", "author_association": "OWNER", "body": "I need to think about what happens if you delete a row that is the target of a foreign key from another row.\r\n\r\nhttps://www.sqlite.org/foreignkeys.html#fk_enable shows that SQLite will only actively enforce these relationships (e.g. throw an error if you try to delete a row that is referenced by another row) if you first run `PRAGMA foreign_keys = ON;` against the connection.\r\n\r\n> Foreign key constraints are disabled by default (for backwards compatibility), so must be enabled separately for each [database connection](https://www.sqlite.org/c3ref/sqlite3.html). (Note, however, that future releases of SQLite might change so that foreign key constraints enabled by default. Careful developers will not make any assumptions about whether or not foreign keys are enabled by default but will instead enable or disable them as necessary.)\r\n\r\nI don't actually believe that the SQLite maintainers will ever make that the default though.\r\n\r\nDatasette doesn't turn these on at the moment, but it could be turned on by a `prepare_connection()` plugin.\r\n\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1425029275, "label": "Delete a single record from an existing table"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1864#issuecomment-1296375536", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1864", "id": 1296375536, "node_id": "IC_kwDOBm6k_c5NRR7w", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-10-30T23:17:11Z", "updated_at": "2022-10-30T23:17:11Z", "author_association": "OWNER", "body": "I'm a bit nervous about calling `.delete()` with the `pk_values` - can I be sure they are in the correct order? https://github.com/simonw/datasette/blob/00632ded30e7cf9f0cf9478680645d1dabe269ae/datasette/views/row.py#L188-L190", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1425029275, "label": "Delete a single record from an existing table"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1864#issuecomment-1296375310", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1864", "id": 1296375310, "node_id": "IC_kwDOBm6k_c5NRR4O", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-10-30T23:16:19Z", "updated_at": "2022-10-30T23:16:19Z", "author_association": "OWNER", "body": "Still needs tests that cover compound primary keys and rowid tables.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1425029275, "label": "Delete a single record from an existing table"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1874#issuecomment-1296363981", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1874", "id": 1296363981, "node_id": "IC_kwDOBm6k_c5NRPHN", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-10-30T22:19:47Z", "updated_at": "2022-10-30T22:19:47Z", "author_association": "OWNER", "body": "Documentation: https://docs.datasette.io/en/1.0-dev/json_api.html#dropping-tables", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1429030341, "label": "API to drop a table"}, "performed_via_github_app": null}