.status_code\r\n```\r\nI hit \"retry\" on that test but I expect it to fail again.\r\n\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1913/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1432012302, "node_id": "I_kwDOBm6k_c5VWsYO", "number": 1877, "title": "Refactor and tidy up final write API code", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2022-11-01T20:00:11Z", "updated_at": "2022-11-29T19:44:16Z", "closed_at": "2022-11-29T19:44:07Z", "author_association": "OWNER", "pull_request": null, "body": "- `views/table.py` has got a bit too big - I think the write classes should be pulled out into a separate module.\r\n- [x] There's duplicate logic for deciding if the table and database exist and checking permissions", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1877/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1450312343, "node_id": "I_kwDOBm6k_c5WcgKX", "number": 1892, "title": "Merge 1.0-dev branch back to main", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 8658075, "label": "Datasette 1.0a0"}, "comments": 3, "created_at": "2022-11-15T20:04:25Z", "updated_at": "2022-11-29T19:40:23Z", "closed_at": "2022-11-29T19:40:23Z", "author_association": "OWNER", "pull_request": null, "body": "I'm committed enough to the 1.0 work now that I'm ready for the `main` branch to reflect that instead.\r\n\r\nIf I need to make any dot-releases against 0.63 I can do those from a branch.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1892/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1468592292, "node_id": "PR_kwDOBm6k_c5D6nzE", "number": 1912, "title": "Merge 1.0-dev (with initial write API) back into main", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2022-11-29T19:31:21Z", "updated_at": "2022-11-29T19:39:37Z", "closed_at": "2022-11-29T19:39:36Z", "author_association": "OWNER", "pull_request": "simonw/datasette/pulls/1912", "body": "See:\r\n- #1892\r\n\r\n\r\n----\n:books: Documentation preview :books:: https://datasette--1912.org.readthedocs.build/en/1912/\n\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1912/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 1450303205, "node_id": "I_kwDOBm6k_c5Wcd7l", "number": 1891, "title": "1.0a0 release notes", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 8658075, "label": "Datasette 1.0a0"}, "comments": 4, "created_at": "2022-11-15T19:58:20Z", "updated_at": "2022-11-29T19:23:41Z", "closed_at": "2022-11-29T19:23:41Z", "author_association": "OWNER", "pull_request": null, "body": "This release will mainly help preview the new Datasette write API:\r\n- #1850", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1891/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1425029275, "node_id": "I_kwDOBm6k_c5U8Dib", "number": 1864, "title": "Delete a single record from an existing table", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 8658075, "label": "Datasette 1.0a0"}, "comments": 4, "created_at": "2022-10-27T04:53:22Z", "updated_at": "2022-11-29T18:54:04Z", "closed_at": "2022-11-29T18:54:04Z", "author_association": "OWNER", "pull_request": null, "body": "API design:\r\n```\r\nPOST /db/table/row-pks/-/delete\r\nOr...\r\nDELETE /db/table/row-pks/-/delete\r\n```\r\nI'm just going to do `POST` for the moment, like I did here:\r\n- #1874\r\n\r\nPermission: `delete-row`\r\n\r\nStill needed:\r\n\r\n- [ ] Tests for rowid tables\r\n- [ ] Tests for compound primary keys", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1864/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1468519699, "node_id": "I_kwDOBm6k_c5Xh9UT", "number": 1911, "title": "`/db/-/create` should support creating tables with compound primary keys", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 8658075, "label": "Datasette 1.0a0"}, "comments": 2, "created_at": "2022-11-29T18:30:47Z", "updated_at": "2022-11-29T18:50:58Z", "closed_at": "2022-11-29T18:48:05Z", "author_association": "OWNER", "pull_request": null, "body": "Found myself needing this to write the tests for:\r\n- #1864 ", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1911/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1425029242, "node_id": "I_kwDOBm6k_c5U8Dh6", "number": 1863, "title": "Update a single record in an existing table", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 8658075, "label": "Datasette 1.0a0"}, "comments": 16, "created_at": "2022-10-27T04:53:17Z", "updated_at": "2022-11-29T18:08:53Z", "closed_at": "2022-11-29T18:06:37Z", "author_association": "OWNER", "pull_request": null, "body": "API design:\r\n\r\n```\r\nPOST /db/table/row-pks/-/update\r\n{\r\n \"field\": \"updated_value\"\r\n}\r\n```\r\nOnly the fields that you pass will be updated.\r\n\r\nMaybe this is the wrong design though? The design for insert currently looks like this:\r\n\r\n- https://github.com/simonw/datasette/issues/1851#issuecomment-1294224185\r\n\r\n```\r\nPOST /db/table/-/insert\r\nAuthorization: Bearer xxx\r\nContent-Type: application/json\r\n{\r\n \"row\": {\r\n \"id\": 1,\r\n \"name\": \"New name\"\r\n }\r\n}\r\n```\r\nI could use the same format for `/-/update`, but in this case the API doesn't require you to pass every field so `\"row\"` doesn't seem like the right key.\r\n\r\nI think I'll go with this:\r\n\r\n```\r\nPOST /db/table/1/-/update\r\nAuthorization: Bearer xxx\r\nContent-Type: application/json\r\n{\r\n \"update\": {\r\n \"name\": \"New name\"\r\n }\r\n}\r\n```\r\nThe benefit of having an `\"update\"` key is that it allows me to use other keys in the future. Maybe a `\"alter\": true` key to indicate that new columns should be added if they are missing.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1863/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1456012874, "node_id": "I_kwDOBm6k_c5WyP5K", "number": 1905, "title": "`publish heroku` failing due to old Python version", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2022-11-19T00:01:45Z", "updated_at": "2022-11-19T01:12:05Z", "closed_at": "2022-11-19T00:52:29Z", "author_association": "OWNER", "pull_request": null, "body": "Reported on Discord: https://discord.com/channels/823971286308356157/823971286941302908/1042814317118115901\r\n\r\n```\r\n-----> Building on the Heroku-22 stack\r\n-----> Determining which buildpack to use for this app\r\n-----> Python app detected\r\n-----> Using Python version specified in runtime.txt\r\n ! Requested runtime 'python-3.8.10' is not available for this stack (heroku-22).\r\n ! For supported versions, see: https://devcenter.heroku.com/articles/python-support\r\n ! Push rejected, failed to compile Python app.\r\n\r\n ! Push failed\r\n \u25b8 Build failed\r\n```\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1905/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1455932972, "node_id": "I_kwDOBm6k_c5Wx8Ys", "number": 1904, "title": "Datasette Lite tests failing due to httpx upgrade", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 8658075, "label": "Datasette 1.0a0"}, "comments": 0, "created_at": "2022-11-18T22:49:31Z", "updated_at": "2022-11-18T22:57:48Z", "closed_at": "2022-11-18T22:52:22Z", "author_association": "OWNER", "pull_request": null, "body": "Same problem as this one:\r\n- https://github.com/simonw/datasette-lite/issues/56\r\n\r\nCaused this failure: https://github.com/simonw/datasette/actions/runs/3500765964", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1904/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1452364777, "node_id": "I_kwDOBm6k_c5WkVPp", "number": 1896, "title": "Extract logic for resolving a URL to a database / table / row", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 8658075, "label": "Datasette 1.0a0"}, "comments": 4, "created_at": "2022-11-16T22:25:20Z", "updated_at": "2022-11-18T22:57:47Z", "closed_at": "2022-11-18T22:56:55Z", "author_association": "OWNER", "pull_request": null, "body": "> In trying to write this I realize that there's a lot of duplicated code with delete row, specifically around resolving the incoming URL into a row (or a database or a table).\r\n>\r\n> Since this is so common, I think it's worth extracting the logic out first.\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1863#issuecomment-1317755263_\r\n ", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1896/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1434911255, "node_id": "I_kwDOCGYnMM5VhwIX", "number": 510, "title": "Cannot enable FTS5 despite it being available", "user": {"value": 1176293, "label": "ar-jan"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2022-11-03T16:03:49Z", "updated_at": "2022-11-18T18:37:52Z", "closed_at": "2022-11-17T10:36:28Z", "author_association": "NONE", "pull_request": null, "body": "When I do `sqlite-utils enable-fts my.db table_name column_name` (with or without `--fts5`), I get an FTS4 virtual table instead of the expected FTS5.\r\n\r\nFTS5 is however available and Python/SQLite versions do not seem to be the issue. I can manually create the FTS5 virtual table, and then Datasette also works with it from this same Python environment.\r\n\r\n`>>> sqlite3.version`\r\n`2.6.0`\r\n`>>> sqlite3.sqlite_version`\r\n`3.39.4`\r\n\r\n`PRAGMA compile_options;` includes `ENABLE_FTS5`.\r\n\r\n`sqlite-utils, version 3.30`.\r\n\r\nAny ideas what's happening and how to fix?", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/510/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1450796965, "node_id": "I_kwDOBm6k_c5WeWel", "number": 1894, "title": "Initialize CodeMirror during DOMContentLoaded instead of onload", "user": {"value": 95570, "label": "bgrins"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2022-11-16T03:52:19Z", "updated_at": "2022-11-18T07:29:02Z", "closed_at": "2022-11-18T07:29:02Z", "author_association": "CONTRIBUTOR", "pull_request": null, "body": "As per https://github.com/simonw/datasette/pull/1893/files#r1023248927 this should prevent a flash between the textarea being replaced by CodeMirror.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1894/reactions\", \"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1452485922, "node_id": "PR_kwDOBm6k_c5DEh-E", "number": 1898, "title": "Use DOMContentLoaded instead of load event for CodeMirror initialization", "user": {"value": 95570, "label": "bgrins"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2022-11-17T00:19:21Z", "updated_at": "2022-11-18T07:29:01Z", "closed_at": "2022-11-18T07:29:01Z", "author_association": "CONTRIBUTOR", "pull_request": "simonw/datasette/pulls/1898", "body": " Closes #1894\r\n\r\n\r\n----\n:books: Documentation preview :books:: https://datasette--1898.org.readthedocs.build/en/1898/\n\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1898/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 1452495049, "node_id": "I_kwDOBm6k_c5Wk1DJ", "number": 1899, "title": "Clicking within the CodeMirror area below the SQL (i.e. when there's only a single line) doesn't cause the editor to get focused ", "user": {"value": 95570, "label": "bgrins"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2022-11-17T00:29:52Z", "updated_at": "2022-11-18T07:28:28Z", "closed_at": "2022-11-18T07:20:53Z", "author_association": "CONTRIBUTOR", "pull_request": null, "body": "After the upgrade to 6 (#1893) I noticed this. I think it's because we're doing overflow:hidden to accomplish the CSS resizer.\r\n\r\nWhen there's a single line of SQL there's a gap below that line where clicking doesn't do anything. It should focus at the end of the line.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1899/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1453813400, "node_id": "I_kwDOBm6k_c5Wp26Y", "number": 1901, "title": "Some plugins show \"home\" breadcrumbs twice in the top left", "user": {"value": 95570, "label": "bgrins"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 8, "created_at": "2022-11-17T18:44:58Z", "updated_at": "2022-11-18T07:22:37Z", "closed_at": "2022-11-18T07:02:56Z", "author_association": "CONTRIBUTOR", "pull_request": null, "body": "\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1901/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1452457263, "node_id": "I_kwDOBm6k_c5Wkr0v", "number": 1897, "title": "Serve schema JSON to the SQL editor to enable autocomplete", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 9, "created_at": "2022-11-16T23:47:45Z", "updated_at": "2022-11-18T05:33:20Z", "closed_at": "2022-11-18T02:54:43Z", "author_association": "OWNER", "pull_request": null, "body": "See:\r\n- https://github.com/simonw/datasette/issues/1893#issuecomment-1317831555\r\n ", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1897/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1430563092, "node_id": "PR_kwDOCGYnMM5B6_6K", "number": 508, "title": "Allow surrogates in parameters", "user": {"value": 7908073, "label": "chapmanjacobd"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2022-10-31T22:11:49Z", "updated_at": "2022-11-17T15:11:16Z", "closed_at": "2022-10-31T22:55:36Z", "author_association": "CONTRIBUTOR", "pull_request": "simonw/sqlite-utils/pulls/508", "body": "closes #507\r\n\r\nhttps://dwheeler.com/essays/fixing-unix-linux-filenames.html\r\n\r\n\r\n----\n:books: Documentation preview :books:: https://sqlite-utils--508.org.readthedocs.build/en/508/\n\r\n", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/508/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 1448143294, "node_id": "I_kwDOBm6k_c5WUOm-", "number": 1890, "title": "Autocomplete text entry for filter values that correspond to facets", "user": {"value": 536941, "label": "fgregg"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 16, "created_at": "2022-11-14T14:11:31Z", "updated_at": "2022-11-17T00:47:36Z", "closed_at": "2022-11-16T03:23:01Z", "author_association": "CONTRIBUTOR", "pull_request": null, "body": "datasette allows users to enter in the value for named parameters into a free-text form field.\r\n\r\nI think it would add a lot of usability, if the form field could be a drop down of options when query value is already a faceted column.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1890/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1450363982, "node_id": "PR_kwDOBm6k_c5C9ZuP", "number": 1893, "title": "Upgrade to CodeMirror 6, add SQL autocomplete", "user": {"value": 95570, "label": "bgrins"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 48, "created_at": "2022-11-15T20:52:35Z", "updated_at": "2022-11-16T23:54:02Z", "closed_at": "2022-11-16T23:49:06Z", "author_association": "CONTRIBUTOR", "pull_request": "simonw/datasette/pulls/1893", "body": "In an effort to get closer to table / column autocomplete I took a shot at https://github.com/simonw/datasette/issues/1796. I haven't done a lot of testing but would be curious if this fixes some of the concerns raised in https://github.com/simonw/datasette/issues/1796#issue-1355148385 for example.\r\n\r\nDone:\r\n* Changed to bundling using rollup as per https://codemirror.net/examples/bundle/\r\n* Restored a fromTextArea-like function from https://codemirror.net/docs/migration/\r\n* Removed old JS and CSS files (no external CSS needed anymore as per https://codemirror.net/examples/styling/)\r\n* Updated instructions for building the bundle\r\n\r\nNot done:\r\n* cmResize had an error, so commented out the resize handle\r\n* Add extraKeys option for shift+enter and tab\r\n\r\n\r\n----\n:books: Documentation preview :books:: https://datasette--1893.org.readthedocs.build/en/1893/\n\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1893/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 1450952393, "node_id": "I_kwDOCGYnMM5We8bJ", "number": 512, "title": "mypy failures in CI", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2022-11-16T06:22:48Z", "updated_at": "2022-11-16T07:49:51Z", "closed_at": "2022-11-16T07:49:50Z", "author_association": "OWNER", "pull_request": null, "body": "https://github.com/simonw/sqlite-utils/actions/runs/3472012235 failed on Python 3.11:\r\n\r\nTruncated output:\r\n```\r\nsqlite_utils/db.py:2467: note: PEP 484 prohibits implicit Optional. Accordingly, mypy has changed its default to no_implicit_optional=True\r\nsqlite_utils/db.py:2467: note: Use https://github.com/hauntsaninja/no_implicit_optional to automatically upgrade your codebase\r\nsqlite_utils/db.py:2530: error: Incompatible default for argument \"where\" (default has type \"None\", argument has type \"str\") [assignment]\r\nsqlite_utils/db.py:2530: note: PEP 484 prohibits implicit Optional. Accordingly, mypy has changed its default to no_implicit_optional=True\r\nsqlite_utils/db.py:2530: note: Use https://github.com/hauntsaninja/no_implicit_optional to automatically upgrade your codebase\r\nsqlite_utils/db.py:2658: error: Argument 1 to \"count_where\" of \"Queryable\" has incompatible type \"Optional[str]\"; expected \"str\" [arg-type]\r\nFound 23 errors in 1 file (checked 51 source files)\r\n```\r\nBest look at https://github.com/hauntsaninja/no_implicit_optional", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/512/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1447388809, "node_id": "I_kwDOBm6k_c5WRWaJ", "number": 1887, "title": "Add a confirm step to the drop table API", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 8658075, "label": "Datasette 1.0a0"}, "comments": 2, "created_at": "2022-11-14T04:59:53Z", "updated_at": "2022-11-15T19:59:59Z", "closed_at": "2022-11-14T05:18:51Z", "author_association": "OWNER", "pull_request": null, "body": "> In playing with the API explorer just now I realized it's way too easy to accidentally drop a table using it.\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1871#issuecomment-1313097057_\r\n\r\nAdded drop table API in:\r\n- #1874", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1887/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1429030341, "node_id": "I_kwDOBm6k_c5VLUXF", "number": 1874, "title": "API to drop a table", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 8658075, "label": "Datasette 1.0a0"}, "comments": 4, "created_at": "2022-10-30T21:55:11Z", "updated_at": "2022-11-15T19:59:53Z", "closed_at": "2022-11-14T05:45:06Z", "author_association": "OWNER", "pull_request": null, "body": "`POST /db/table/-/drop`\r\n\r\nRequire `drop-table` permission.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1874/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1425011030, "node_id": "I_kwDOBm6k_c5U7_FW", "number": 1862, "title": "Create a new table from one or more records, `sqlite-utils` style", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 8658075, "label": "Datasette 1.0a0"}, "comments": 5, "created_at": "2022-10-27T04:25:02Z", "updated_at": "2022-11-15T19:59:47Z", "closed_at": "2022-11-15T06:42:09Z", "author_association": "OWNER", "pull_request": null, "body": "It's interesting to also think about what the form-based UI for this could look like - since that would involve users creating new columns of different types on the fly.\r\n\r\nWill need the `create-table` permission.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1862/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1435294468, "node_id": "I_kwDOBm6k_c5VjNsE", "number": 1882, "title": "`/db/-/create` API for creating tables", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 8658075, "label": "Datasette 1.0a0"}, "comments": 12, "created_at": "2022-11-03T21:44:32Z", "updated_at": "2022-11-15T19:59:43Z", "closed_at": "2022-11-15T06:00:41Z", "author_association": "OWNER", "pull_request": null, "body": "> It really feels like this should be accompanied by a `/db/-/create` API for creating tables. I had to add that to `sqlite-utils` eventually (initially it only supported creating by passing in an example document):\r\n>\r\n> https://sqlite-utils.datasette.io/en/stable/cli.html#cli-create-table\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1862#issuecomment-1299073433_\r\n ", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1882/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1426001541, "node_id": "I_kwDOBm6k_c5U_w6F", "number": 1866, "title": "API for bulk inserting records into a table", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 8658075, "label": "Datasette 1.0a0"}, "comments": 12, "created_at": "2022-10-27T17:19:25Z", "updated_at": "2022-11-15T19:59:34Z", "closed_at": "2022-10-30T06:04:07Z", "author_association": "OWNER", "pull_request": null, "body": "Similar to https://github.com/simonw/datasette-insert/blob/0.8/README.md#inserting-data-and-creating-tables\r\n\r\nI expect this to become by far the most common way that data gets into a Datasette instance - more so than the individual row API in:\r\n- #1851 ", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1866/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1421544654, "node_id": "I_kwDOBm6k_c5UuwzO", "number": 1851, "title": "API to insert a single record into an existing table", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 8658075, "label": "Datasette 1.0a0"}, "comments": 22, "created_at": "2022-10-24T22:24:21Z", "updated_at": "2022-11-15T19:59:18Z", "closed_at": "2022-10-28T00:59:25Z", "author_association": "OWNER", "pull_request": null, "body": "Controlled by a new `insert-row` permission.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1851/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1426195437, "node_id": "I_kwDOBm6k_c5VAgPt", "number": 1868, "title": "Design URLs for the write API", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 8658075, "label": "Datasette 1.0a0"}, "comments": 5, "created_at": "2022-10-27T19:55:30Z", "updated_at": "2022-11-15T19:59:14Z", "closed_at": "2022-10-27T20:07:01Z", "author_association": "OWNER", "pull_request": null, "body": "My original design for this issue:\r\n- #1851\r\n\r\nWas `POST /db/table` with JSON of `{\"insert\": {...}}`.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1868/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1447439985, "node_id": "I_kwDOBm6k_c5WRi5x", "number": 1888, "title": "API explorer should take immutability into account", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 8658075, "label": "Datasette 1.0a0"}, "comments": 1, "created_at": "2022-11-14T06:00:14Z", "updated_at": "2022-11-15T19:59:10Z", "closed_at": "2022-11-14T06:04:48Z", "author_association": "OWNER", "pull_request": null, "body": "Refs:\r\n- #1871\r\n\r\nI noticed the API explorer doesn't show any links on https://latest-1-0-dev.datasette.io/-/api because the `fixtures` database is immutable.\r\n\r\nIt should still show read examples there.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1888/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1427293909, "node_id": "I_kwDOBm6k_c5VEsbV", "number": 1871, "title": "API explorer tool", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 8658075, "label": "Datasette 1.0a0"}, "comments": 24, "created_at": "2022-10-28T13:49:11Z", "updated_at": "2022-11-15T19:59:05Z", "closed_at": "2022-11-14T04:59:59Z", "author_association": "OWNER", "pull_request": null, "body": "The API will be much easier to develop if there's a page that helps you execute JSON POSTs against it.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1871/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1423369494, "node_id": "I_kwDOBm6k_c5U1uUW", "number": 1859, "title": "datasette create-token CLI command", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 8658075, "label": "Datasette 1.0a0"}, "comments": 3, "created_at": "2022-10-26T03:12:59Z", "updated_at": "2022-11-15T19:59:00Z", "closed_at": "2022-10-26T04:31:39Z", "author_association": "OWNER", "pull_request": null, "body": "The CLI equivalent of the `/-/create-token` page.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1859/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1423364990, "node_id": "I_kwDOBm6k_c5U1tN-", "number": 1858, "title": "`max_signed_tokens_ttl` setting for a maximum duration on API tokens", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 8658075, "label": "Datasette 1.0a0"}, "comments": 4, "created_at": "2022-10-26T03:05:53Z", "updated_at": "2022-11-15T19:58:52Z", "closed_at": "2022-10-27T03:15:05Z", "author_association": "OWNER", "pull_request": null, "body": "It's currently possible to use `/-/create-token` to create a token that lasts forever.\r\n\r\nSome administrators may wish to have a maximum expiry instead. I should support that with a setting.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1858/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1423347412, "node_id": "I_kwDOBm6k_c5U1o7U", "number": 1857, "title": "Prevent API tokens from using /-/create-token to create more tokens", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 8658075, "label": "Datasette 1.0a0"}, "comments": 1, "created_at": "2022-10-26T02:38:09Z", "updated_at": "2022-11-15T19:57:11Z", "closed_at": "2022-10-26T02:57:26Z", "author_association": "OWNER", "pull_request": null, "body": "> It strikes me that users should NOT be able to use a token to create additional tokens.\r\n>\r\n> The current design actually does allow that, since the `dstok_` Bearer token can be used to authenticate calls to the `/-/create-token` page.\r\n>\r\n> So I think I need a mechanism whereby that page can only allow access to users authenticated by cookie.\r\n> \r\n> Not obvious how to do that though, since Datasette's authentication actor system is designed to abstract that detail away!\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1850#issuecomment-1291417100_", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1857/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1423336122, "node_id": "I_kwDOBm6k_c5U1mK6", "number": 1856, "title": "allow_signed_tokens setting for disabling API signed token mechanism", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 8658075, "label": "Datasette 1.0a0"}, "comments": 3, "created_at": "2022-10-26T02:20:55Z", "updated_at": "2022-11-15T19:57:05Z", "closed_at": "2022-10-26T02:58:35Z", "author_association": "OWNER", "pull_request": null, "body": "Had some design thoughts here: https://github.com/simonw/datasette/issues/1852#issuecomment-1291272280\r\n\r\nI liked this option the most:\r\n\r\n --setting allow_create_tokens off", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1856/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1421552095, "node_id": "I_kwDOBm6k_c5Uuynf", "number": 1852, "title": "Default API token authentication mechanism", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 8658075, "label": "Datasette 1.0a0"}, "comments": 30, "created_at": "2022-10-24T22:31:07Z", "updated_at": "2022-11-15T19:57:00Z", "closed_at": "2022-10-26T02:19:54Z", "author_association": "OWNER", "pull_request": null, "body": "API authentication will be via `Authorization: Bearer XXX` request headers.\r\n\r\nI'm inclined to add a default token mechanism to Datasette based on tokens that are signed with the `DATASETTE_SECRET`. Maybe the root user can access `/-/create-token` which provides a UI for generating a time-limited signed token? Could also have a `datasette token` command for creating such tokens at the command-line.\r\n\r\nPlugins can then define alternative ways of creating tokens, such as the existing https://datasette.io/plugins/datasette-auth-tokens plugin.\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1850#issuecomment-1289706439_\r\n ", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1852/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1435917503, "node_id": "I_kwDOBm6k_c5Vlly_", "number": 1883, "title": "Errors when using table filters behind a proxy", "user": {"value": 31312775, "label": "mattmalcher"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 13, "created_at": "2022-11-04T11:18:47Z", "updated_at": "2022-11-11T09:20:22Z", "closed_at": "2022-11-11T06:54:58Z", "author_association": "NONE", "pull_request": null, "body": "Using datasette==0.63 table filters do not respect the `base_url` setting as described [here](https://docs.datasette.io/en/stable/deploying.html#running-datasette-behind-a-proxy)\r\n\r\nTo reproduce, go to:\r\nhttps://datasette-apache-proxy-demo.datasette.io/prefix/fixtures/binary_data\r\n\r\nThen use the table filter buttons.\r\nThe `/prefix/` is dropped, resulting in URL not found:\r\nhttps://datasette-apache-proxy-demo.datasette.io/fixtures/binary_data?_sort=rowid&rowid__exact=1\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1883/reactions\", \"total_count\": 1, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 1}", "draft": null, "state_reason": "completed"}
{"id": 1436539554, "node_id": "I_kwDOCGYnMM5Vn9qi", "number": 511, "title": "[insert_all, upsert_all] IntegrityError: constraint failed", "user": {"value": 7908073, "label": "chapmanjacobd"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2022-11-04T19:21:48Z", "updated_at": "2022-11-04T22:59:54Z", "closed_at": "2022-11-04T22:54:09Z", "author_association": "CONTRIBUTOR", "pull_request": null, "body": "My understand is that `INSERT OR IGNORE` will ignore when inserts would cause duplicate keys so I'm not sure exactly why the error is raised from `sqlite3`.\r\n\r\n```\r\nimport argparse\r\nfrom pathlib import Path\r\n\r\nfrom xklb import db, utils\r\nfrom xklb.utils import log\r\n\r\n\r\ndef parse_args() -> argparse.Namespace:\r\n parser = argparse.ArgumentParser()\r\n parser.add_argument(\"database\")\r\n parser.add_argument(\"dbs\", nargs=\"*\")\r\n parser.add_argument(\"--upsert\")\r\n parser.add_argument(\"--db\", \"-db\", help=argparse.SUPPRESS)\r\n parser.add_argument(\"--verbose\", \"-v\", action=\"count\", default=0)\r\n args = parser.parse_args()\r\n\r\n if args.db:\r\n args.database = args.db\r\n Path(args.database).touch()\r\n args.db = db.connect(args)\r\n log.info(utils.dict_filter_bool(args.__dict__))\r\n\r\n return args\r\n\r\n\r\ndef merge_db(args, source_db):\r\n source_db = str(Path(source_db).resolve())\r\n\r\n s_db = db.connect(argparse.Namespace(database=source_db, verbose=args.verbose))\r\n for table in [s for s in s_db.table_names() if not \"_fts\" in s and not s.startswith(\"sqlite_\")]:\r\n log.info(\"[%s]: %s\", source_db, table)\r\n with s_db.conn:\r\n data = s_db[table].rows\r\n\r\n with args.db.conn:\r\n if args.upsert:\r\n args.db[table].upsert_all(data, pk=args.upsert.split(\",\"), alter=True)\r\n else:\r\n args.db[table].insert_all(data, alter=True, replace=True)\r\n\r\n\r\ndef merge_dbs():\r\n args = parse_args()\r\n for s_db in args.dbs:\r\n merge_db(args, s_db)\r\n\r\n\r\nif __name__ == \"__main__\":\r\n merge_dbs()\r\n\r\n```\r\n\r\n```\r\n$ lb-dev merge video.db tube_71.db --upsert path -vv\r\nSQL: INSERT OR IGNORE INTO [media]([path]) VALUES(?); - params: ['https://archive.org/details/088ghostofachanceroygetssackedrevengeofthelivinglunchdvdripxvidphz']\r\n...\r\nFile ~/.local/lib/python3.10/site-packages/sqlite_utils/db.py:3122, in Table.insert_all(self, records, pk, foreign_keys, column_order, not_null, defaults, batch_size, hash_id, hash_id_columns, alter, ignore, replace, truncate, extracts, conversions, columns, upsert, analyze)\r\n 3116 all_columns += [\r\n 3117 column for column in record if column not in all_columns\r\n 3118 ]\r\n 3120 first = False\r\n-> 3122 self.insert_chunk(\r\n 3123 alter,\r\n 3124 extracts,\r\n 3125 chunk,\r\n 3126 all_columns,\r\n 3127 hash_id,\r\n 3128 hash_id_columns,\r\n 3129 upsert,\r\n 3130 pk,\r\n 3131 conversions,\r\n 3132 num_records_processed,\r\n 3133 replace,\r\n 3134 ignore,\r\n 3135 )\r\n 3137 if analyze:\r\n 3138 self.analyze()\r\n\r\nFile ~/.local/lib/python3.10/site-packages/sqlite_utils/db.py:2887, in Table.insert_chunk(self, alter, extracts, chunk, all_columns, hash_id, hash_id_columns, upsert, pk, conversions, num_records_processed, replace, ignore)\r\n 2885 for query, params in queries_and_params:\r\n 2886 try:\r\n-> 2887 result = self.db.execute(query, params)\r\n 2888 except OperationalError as e:\r\n 2889 if alter and (\" column\" in e.args[0]):\r\n 2890 # Attempt to add any missing columns, then try again\r\n\r\nFile ~/.local/lib/python3.10/site-packages/sqlite_utils/db.py:484, in Database.execute(self, sql, parameters)\r\n 482 self._tracer(sql, parameters)\r\n 483 if parameters is not None:\r\n--> 484 return self.conn.execute(sql, parameters)\r\n 485 else:\r\n 486 return self.conn.execute(sql)\r\n\r\nIntegrityError: constraint failed\r\n> /home/xk/.local/lib/python3.10/site-packages/sqlite_utils/db.py(484)execute()\r\n 482 self._tracer(sql, parameters)\r\n 483 if parameters is not None:\r\n--> 484 return self.conn.execute(sql, parameters)\r\n 485 else:\r\n 486 return self.conn.execute(sql)\r\n```\r\n\r\n```\r\nsqlite3 --version\r\n3.36.0 2021-06-18 18:36:39\r\n```", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/511/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 473083260, "node_id": "MDU6SXNzdWU0NzMwODMyNjA=", "number": 50, "title": "\"Too many SQL variables\" on large inserts", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2019-07-25T21:43:31Z", "updated_at": "2022-11-04T14:38:36Z", "closed_at": "2019-07-28T11:59:33Z", "author_association": "OWNER", "pull_request": null, "body": "Reported here: https://github.com/dogsheep/healthkit-to-sqlite/issues/9\r\n\r\nIt looks like there's a default limit of 999 variables - we need to be smart about that, maybe dynamically lower the batch size based on the number of columns.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/50/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1429029604, "node_id": "I_kwDOCGYnMM5VLULk", "number": 506, "title": "Make `cursor.rowcount` accessible (wontfix)", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2022-10-30T21:51:55Z", "updated_at": "2022-11-01T17:37:47Z", "closed_at": "2022-11-01T17:37:13Z", "author_association": "OWNER", "pull_request": null, "body": "In building this Datasette feature on top of `sqlite-utils` I thought it might be useful to expose the number of rows that had been affected by a bulk insert or update - the `cursor.rowcount`:\r\n\r\n- https://github.com/simonw/datasette/issues/1866\r\n\r\nThis isn't currently exposed by `sqlite-utils`.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/506/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1431786951, "node_id": "I_kwDOBm6k_c5VV1XH", "number": 1876, "title": "SQL query should wrap on SQL interrupted screen", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2022-11-01T17:14:01Z", "updated_at": "2022-11-01T17:22:33Z", "closed_at": "2022-11-01T17:22:33Z", "author_association": "OWNER", "pull_request": null, "body": "Just saw this:\r\n\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1876/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1430325103, "node_id": "I_kwDOCGYnMM5VQQdv", "number": 507, "title": "conn.execute: UnicodeEncodeError: 'utf-8' codec can't encode character", "user": {"value": 7908073, "label": "chapmanjacobd"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2022-10-31T18:49:51Z", "updated_at": "2022-11-01T00:40:17Z", "closed_at": "2022-11-01T00:40:16Z", "author_association": "CONTRIBUTOR", "pull_request": null, "body": "I'm not really sure what caused this and it happened in the middle of my program (after running for 35775 seconds).\r\n\r\n```\r\nExtracting metadata 49.9% (chunk 9893 of 19831)\r\n...\r\n File \"/home/xk/.local/lib/python3.10/site-packages/xklb/fs_extract.py\", line 90, in extract_chunk\r\n args.db[\"media\"].insert_all(utils.list_dict_filter_bool(media), pk=\"path\", alter=True, replace=True)\r\n File \"/home/xk/.local/lib/python3.10/site-packages/sqlite_utils/db.py\", line 3107, in insert_all\r\n self.insert_chunk(\r\n File \"/home/xk/.local/lib/python3.10/site-packages/sqlite_utils/db.py\", line 2872, in insert_chunk\r\n result = self.db.execute(query, params)\r\n File \"/home/xk/.local/lib/python3.10/site-packages/sqlite_utils/db.py\", line 483, in execute\r\n return self.conn.execute(sql, parameters)\r\nUnicodeEncodeError: 'utf-8' codec can't encode character '\\udcc3' in position 62: surrogates not allowed\r\n```\r\n\r\nThis might be relevant: https://stackoverflow.com/questions/31898353/python-cant-encode-with-surrogateescape\r\n\r\nI'm going to try re-running with \r\n\r\n```py\r\n def execute(\r\n self, sql: str, parameters: Optional[Union[Iterable, dict]] = None\r\n ) -> sqlite3.Cursor:\r\n \"\"\"\r\n Execute SQL query and return a ``sqlite3.Cursor``.\r\n\r\n :param sql: SQL query to execute\r\n :param parameters: Parameters to use in that query - an iterable for ``where id = ?``\r\n parameters, or a dictionary for ``where id = :id``\r\n \"\"\"\r\n try:\r\n if self._tracer:\r\n self._tracer(sql, parameters)\r\n if parameters is not None:\r\n return self.conn.execute(sql, parameters)\r\n else:\r\n return self.conn.execute(sql)\r\n except UnicodeEncodeError:\r\n sql = sql.encode('utf-8', 'surrogatepass').decode('utf-8')\r\n if parameters is not None:\r\n parameters = parameters.encode('utf-8', 'surrogatepass').decode('utf-8')\r\n return self.execute(sql, parameters)\r\n```", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/507/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1428560020, "node_id": "I_kwDOBm6k_c5VJhiU", "number": 1872, "title": "SITE-BUSTING ERROR: \"render_template() called before await ds.invoke_startup()\"", "user": {"value": 192568, "label": "mroswell"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2022-10-30T02:28:39Z", "updated_at": "2022-10-30T06:26:01Z", "closed_at": "2022-10-30T06:26:01Z", "author_association": "CONTRIBUTOR", "pull_request": null, "body": "1. My https://list.saferdisinfectants.org/disinfectants/listN page (linked from https://SaferDisinfectants.org ) has been running beautifully for a year and a half, including a GitHub Actions workflow that's been routinely updating the database.\r\n\r\n2. I received a recent report that the list page is down. I don't know when it went down, but the content is replaced with: \"render_template() called before await ds.invoke_startup()\"\r\n\r\n3. The local datasette repo runs without incident.\r\n\r\n4. The site is hosted on vercel, linked to my github repo. Perhaps some vercel changes were made, but not by anyone on our side. Here is a screenshot of the current project settings:\r\n\r\n\r\n\r\nHere a screenshot of the latest deployment status:\r\n\r\n\r\nThis is my repository:\r\nhttps://github.com/mroswell/list-N\r\n(I notice: datasette==0.59 in my requirements.txt file)\r\n\r\nBecause it's been long while since I actively worked on this or any other datasette project, I forget a lot of what I knew at one point. Perhaps some configuration file could be missing? Or perhaps I just need to know the right incantation to add to that vercel settings page. \r\n\r\nHelp is welcome as the nonprofit org is soon hosting its annual conference, and we'd love to have the page working again.\r\n\r\n\r\n\r\n\r\n\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1872/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1426253476, "node_id": "I_kwDOBm6k_c5VAuak", "number": 1869, "title": "Release 0.63", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2022-10-27T20:53:01Z", "updated_at": "2022-10-27T22:24:38Z", "closed_at": "2022-10-27T22:11:33Z", "author_association": "OWNER", "pull_request": null, "body": "Most of the release notes are already written:\r\n- https://github.com/simonw/datasette/releases/tag/0.63a0\r\n- https://github.com/simonw/datasette/releases/tag/0.63a1", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1869/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1342430983, "node_id": "I_kwDOBm6k_c5QA98H", "number": 1786, "title": "Adjust height of textarea for no JS case", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2022-08-18T01:15:15Z", "updated_at": "2022-10-27T21:50:12Z", "closed_at": "2022-08-18T16:06:09Z", "author_association": "OWNER", "pull_request": null, "body": "Datasette Lite: https://lite.datasette.io/?sql=https://gist.githubusercontent.com/simonw/1f8a91123ccefd8844187225b1832d7a/raw/5069075b86aa79358fbab3d4482d1d269077d632/recipes.sql#/data?sql=select+id%2C+name%2C+ingredients%2C+%28%0A++select+json_group_array%28value%29+from+json_each%28ingredients%29%0A++where+value+in+%28select+value+from+json_each%28%3Ap0%29%29%0A%29+as+matching_ingredients%0Afrom+recipes%0Awhere+json_array_length%28matching_ingredients%29+%3E+0%0Aorder+by+json_array_length%28matching_ingredients%29+desc&p0=%5B%22sugar%22%2C+%22cheese%22%5D\r\n\r\n![46F8101E-8CE3-4F61-B200-F865E6B5DBCC](https://user-images.githubusercontent.com/9599/185270723-f55513b0-b561-434d-9d7c-4fe5be9756e0.jpeg)\r\n\r\n\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1786/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1401155623, "node_id": "PR_kwDOBm6k_c5AZLzm", "number": 1839, "title": "Bump black from 22.8.0 to 22.10.0", "user": {"value": 49699333, "label": "dependabot[bot]"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2022-10-07T13:13:41Z", "updated_at": "2022-10-27T20:51:46Z", "closed_at": "2022-10-27T20:51:45Z", "author_association": "CONTRIBUTOR", "pull_request": "simonw/datasette/pulls/1839", "body": "Bumps [black](https://github.com/psf/black) from 22.8.0 to 22.10.0.\n\nRelease notes
\nSourced from black's releases.
\n\n22.10.0
\nHighlights
\n\n- Runtime support for Python 3.6 has been removed. Formatting 3.6 code will still be\nsupported until further notice.
\n
\nStable style
\n\n- Fix a crash when
# fmt: on
is used on a different block level than # fmt: off
\n(#3281) \n
\nPreview style
\n\n- Fix a crash when formatting some dicts with parenthesis-wrapped long string keys\n(#3262)
\n
\nConfiguration
\n\n.ipynb_checkpoints
directories are now excluded by default (#3293) \n- Add
--skip-source-first-line
/ -x
option to ignore the first line of source code\nwhile formatting (#3299) \n
\nPackaging
\n\n- Executables made with PyInstaller will no longer crash when formatting several files\nat once on macOS. Native x86-64 executables for macOS are available once again.\n(#3275)
\n- Hatchling is now used as the build backend. This will not have any effect for users\nwho install Black with its wheels from PyPI. (#3233)
\n- Faster compiled wheels are now available for CPython 3.11 (#3276)
\n
\nBlackd
\n\n- Windows style (CRLF) newlines will be preserved (#3257).
\n
\nIntegrations
\n\n- Vim plugin: add flag (
g:black_preview
) to enable/disable the preview style (#3246) \n- Update GitHub Action to support formatting of Jupyter Notebook files via a
jupyter
\noption (#3282) \n- Update GitHub Action to support use of version specifiers (e.g.
<23
) for Black\nversion (#3265) \n
\n
\n \n\nChangelog
\nSourced from black's changelog.
\n\n22.10.0
\nHighlights
\n\n- Runtime support for Python 3.6 has been removed. Formatting 3.6 code will still be\nsupported until further notice.
\n
\nStable style
\n\n- Fix a crash when
# fmt: on
is used on a different block level than # fmt: off
\n(#3281) \n
\nPreview style
\n\n- Fix a crash when formatting some dicts with parenthesis-wrapped long string keys\n(#3262)
\n
\nConfiguration
\n\n.ipynb_checkpoints
directories are now excluded by default (#3293) \n- Add
--skip-source-first-line
/ -x
option to ignore the first line of source code\nwhile formatting (#3299) \n
\nPackaging
\n\n- Executables made with PyInstaller will no longer crash when formatting several files\nat once on macOS. Native x86-64 executables for macOS are available once again.\n(#3275)
\n- Hatchling is now used as the build backend. This will not have any effect for users\nwho install Black with its wheels from PyPI. (#3233)
\n- Faster compiled wheels are now available for CPython 3.11 (#3276)
\n
\nBlackd
\n\n- Windows style (CRLF) newlines will be preserved (#3257).
\n
\nIntegrations
\n\n- Vim plugin: add flag (
g:black_preview
) to enable/disable the preview style (#3246) \n- Update GitHub Action to support formatting of Jupyter Notebook files via a
jupyter
\noption (#3282) \n- Update GitHub Action to support use of version specifiers (e.g.
<23
) for Black\nversion (#3265) \n
\n
\n \n\nCommits
\n\n \n
\n\n\n[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=black&package-manager=pip&previous-version=22.8.0&new-version=22.10.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)\n\nDependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.\n\n[//]: # (dependabot-automerge-start)\n[//]: # (dependabot-automerge-end)\n\n---\n\n\nDependabot commands and options
\n
\n\nYou can trigger Dependabot actions by commenting on this PR:\n- `@dependabot rebase` will rebase this PR\n- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it\n- `@dependabot merge` will merge this PR after your CI passes on it\n- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it\n- `@dependabot cancel merge` will cancel a previously requested merge and block automerging\n- `@dependabot reopen` will reopen this PR if it is closed\n- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually\n- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)\n- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)\n- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)\n\n\n \r\n\r\n\r\n----\n:books: Documentation preview :books:: https://datasette--1839.org.readthedocs.build/en/1839/\n\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1839/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 1400121355, "node_id": "PR_kwDOBm6k_c5AVujU", "number": 1835, "title": "use inspect data for hash and file size", "user": {"value": 536941, "label": "fgregg"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2022-10-06T18:25:24Z", "updated_at": "2022-10-27T20:51:30Z", "closed_at": "2022-10-06T20:06:07Z", "author_association": "CONTRIBUTOR", "pull_request": "simonw/datasette/pulls/1835", "body": "`inspect_data` should already include the hash and the db file size, so this PR takes advantage of using those instead of always recalculating. should help a lot on startup with large DBs.\r\n\r\ncloses #1834 ", "repo": {"value": 107914493, "label": "datasette"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1835/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 1400431789, "node_id": "PR_kwDOBm6k_c5AWyQK", "number": 1837, "title": "Make hash and size a lazy property", "user": {"value": 536941, "label": "fgregg"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2022-10-06T23:51:22Z", "updated_at": "2022-10-27T20:51:21Z", "closed_at": "2022-10-27T20:51:20Z", "author_association": "CONTRIBUTOR", "pull_request": "simonw/datasette/pulls/1837", "body": "Many apologies, @simonw. My previous PR #1835 did not really solve the problem because the name of the database is often not known to database object in the init method.\r\n\r\nI took a cue from how you dealt with this issue and made hash a lazy property and did something similar with size.\r\n\r\n\r\n----\n:books: Documentation preview :books:: https://datasette--1837.org.readthedocs.build/en/1837/\n\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1837/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 1424378012, "node_id": "I_kwDOBm6k_c5U5kic", "number": 1860, "title": "SQL query field can't begin by a comment", "user": {"value": 562352, "label": "CharlesNepote"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 12, "created_at": "2022-10-26T16:55:31Z", "updated_at": "2022-10-27T18:57:37Z", "closed_at": "2022-10-27T04:21:40Z", "author_association": "NONE", "pull_request": null, "body": "![image](https://user-images.githubusercontent.com/562352/198085197-f26fcd61-4dac-4ca4-a346-e70f88a30ecc.png)\r\n\r\nSQL comments are **very** useful to explain the meaning of the query. It's currently impossible to put it at the beginning of the field as seen on the screen capture: it leads to an error: `Statement must be a SELECT`.\r\n\r\nIt would be great to make it possible because:\r\n* as the request is the title of the page:\r\n * it eases the search with search engines\r\n * it eases the search in the browsers' url field\r\n* it acts as a kind of title: the global meaning of the query is immediately understandable\r\n* some tools, such as Slack, are shortening long URLs and displaying the beginning of the URLs (eg. `https://example.org/products?sql=select+%28length%28data_quality_errors_ta[...]+%21%3D+%22%22+group+by+NB_of_issues+order+by+NB_of_issues+desc+limit+200`)\r\n\r\nBeginning a query with a comment is possible with SQLite.\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1860/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1425682079, "node_id": "I_kwDOBm6k_c5U-i6f", "number": 1865, "title": "Stop syncing main to master", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2022-10-27T13:55:38Z", "updated_at": "2022-10-27T13:58:27Z", "closed_at": "2022-10-27T13:56:13Z", "author_association": "OWNER", "pull_request": null, "body": "I think it's been long enough now that I can drop the code that syncs the main branch to master.\r\n\r\nI originally added this for people who might be using `datasette publish ... --branch master` - which might only have been me anyway!", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1865/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 639072811, "node_id": "MDU6SXNzdWU2MzkwNzI4MTE=", "number": 849, "title": "Rename master branch to main", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 3268330, "label": "Datasette 1.0"}, "comments": 10, "created_at": "2020-06-15T19:05:54Z", "updated_at": "2022-10-27T13:57:08Z", "closed_at": "2020-09-15T20:37:14Z", "author_association": "OWNER", "pull_request": null, "body": "I was waiting for consensus to form around this (and kind-of hoping for `trunk` since I like the tree metaphor) and it looks like `main` is it.\r\n\r\nI've seen convincing arguments against `trunk` too - it indicates that the branch has some special significance like in Subversion (where all branches come from trunk) when it doesn't. So `main` is better anyway.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/849/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1420174670, "node_id": "I_kwDOBm6k_c5UpiVO", "number": 1849, "title": "NoneType' object has no attribute 'actor'", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 5, "created_at": "2022-10-24T04:02:15Z", "updated_at": "2022-10-26T21:13:40Z", "closed_at": "2022-10-26T21:13:40Z", "author_association": "OWNER", "pull_request": null, "body": "```\r\n File \"/usr/local/lib/python3.10/site-packages/datasette/templates/_crumbs.html\", line 3, in template\r\n {% set items=crumb_items(request=request, database=database, table=table) %}\r\n File \"jinja2/async_utils.py\", line 65, in auto_await\r\n return await t.cast(\"t.Awaitable[V]\", value)\r\n File \"datasette/app.py\", line 638, in _crumb_items\r\n actor=request.actor, action=\"view-instance\", default=True\r\n```\r\nFrom Sentry.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1849/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 642297505, "node_id": "MDU6SXNzdWU2NDIyOTc1MDU=", "number": 857, "title": "Comprehensive documentation for variables made available to templates", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 3268330, "label": "Datasette 1.0"}, "comments": 1, "created_at": "2020-06-20T03:19:43Z", "updated_at": "2022-10-26T02:58:17Z", "closed_at": "2022-10-26T02:58:17Z", "author_association": "OWNER", "pull_request": null, "body": "Needed for the Datasette 1.0 release, so template authors can trust that Datasette is unlikely to break their templates.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/857/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1423182778, "node_id": "I_kwDOCGYnMM5U1Au6", "number": 505, "title": "Release sqlite-utils 3.30", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2022-10-25T22:20:05Z", "updated_at": "2022-10-25T22:41:26Z", "closed_at": "2022-10-25T22:41:16Z", "author_association": "OWNER", "pull_request": null, "body": "https://github.com/simonw/sqlite-utils/compare/3.29...defa2974c6d3abc19be28d6b319649b8028dc966", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/505/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1392690202, "node_id": "I_kwDOCGYnMM5TAsQa", "number": 495, "title": "Support JSON values returned from .convert() functions", "user": {"value": 649467, "label": "mhalle"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2022-09-30T16:33:49Z", "updated_at": "2022-10-25T21:23:37Z", "closed_at": "2022-10-25T21:23:28Z", "author_association": "NONE", "pull_request": null, "body": "When using the convert function on a JSON column, the result of the conversion function must be a string. If the return value is either a dict (object) or a list (array), the convert call will error out with an unhelpful user defined function exception. \r\n\r\nIt makes sense that since the original column value was a string and required conversion to data structures, the result should be converted back into a JSON string as well. However, other functions auto-convert to JSON string representation, so the fact that convert doesn't could be surprising.\r\n\r\nAt least the documentation should note this requirement, because the sqlite error messages won't readily reveal the issue.\r\n\r\nJf only sqlite's JSON column type meant something :)", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/495/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1393212964, "node_id": "I_kwDOCGYnMM5TCr4k", "number": 497, "title": "column_names", "user": {"value": 7908073, "label": "chapmanjacobd"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2022-10-01T03:34:21Z", "updated_at": "2022-10-25T21:09:28Z", "closed_at": "2022-10-25T21:09:28Z", "author_association": "CONTRIBUTOR", "pull_request": null, "body": "It would be nice to have a `column_names`. Similar to `table_names`.\r\n\r\nOr if you could get one or all of the following syntax to work for both Database and Table that might be even better: \r\n\r\nStyle 1\r\n- `if 'table1' in db`\r\n- `if 'col1' in db['table1']`\r\n\r\nStyle 2\r\n- `if 'table1' in db.tables`\r\n- `if 'col1' in db['table1'].columns`\r\n\r\nmaybe the table ones actually work but I'm too lazy to check. I just know that I have to do:\r\n\r\n `[c.name for c in db['table1'].columns]`\r\n\r\nEdit: This is possible with `columns_dict`. I have actually used that before but I forgot about it. Feel free to close, but I do think accessing this data could be more consistent and intuitive.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/497/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1423069384, "node_id": "I_kwDOCGYnMM5U0lDI", "number": 504, "title": "db.close() method, calling db.conn.close()", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2022-10-25T20:50:50Z", "updated_at": "2022-10-25T21:00:29Z", "closed_at": "2022-10-25T20:57:47Z", "author_association": "OWNER", "pull_request": null, "body": "I ended up needing to use `db.conn.close()` to fix this issue:\r\n- #503\r\n\r\nI think `.close()` should be a method on `Database` itself.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/504/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1423000702, "node_id": "I_kwDOCGYnMM5U0UR-", "number": 503, "title": "test_recreate failing on Windows Python 3.11", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 10, "created_at": "2022-10-25T20:01:41Z", "updated_at": "2022-10-25T20:47:34Z", "closed_at": "2022-10-25T20:45:43Z", "author_association": "OWNER", "pull_request": null, "body": "https://github.com/simonw/sqlite-utils/actions/runs/3323672128/jobs/5494726927\r\n\r\nRelated:\r\n- #502\r\n\r\n```\r\nFAILED tests/test_recreate.py::test_recreate[True-True] - \r\n PermissionError: [WinError 32] The process cannot access the file because it is being used by another process:\r\n 'C:\\\\Users\\\\runneradmin\\\\AppData\\\\Local\\\\Temp\\\\pytest-of-runneradmin\\\\pytest-0\\\\test_recreate_True_True_0\\\\data.db'\r\nFAILED tests/test_recreate.py::test_recreate[False-True] - \r\n PermissionError: [WinError 32] The process cannot access the file because it is being used by another process:\r\n 'C:\\\\Users\\\\runneradmin\\\\AppData\\\\Local\\\\Temp\\\\pytest-of-runneradmin\\\\pytest-0\\\\test_recreate_False_True_0\\\\data.db'\r\n```", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/503/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1422973111, "node_id": "I_kwDOBm6k_c5U0Ni3", "number": 1854, "title": "Flaky test: test_serve_localhost_http", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2022-10-25T19:37:35Z", "updated_at": "2022-10-25T19:53:02Z", "closed_at": "2022-10-25T19:53:02Z", "author_association": "OWNER", "pull_request": null, "body": "Failing on Python 3.10 at the moment: https://github.com/simonw/datasette/actions/runs/3323629947/jobs/5494340302", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1854/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1422915587, "node_id": "I_kwDOBm6k_c5Uz_gD", "number": 1853, "title": "Upgrade Datasette Docker to Python 3.11", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 7, "created_at": "2022-10-25T18:44:31Z", "updated_at": "2022-10-25T19:28:56Z", "closed_at": "2022-10-25T19:05:16Z", "author_association": "OWNER", "pull_request": null, "body": "Related:\r\n- #1768\r\n\r\nI think this base image looks right: [3.11.0-slim-bullseye](https://hub.docker.com/layers/library/python/3.11.0-slim-bullseye/images/sha256-244c0b0e6e7608a16f87382fc8a5ef3c330d042113a9a7b6fc15a95360181651?context=explore)", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1853/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1422954582, "node_id": "I_kwDOCGYnMM5U0JBW", "number": 502, "title": "Fix tests for Python 3.11", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2022-10-25T19:20:31Z", "updated_at": "2022-10-25T19:23:47Z", "closed_at": "2022-10-25T19:23:47Z", "author_association": "OWNER", "pull_request": null, "body": "The way errors are represented has changed: https://github.com/simonw/sqlite-utils/actions/runs/3323588047/jobs/5494127154\r\n```\r\n_________________________ test_query_invalid_function __________________________\r\n\r\ndb_path = '/tmp/pytest-of-runner/pytest-0/test_query_invalid_function0/test.db'\r\n\r\n def test_query_invalid_function(db_path):\r\n result = CliRunner().invoke(\r\n cli.cli, [db_path, \"select bad()\", \"--functions\", \"def invalid_python\"]\r\n )\r\n assert result.exit_code == 1\r\n> assert (\r\n result.output.strip()\r\n == \"Error: Error in functions definition: invalid syntax (, line 1)\"\r\n )\r\nE AssertionError: assert 'Error: Error...ing>, line 1)' == 'Error: Error...ing>, line 1)'\r\nE - Error: Error in functions definition: invalid syntax (, line 1)\r\nE ? ^^^^^^ ^^^^^^\r\nE + Error: Error in functions definition: expected '(' (, line 1)\r\nE ? ^^^^^^^ ^^^\r\n```", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/502/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1420090659, "node_id": "I_kwDOBm6k_c5UpN0j", "number": 1848, "title": "Private database page should show padlock on every table", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2022-10-24T02:28:38Z", "updated_at": "2022-10-24T02:50:29Z", "closed_at": "2022-10-24T02:42:34Z", "author_association": "OWNER", "pull_request": null, "body": "Following:\r\n- #1829\r\n\r\nhttps://latest.datasette.io/_internal looks like this:\r\n\r\n\r\n\r\nBut those queries and tables are private too, and should also show the padlock icon.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1848/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1396948693, "node_id": "I_kwDOBm6k_c5TQ77V", "number": 1829, "title": "Table/database that is private due to inherited permissions does not show padlock", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 8, "created_at": "2022-10-04T23:14:16Z", "updated_at": "2022-10-24T02:23:46Z", "closed_at": "2022-10-24T02:11:37Z", "author_association": "OWNER", "pull_request": null, "body": "I noticed that a table page that is private because the database or instance is private, e.g. this one:\r\n\r\n\r\n\r\nIs not displaying the padlock icon that indicates the table is not visible to the public.\r\n\r\n\r\n\r\nSame issue for the database page too, which in this case is private due to `view-instance`.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1829/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1408561039, "node_id": "PR_kwDOBm6k_c5Axrpb", "number": 1842, "title": "check_visibility can now take multiple permissions into account", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2022-10-14T00:06:04Z", "updated_at": "2022-10-24T02:11:36Z", "closed_at": "2022-10-24T02:11:36Z", "author_association": "OWNER", "pull_request": "simonw/datasette/pulls/1842", "body": "Refs #1829\r\n\r\n- [x] Fix table page\r\n- [x] Fix database page\r\n- [x] Fix query page\r\n- [x] Fix row page\r\n- [x] Tests\r\n- [x] Updated documentation for `check_visibility` method, to cover the new `permissions=` keyword argument\r\n\r\nAlso this fix is currently only applied on the table page - needs to be applied on database, row and query pages too.\r\n\r\n\r\n----\r\n:books: Documentation preview :books:: https://datasette--1842.org.readthedocs.build/en/1842/\r\n\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1842/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}