{"id": 1205687423, "node_id": "I_kwDOCGYnMM5H3VR_", "number": 426, "title": "CLI docs should link to Python docs and vice versa", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": {"value": 9599, "label": "simonw"}, "milestone": null, "comments": 1, "created_at": "2022-04-15T16:05:15Z", "updated_at": "2023-07-22T22:13:22Z", "closed_at": "2023-07-22T22:13:22Z", "author_association": "OWNER", "pull_request": null, "body": "For every command/API method there should be a link to the equivalent in the other form factor.\r\n\r\nMaybe also link to the API and CLI reference pages too.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/426/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1786243905, "node_id": "I_kwDOCGYnMM5qd-tB", "number": 564, "title": "Document that running `db.transform()` tidies up the schema indentation", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2023-07-03T13:59:28Z", "updated_at": "2023-07-22T22:15:34Z", "closed_at": "2023-07-22T22:15:34Z", "author_association": "OWNER", "pull_request": null, "body": "> ... and it turns out running `.transform()` with no arguments still fixes the format of the schema!\r\n\r\n```pycon\r\n>>> db[\"log\"].add_column(\"foo\", str)\r\n<Table log (id, name2, age, weight, foo)>\r\n>>> db[\"log\"].add_column(\"bar\", str)\r\n<Table log (id, name2, age, weight, foo, bar)>\r\n>>> db[\"log\"].add_column(\"baz\", str)\r\n<Table log (id, name2, age, weight, foo, bar, baz)>\r\n>>> print(db[\"log\"].schema)\r\nCREATE TABLE \"log\" (\r\n   [id] INTEGER PRIMARY KEY,\r\n   [name2] TEXT,\r\n   [age] INTEGER,\r\n   [weight] FLOAT\r\n, [foo] TEXT, [bar] TEXT, [baz] TEXT)\r\n>>> db[\"log\"].transform()\r\n<Table log (id, name2, age, weight, foo, bar, baz)>\r\n>>> print(db[\"log\"].schema)\r\nCREATE TABLE \"log\" (\r\n   [id] INTEGER PRIMARY KEY,\r\n   [name2] TEXT,\r\n   [age] INTEGER,\r\n   [weight] FLOAT,\r\n   [foo] TEXT,\r\n   [bar] TEXT,\r\n   [baz] TEXT\r\n)\r\n```\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/llm/issues/65#issuecomment-1618347727_\r\n            ", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/564/reactions\", \"total_count\": 1, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 1}", "draft": null, "state_reason": "completed"}
{"id": 1786258502, "node_id": "I_kwDOCGYnMM5qeCRG", "number": 565, "title": "Table renaming: db.rename_table() and sqlite-utils rename-table", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2023-07-03T14:07:42Z", "updated_at": "2023-07-22T22:12:40Z", "closed_at": "2023-07-22T22:12:40Z", "author_association": "OWNER", "pull_request": null, "body": "> I find myself wanting two new features in `sqlite-utils`:\r\n> - The ability to have the new transformed table set to a specific name, while keeping the old table around\r\n> - The ability to rename a table (`sqlite-utils` doesn't have a table rename function at all right now)\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/llm/issues/65#issuecomment-1618375042_\r\n            ", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/565/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1801394744, "node_id": "I_kwDOCGYnMM5rXxo4", "number": 567, "title": "Plugin system", "user": {"value": 15178711, "label": "asg017"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 9, "created_at": "2023-07-12T17:02:14Z", "updated_at": "2023-07-22T22:59:37Z", "closed_at": "2023-07-22T22:59:36Z", "author_association": "CONTRIBUTOR", "pull_request": null, "body": "I'd like there to be a plugin system for sqlite-utils, similar to the datasette/llm plugins. I'd like to make plugins that would do things like:\r\n\r\n- Register SQLite extensions for more SQL functions + virtual tables\r\n- Register new subcommands\r\n- Different input file formats for `sqlite-utils memory`\r\n- Different output file formats (in addition to `--csv` `--tsv` `--nl` etc.\r\n\r\nA few real-world use-cases of plugins I'd like to see in sqlite-utils:\r\n\r\n- Register many of my sqlite extensions in sqlite-utils (`sqlite-http`, `sqlite-lines`, `sqlite-regex`, etc.)\r\n- New subcommands to work with `sqlite-vss` vector tables\r\n- Input/ouput Parquet/Avro/Arrow IPC files with `sqlite-arrow`", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/567/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1816830546, "node_id": "I_kwDODEm0Qs5sSqJS", "number": 73, "title": "Twitter v1 API shutdown", "user": {"value": 6341745, "label": "david-perez"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2023-07-22T16:57:41Z", "updated_at": "2023-07-22T16:57:41Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "I've been using this project reliably over the past two years to periodically download my liked tweets, but unfortunately since 19th July I get:\r\n\r\n```\r\n[2023-07-19 21:00:04.937536]   File \"/home/pi/code/liked-tweets/lib/python3.7/site-packages/twitter_to_sqlite/utils.py\", line 202, in fetch_timeline\r\n[2023-07-19 21:00:04.937606]     raise Exception(str(tweets[\"errors\"]))\r\n[2023-07-19 21:00:04.937678] Exception: [{'message': 'You currently have access to a subset of Twitter API v2 endpoints and limited v1.1 endpoints (e.g. media post, oauth) only. If you need access to this endpoint, you may need a different access level. You can learn more here: https://developer.twitter.com/en/portal/product', 'code': 453}]\r\n```\r\n\r\nIt appears like Twitter has now shut down their v1 endpoints, which is rather gracious of them, considering they [announced they'd be deprecated on 29th April](https://twittercommunity.com/t/reminder-to-migrate-to-the-new-free-basic-or-enterprise-plans-of-the-twitter-api/189737).\r\n\r\nUnfortunately [retrieving likes using the v2 API](https://developer.twitter.com/en/docs/twitter-api/tweets/likes/introduction) is not part of their [free plan](https://developer.twitter.com/en/portal/products). In fact, with the free plan one can only post and delete tweets and retrieve information about oneself.\r\n\r\nSo I'm afraid this is the end of this very nice project. It was very useful, thank you!\r\n", "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/73/reactions\", \"total_count\": 1, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 1}", "draft": null, "state_reason": null}
{"id": 1816851056, "node_id": "I_kwDOCGYnMM5sSvJw", "number": 568, "title": "table.create(..., replace=True)", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 7, "created_at": "2023-07-22T18:12:22Z", "updated_at": "2023-07-22T19:25:35Z", "closed_at": "2023-07-22T19:15:44Z", "author_association": "OWNER", "pull_request": null, "body": "Found myself using this pattern to quickly prototype a schema:\r\n\r\n```python\r\nimport sqlite_utils\r\ndb = sqlite_utils.Database(memory=True)\r\n\r\nprint(db[\"answers_chunks\"].create({\r\n    \"id\": int,\r\n    \"content\": str,\r\n    \"embedding_type_id\": int,\r\n    \"embedding\": bytes,\r\n    \"embedding_content_md5\": str,\r\n    \"source\": str,\r\n}, pk=\"id\", transform=True).schema)\r\n```\r\n\r\n<img width=\"472\" alt=\"image\" src=\"https://github.com/simonw/datasette-answers/assets/9599/30237f7c-4964-4190-984c-d824c57939a9\">\r\n\r\nUsing `replace=True` to drop and then recreate the table would be neat here, and would be consistent with other places that use `replace=True`.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/568/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1816852402, "node_id": "I_kwDOCGYnMM5sSvey", "number": 569, "title": "register_command plugin hook", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2023-07-22T18:17:27Z", "updated_at": "2023-07-22T19:19:35Z", "closed_at": "2023-07-22T19:19:35Z", "author_association": "OWNER", "pull_request": null, "body": "> I'm going to start by adding the `register_command` hook using the exact same pattern as Datasette and LLM.\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/567#issuecomment-1646643450_\r\n            ", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/569/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1816857105, "node_id": "I_kwDOCGYnMM5sSwoR", "number": 570, "title": "`sqlite-utils install -e` option", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2023-07-22T18:32:23Z", "updated_at": "2023-07-22T18:55:59Z", "closed_at": "2023-07-22T18:32:56Z", "author_association": "OWNER", "pull_request": null, "body": "As seen in LLM.\r\n\r\nNeeded while working on:\r\n- #567", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/570/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1816876211, "node_id": "I_kwDOCGYnMM5sS1Sz", "number": 571, "title": "`.transform(keep_table=...)` option", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2023-07-22T19:49:29Z", "updated_at": "2023-07-22T22:32:18Z", "closed_at": "2023-07-22T22:32:18Z", "author_association": "OWNER", "pull_request": null, "body": ">> Also need a design for an option for the `.transform()` method to indicate that the new table should be created with a new name without dropping the old one.\r\n>\r\n> I think `keep_table=\"name_of_table\"` is good for this.\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/565#issuecomment-1646657324_\r\n            ", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/571/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1816877910, "node_id": "I_kwDOCGYnMM5sS1tW", "number": 572, "title": "Don't test Python 3.7 against textual", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2023-07-22T19:57:03Z", "updated_at": "2023-07-22T22:16:50Z", "closed_at": "2023-07-22T22:16:50Z", "author_association": "OWNER", "pull_request": null, "body": "Spotted this in the GitHub Actions logs:\r\n\r\n![IMG_5046](https://github.com/simonw/sqlite-utils/assets/9599/81fb1093-cd8a-4019-a612-2e49b500c933)\r\n", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/572/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1816917522, "node_id": "PR_kwDOCGYnMM5WJ6Jm", "number": 573, "title": "feat: Implement a prepare_connection plugin hook", "user": {"value": 15178711, "label": "asg017"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2023-07-22T22:48:44Z", "updated_at": "2023-07-22T22:59:09Z", "closed_at": "2023-07-22T22:59:09Z", "author_association": "CONTRIBUTOR", "pull_request": "simonw/sqlite-utils/pulls/573", "body": "Just like the [Datasette prepare_connection hook](https://docs.datasette.io/en/stable/plugin_hooks.html#prepare-connection-conn-database-datasette), this PR adds a similar hook for the `sqlite-utils` plugin system. \r\n\r\nThe sole argument is `conn`, since I don't believe a `database` or `datasette` argument would be relevant here. \r\n\r\nI want to do this so I can release `sqlite-utils` plugins for my [SQLite extensions](https://github.com/asg017/sqlite-ecosystem), similar to the Datasette plugins I've release for them. \r\n\r\nAn example plugin: https://gist.github.com/asg017/d7cdf0d56e2be87efda28cebee27fa3c\r\n\r\n```bash\r\n$ sqlite-utils install https://gist.github.com/asg017/d7cdf0d56e2be87efda28cebee27fa3c/archive/5f5ad549a40860787629c69ca120a08c32519e99.zip\r\n\r\n$ sqlite-utils memory 'select hello(\"alex\") as response'\r\n[{\"response\": \"Hello, alex!\"}]\r\n```\r\nRefs:\r\n- #574 \r\n\r\n<!-- readthedocs-preview sqlite-utils start -->\r\n----\r\n:books: Documentation preview :books:: https://sqlite-utils--573.org.readthedocs.build/en/573/\r\n\r\n<!-- readthedocs-preview sqlite-utils end -->", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/573/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 1816918185, "node_id": "I_kwDOCGYnMM5sS_ip", "number": 574, "title": "`prepare_connection()` plugin hook", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2023-07-22T22:52:47Z", "updated_at": "2023-07-22T23:13:14Z", "closed_at": "2023-07-22T22:59:10Z", "author_association": "OWNER", "pull_request": null, "body": "> Splitting off an issue for `prepare_connection()` since Alex got the PR in seconds before I shipped 3.34!\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/567#issuecomment-1646686424_\r\n            ", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/574/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1816919568, "node_id": "I_kwDOCGYnMM5sS_4Q", "number": 575, "title": "Python API ability to opt-out of connection plugins", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2023-07-22T23:01:13Z", "updated_at": "2023-07-22T23:17:22Z", "closed_at": "2023-07-22T23:08:22Z", "author_association": "OWNER", "pull_request": null, "body": "Plugins affecting the CLI by default makes sense to me.\r\n\r\nI'm less confident about them _always_ affecting users of the Python API.\r\n\r\nI'm going to have them apply by default, but I'm going to add a mechanism to opt-out on an individual database basis. Basically this:\r\n\r\n```python\r\nfrom sqlite_utils import Database\r\ndb = Database(memory=True, execute_plugins=False)\r\n# Anything using db from here on will not execute plugins\r\n```\r\ncc @asg017 \r\n\r\nRefs:\r\n- #567 \r\n- #574 ", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/575/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}