{"id": 347058326, "node_id": "MDExOlB1bGxSZXF1ZXN0MjA1NzcwOTk2", "number": 1, "title": "Make .indexes compatible with older SQLite versions", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2018-08-02T15:17:05Z", "updated_at": "2018-08-02T15:17:30Z", "closed_at": "2018-08-02T15:17:30Z", "author_association": "OWNER", "pull_request": "simonw/sqlite-utils/pulls/1", "body": "Older SQLite versions return a different set of columns from the PRAGMA we are using.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/1/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null} {"id": 403028630, "node_id": "MDExOlB1bGxSZXF1ZXN0MjQ3NTc2OTQy", "number": 4, "title": "Fts5", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2019-01-25T06:54:05Z", "updated_at": "2019-01-25T06:54:33Z", "closed_at": "2019-01-25T06:54:33Z", "author_association": "OWNER", "pull_request": "simonw/sqlite-utils/pulls/4", "body": "", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/4/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null} {"id": 403396009, "node_id": "MDExOlB1bGxSZXF1ZXN0MjQ3ODYxNDE5", "number": 5, "title": "Run Travis tests against Python 3.8-dev", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2019-01-26T02:30:55Z", "updated_at": "2019-01-26T02:37:54Z", "closed_at": "2019-01-26T02:37:54Z", "author_association": "OWNER", "pull_request": "simonw/sqlite-utils/pulls/5", "body": "", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/5/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null} {"id": 405801771, "node_id": "MDExOlB1bGxSZXF1ZXN0MjQ5NjgwOTQ0", "number": 9, "title": ":pencil: Updates my_database.py to my_database.db", "user": {"value": 50527, "label": "jefftriplett"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2019-02-01T17:35:43Z", "updated_at": "2019-02-24T03:55:04Z", "closed_at": "2019-02-24T03:55:04Z", "author_association": "CONTRIBUTOR", "pull_request": "simonw/sqlite-utils/pulls/9", "body": "I noticed that both `.py` and `.db` were used in the docs and assumed you'd prefer `.db`. ", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/9/reactions\", \"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null} {"id": 413778585, "node_id": "MDExOlB1bGxSZXF1ZXN0MjU1NjU4MTEy", "number": 12, "title": "Support for numpy types, closes #11", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2019-02-24T03:57:32Z", "updated_at": "2019-02-24T04:02:20Z", "closed_at": "2019-02-24T04:02:20Z", "author_association": "OWNER", "pull_request": "simonw/sqlite-utils/pulls/12", "body": "", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/12/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null} {"id": 413857257, "node_id": "MDU6SXNzdWU0MTM4NTcyNTc=", "number": 15, "title": "Ability to add columns to tables", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2019-02-24T19:20:51Z", "updated_at": "2019-02-24T20:04:40Z", "closed_at": "2019-02-24T20:04:40Z", "author_association": "OWNER", "pull_request": null, "body": "Makes sense to do this before foreign keys in #2\r\n\r\nPython:\r\n\r\n db[\"table\"].add_column(\"new_column\", int)\r\n\r\nCLI:\r\n\r\n $ sqlite-utils add-column table new_column INTEGER\r\n", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/15/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 432217625, "node_id": "MDU6SXNzdWU0MzIyMTc2MjU=", "number": 19, "title": "Incorrect help text for enable-fts command", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 4348046, "label": "1.0"}, "comments": 0, "created_at": "2019-04-11T19:46:44Z", "updated_at": "2019-05-25T00:44:31Z", "closed_at": "2019-05-25T00:44:31Z", "author_association": "OWNER", "pull_request": null, "body": "I clearly copied-and-pasted this from the `tables` command without updating it:\r\n\r\nhttps://github.com/simonw/sqlite-utils/blob/0b1af42ead3b3902347951180b3364ce1942da6e/sqlite_utils/cli.py#L216-L222", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/19/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 458941203, "node_id": "MDU6SXNzdWU0NTg5NDEyMDM=", "number": 29, "title": "Prevent accidental add-foreign-key with invalid column", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2019-06-20T23:57:24Z", "updated_at": "2019-06-20T23:58:26Z", "closed_at": "2019-06-20T23:58:26Z", "author_association": "OWNER", "pull_request": null, "body": "You can corrupt your database by running:\r\n\r\n $ sqlite-utils add-foreign-key my.db table non_existent_column other_table other_column\r\n", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/29/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 461237618, "node_id": "MDU6SXNzdWU0NjEyMzc2MTg=", "number": 31, "title": "Mechanism for adding multiple foreign key constraints at once", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2019-06-27T00:04:30Z", "updated_at": "2019-06-29T06:27:40Z", "closed_at": "2019-06-29T06:27:40Z", "author_association": "OWNER", "pull_request": null, "body": "Needed by [db-to-sqlite](https://github.com/simonw/db-to-sqlite). It currently works by collecting all of the foreign key relationships it can find and then applying them at the end of the process.\r\n\r\nThe problem is, the `add_foreign_key()` method looks like this:\r\n\r\nhttps://github.com/simonw/sqlite-utils/blob/86bd2bba689e25f09551d611ccfbee1e069e5b66/sqlite_utils/db.py#L498-L516\r\n\r\nThat means it's doing a full `VACUUM` for every single relationship it sets up - and if you have hundreds of foreign key relationships in your database this can take hours.\r\n\r\nI think the right solution is to have a `.add_foreign_keys(list_of_args)` method which does the bulk operation and then a single `VACUUM`. `.add_foreign_key(...)` can then call the bulk action with a single list item.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/31/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 462423972, "node_id": "MDExOlB1bGxSZXF1ZXN0MjkzMTE3MTgz", "number": 34, "title": "sqlite-utils index-foreign-keys / db.index_foreign_keys()", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2019-06-30T16:43:40Z", "updated_at": "2019-06-30T23:50:55Z", "closed_at": "2019-06-30T23:50:55Z", "author_association": "OWNER", "pull_request": "simonw/sqlite-utils/pulls/34", "body": "Refs #33\r\n\r\n- [x] `sqlite-utils index-foreign-keys` command\r\n- [x] `db.index_foreign_keys()` method\r\n- [x] unit tests\r\n- [x] documentation", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/34/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null} {"id": 462817589, "node_id": "MDU6SXNzdWU0NjI4MTc1ODk=", "number": 36, "title": "Support compound primary keys", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2019-07-01T17:00:07Z", "updated_at": "2019-07-15T04:28:52Z", "closed_at": "2019-07-15T04:28:52Z", "author_association": "OWNER", "pull_request": null, "body": "This should work:\r\n```python\r\ntable = db[\"dog_breeds\"].insert({\r\n \"dog_id\": 1,\r\n \"breed_id\": 2\r\n}, pk=(\"dog_id\", \"breed_id\"))\r\n```\r\nNeeded for m2m work in #23", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/36/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 467864071, "node_id": "MDU6SXNzdWU0Njc4NjQwNzE=", "number": 39, "title": "table.get(...) method", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2019-07-14T17:20:51Z", "updated_at": "2019-07-15T04:28:53Z", "closed_at": "2019-07-15T04:28:53Z", "author_association": "OWNER", "pull_request": null, "body": "Utility method for fetching a record by its primary key.\r\n\r\nAccepts a single value (for primary key / rowid tables) or a list/tuple of values (for compound primary keys, refs #36).\r\n\r\nRaises a `NotFoundError` if the record cannot be found.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/39/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 470131537, "node_id": "MDU6SXNzdWU0NzAxMzE1Mzc=", "number": 41, "title": "sqlite-utils insert --tsv option", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2019-07-19T04:27:21Z", "updated_at": "2019-07-19T04:50:47Z", "closed_at": "2019-07-19T04:50:47Z", "author_association": "OWNER", "pull_request": null, "body": "Right now we only support ingesting CSV, but sometimes interesting data is released as TSV.\r\n\r\nhttps://www.washingtonpost.com/national/2019/07/18/how-download-use-dea-pain-pills-database/ for example.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/41/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 471684708, "node_id": "MDExOlB1bGxSZXF1ZXN0MzAwMjg2NTM1", "number": 45, "title": "Implemented table.lookup(...), closes #44", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2019-07-23T13:03:30Z", "updated_at": "2019-07-23T13:07:00Z", "closed_at": "2019-07-23T13:07:00Z", "author_association": "OWNER", "pull_request": "simonw/sqlite-utils/pulls/45", "body": "", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/45/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null} {"id": 471797101, "node_id": "MDExOlB1bGxSZXF1ZXN0MzAwMzc3NTk5", "number": 47, "title": "extracts= table parameter", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2019-07-23T16:30:29Z", "updated_at": "2019-07-23T17:00:43Z", "closed_at": "2019-07-23T17:00:43Z", "author_association": "OWNER", "pull_request": "simonw/sqlite-utils/pulls/47", "body": "Still needs docs. Refs #46", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/47/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null} {"id": 476436920, "node_id": "MDExOlB1bGxSZXF1ZXN0MzAzOTkwNjgz", "number": 53, "title": "Work in progress: m2m() method for creating many-to-many records", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2019-08-03T10:03:56Z", "updated_at": "2019-08-04T03:38:10Z", "closed_at": "2019-08-04T03:37:33Z", "author_association": "OWNER", "pull_request": "simonw/sqlite-utils/pulls/53", "body": "- [x] `table.insert({\"name\": \"Barry\"}).m2m(\"tags\", lookup={\"tag\": \"Coworker\"})`\r\n- [x] Explicit table name `.m2m(\"humans\", ..., m2m_table=\"relationships\")`\r\n- [x] Automatically use an existing m2m table if a single obvious candidate exists (a table with two foreign keys in the correct directions)\r\n- [x] Require the explicit `m2m_table=` argument if multiple candidates for the m2m table exist\r\n- [x] Documentation\r\n\r\nRefs #23", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/53/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null} {"id": 488338965, "node_id": "MDU6SXNzdWU0ODgzMzg5NjU=", "number": 59, "title": "Ability to introspect triggers", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2019-09-02T23:47:16Z", "updated_at": "2019-09-03T01:52:36Z", "closed_at": "2019-09-03T00:09:42Z", "author_association": "OWNER", "pull_request": null, "body": "Now that we're creating triggers (thanks to @amjith in #57) it would be neat if we could introspect them too.\r\n\r\nI'm thinking:\r\n\r\n`db.triggers` - lists all triggers for the database\r\n`db[\"tablename\"].triggers` - lists triggers for that table\r\n\r\nThe underlying query for this is `select * from sqlite_master where type = 'trigger'`\r\n\r\nI'll return the trigger information in a new namedtuple, similar to how Indexes and ForeignKeys work.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/59/reactions\", \"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 488341021, "node_id": "MDExOlB1bGxSZXF1ZXN0MzEzMzgzMzE3", "number": 60, "title": "db.triggers and table.triggers introspection", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2019-09-03T00:04:32Z", "updated_at": "2019-09-03T00:09:42Z", "closed_at": "2019-09-03T00:09:42Z", "author_association": "OWNER", "pull_request": "simonw/sqlite-utils/pulls/60", "body": "Closes #59", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/60/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null} {"id": 519032008, "node_id": "MDExOlB1bGxSZXF1ZXN0MzM3ODQ3NTcz", "number": 64, "title": "test_insert_upsert_all_empty_list", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2019-11-07T04:24:45Z", "updated_at": "2019-11-07T04:32:38Z", "closed_at": "2019-11-07T04:32:38Z", "author_association": "OWNER", "pull_request": "simonw/sqlite-utils/pulls/64", "body": "", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/64/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null} {"id": 519039316, "node_id": "MDExOlB1bGxSZXF1ZXN0MzM3ODUzMzk0", "number": 65, "title": "Release 1.12.1", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2019-11-07T04:51:29Z", "updated_at": "2019-11-07T04:58:48Z", "closed_at": "2019-11-07T04:58:47Z", "author_association": "OWNER", "pull_request": "simonw/sqlite-utils/pulls/65", "body": "", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/65/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null} {"id": 543738004, "node_id": "MDExOlB1bGxSZXF1ZXN0MzU3OTkyNTg4", "number": 72, "title": "Fixed implementation of upsert", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2019-12-30T05:08:05Z", "updated_at": "2019-12-30T05:29:24Z", "closed_at": "2019-12-30T05:29:24Z", "author_association": "OWNER", "pull_request": "simonw/sqlite-utils/pulls/72", "body": "Refs #66", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/72/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null} {"id": 557830332, "node_id": "MDExOlB1bGxSZXF1ZXN0MzY5MzQ4MDg0", "number": 78, "title": "New conversions= feature, refs #77", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2020-01-31T00:02:33Z", "updated_at": "2020-09-22T07:48:29Z", "closed_at": "2020-01-31T00:24:31Z", "author_association": "OWNER", "pull_request": "simonw/sqlite-utils/pulls/78", "body": "", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/78/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null} {"id": 561460274, "node_id": "MDU6SXNzdWU1NjE0NjAyNzQ=", "number": 84, "title": ".upsert() with hash_id throws error", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2020-02-07T07:08:19Z", "updated_at": "2020-02-07T07:17:11Z", "closed_at": "2020-02-07T07:17:11Z", "author_association": "OWNER", "pull_request": null, "body": "```python\r\ndb[table_name].upsert_all(rows, hash_id=\"pk\")\r\n```\r\nThis throws an error: `PrimaryKeyRequired('upsert() requires a pk')`\r\n\r\nThe problem is, if you try this:\r\n\r\n```python\r\ndb[table_name].upsert_all(rows, hash_id=\"pk\", pk=\"pk\")\r\n```\r\nYou get this error: `AssertionError('Use either pk= or hash_id=')`\r\n\r\n`hash_id=` should imply that `pk=` that column.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/84/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 573740712, "node_id": "MDU6SXNzdWU1NzM3NDA3MTI=", "number": 90, "title": "Cannot .enable_fts() for columns with spaces in their names", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2020-03-02T06:06:03Z", "updated_at": "2020-03-02T06:10:49Z", "closed_at": "2020-03-02T06:10:49Z", "author_association": "OWNER", "pull_request": null, "body": "```\r\nimport sqlite_utils\r\ndb = sqlite_utils.Database(memory=True) \r\ndb[\"test\"].insert({\"space in name\": \"hello\"}) \r\ndb[\"test\"].enable_fts([\"space in name\"]) \r\n---------------------------------------------------------------------------\r\nOperationalError Traceback (most recent call last)\r\n in \r\n----> 1 db['test'].enable_fts([\"space in name\"])\r\n\r\n/usr/local/lib/python3.7/site-packages/sqlite_utils/db.py in enable_fts(self, columns, fts_version, create_triggers)\r\n 755 )\r\n 756 self.db.conn.executescript(sql)\r\n--> 757 self.populate_fts(columns)\r\n 758 \r\n 759 if create_triggers:\r\n\r\n/usr/local/lib/python3.7/site-packages/sqlite_utils/db.py in populate_fts(self, columns)\r\n 787 table=self.name, columns=\", \".join(columns)\r\n 788 )\r\n--> 789 self.db.conn.executescript(sql)\r\n 790 return self\r\n 791 \r\n\r\nOperationalError: near \"in\": syntax error\r\n```", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/90/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 581795570, "node_id": "MDU6SXNzdWU1ODE3OTU1NzA=", "number": 93, "title": "Support more string values for types in .add_column()", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2020-03-15T19:32:49Z", "updated_at": "2020-09-24T20:36:46Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "https://sqlite-utils.readthedocs.io/en/2.4.2/python-api.html#adding-columns says:\r\n> SQLite types you can specify are \"TEXT\", \"INTEGER\", \"FLOAT\" or \"BLOB\".\r\n\r\nAs discovered in #92 this isn't the right list of values. I should expand this to match https://www.sqlite.org/datatype3.html", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/93/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 586477757, "node_id": "MDU6SXNzdWU1ODY0Nzc3NTc=", "number": 94, "title": "If column data is a mixture of integers and nulls, detected type should be INTEGER", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2020-03-23T19:51:46Z", "updated_at": "2020-03-23T19:57:10Z", "closed_at": "2020-03-23T19:57:10Z", "author_association": "OWNER", "pull_request": null, "body": "It looks like detected type for that case is TEXT at the moment.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/94/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 586486367, "node_id": "MDU6SXNzdWU1ODY0ODYzNjc=", "number": 95, "title": "Columns with only null values are no longer created in the database", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2020-03-23T20:07:42Z", "updated_at": "2020-03-23T20:31:15Z", "closed_at": "2020-03-23T20:31:15Z", "author_association": "OWNER", "pull_request": null, "body": "Bug introduced in #94, and released in `2.4.3`.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/95/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 601392318, "node_id": "MDU6SXNzdWU2MDEzOTIzMTg=", "number": 101, "title": "README should include an example of CLI data insertion", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2020-04-16T19:45:37Z", "updated_at": "2020-04-17T23:59:49Z", "closed_at": "2020-04-17T23:59:49Z", "author_association": "OWNER", "pull_request": null, "body": "Maybe using `curl` from the GitHub API.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/101/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 602569315, "node_id": "MDU6SXNzdWU2MDI1NjkzMTU=", "number": 102, "title": "Can't store an array or dictionary containing a bytes value", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2020-04-18T22:49:21Z", "updated_at": "2020-05-01T20:45:45Z", "closed_at": "2020-05-01T20:45:45Z", "author_association": "OWNER", "pull_request": null, "body": "```\r\nIn [1]: import sqlite_utils \r\n\r\nIn [2]: db = sqlite_utils.Database(memory=True) \r\n\r\nIn [3]: db[\"t\"].insert({\"id\": 1, \"data\": {\"foo\": b\"bytes\"}}) \r\n---------------------------------------------------------------------------\r\nTypeError Traceback (most recent call last)\r\n in \r\n----> 1 db[\"t\"].insert({\"id\": 1, \"data\": {\"foo\": b\"bytes\"}})\r\n\r\n~/Dropbox/Development/sqlite-utils/sqlite_utils/db.py in insert(self, record, pk, foreign_keys, column_order, not_null, defaults, hash_id, alter, ignore, replace, extracts, conversions, columns)\r\n 950 extracts=extracts,\r\n 951 conversions=conversions,\r\n--> 952 columns=columns,\r\n 953 )\r\n 954 \r\n\r\n~/Dropbox/Development/sqlite-utils/sqlite_utils/db.py in insert_all(self, records, pk, foreign_keys, column_order, not_null, defaults, batch_size, hash_id, alter, ignore, replace, extracts, conversions, columns, upsert)\r\n 1052 for key in all_columns:\r\n 1053 value = jsonify_if_needed(\r\n-> 1054 record.get(key, None if key != hash_id else _hash(record))\r\n 1055 )\r\n 1056 if key in extracts:\r\n\r\n~/Dropbox/Development/sqlite-utils/sqlite_utils/db.py in jsonify_if_needed(value)\r\n 1318 def jsonify_if_needed(value):\r\n 1319 if isinstance(value, (dict, list, tuple)):\r\n-> 1320 return json.dumps(value)\r\n 1321 elif isinstance(value, (datetime.time, datetime.date, datetime.datetime)):\r\n 1322 return value.isoformat()\r\n\r\n/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/__init__.py in dumps(obj, skipkeys, ensure_ascii, check_circular, allow_nan, cls, indent, separators, default, sort_keys, **kw)\r\n 229 cls is None and indent is None and separators is None and\r\n 230 default is None and not sort_keys and not kw):\r\n--> 231 return _default_encoder.encode(obj)\r\n 232 if cls is None:\r\n 233 cls = JSONEncoder\r\n\r\n/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/encoder.py in encode(self, o)\r\n 197 # exceptions aren't as detailed. The list call should be roughly\r\n 198 # equivalent to the PySequence_Fast that ''.join() would do.\r\n--> 199 chunks = self.iterencode(o, _one_shot=True)\r\n 200 if not isinstance(chunks, (list, tuple)):\r\n 201 chunks = list(chunks)\r\n\r\n/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/encoder.py in iterencode(self, o, _one_shot)\r\n 255 self.key_separator, self.item_separator, self.sort_keys,\r\n 256 self.skipkeys, _one_shot)\r\n--> 257 return _iterencode(o, 0)\r\n 258 \r\n 259 def _make_iterencode(markers, _default, _encoder, _indent, _floatstr,\r\n\r\n/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/encoder.py in default(self, o)\r\n 177 \r\n 178 \"\"\"\r\n--> 179 raise TypeError(f'Object of type {o.__class__.__name__} '\r\n 180 f'is not JSON serializable')\r\n 181 \r\n\r\nTypeError: Object of type bytes is not JSON serializable\r\n```", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/102/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 610853393, "node_id": "MDU6SXNzdWU2MTA4NTMzOTM=", "number": 104, "title": "--schema option to \"sqlite-utils tables\"", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2020-05-01T16:55:49Z", "updated_at": "2020-05-01T17:12:37Z", "closed_at": "2020-05-01T17:12:37Z", "author_association": "OWNER", "pull_request": null, "body": "Adds output showing the table schema.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/104/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 665701216, "node_id": "MDU6SXNzdWU2NjU3MDEyMTY=", "number": 123, "title": "--raw option for outputting binary content", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2020-07-26T03:35:39Z", "updated_at": "2020-07-26T16:44:11Z", "closed_at": "2020-07-26T16:44:11Z", "author_association": "OWNER", "pull_request": null, "body": "Related to the `insert-files` work in #122 - it should be easy to get binary data back out of the database again.\r\n\r\nOne way to do that could be:\r\n\r\n sqlite-utils files.db \"select content from files where key = 'foo.jpg'\" --raw\r\n\r\nThe `--raw` option would cause just the contents of the first column to be output directly to stdout.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/123/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 688352145, "node_id": "MDU6SXNzdWU2ODgzNTIxNDU=", "number": 141, "title": "insert-files support for compressed values", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2020-08-28T20:59:46Z", "updated_at": "2020-09-24T20:36:08Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "The `sqlar` format supports this, it would be useful if `insert-files` could support this too.\r\n\r\nhttps://www.sqlite.org/sqlar.html", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/141/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 695359607, "node_id": "MDU6SXNzdWU2OTUzNTk2MDc=", "number": 150, "title": "Feature for tracing SQL queries", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2020-09-07T19:43:08Z", "updated_at": "2020-09-07T21:57:01Z", "closed_at": "2020-09-07T21:57:01Z", "author_association": "OWNER", "pull_request": null, "body": "Debugging `sqlite-utils` when something weird happens (e.g. #149) can be a bit tricky since it runs a bunch of different SQL statements behind the scenes.\r\n\r\nAn optional \"tracing\" mechanism for seeing what SQL is being executed would be useful.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/150/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 695360889, "node_id": "MDExOlB1bGxSZXF1ZXN0NDgxNjE2NzA0", "number": 151, "title": "Tracer mechanism for seeing underlying SQL", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2020-09-07T19:46:43Z", "updated_at": "2020-09-07T21:57:00Z", "closed_at": "2020-09-07T21:57:00Z", "author_association": "OWNER", "pull_request": "simonw/sqlite-utils/pulls/151", "body": "Refs #150. Needs tests and documentation, including for the new `db.execute()` and `db.executescript()` methods.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/151/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null} {"id": 706091046, "node_id": "MDU6SXNzdWU3MDYwOTEwNDY=", "number": 165, "title": "Make .transform() a keyword arguments only function", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 5897911, "label": "2.20"}, "comments": 0, "created_at": "2020-09-22T05:37:29Z", "updated_at": "2020-09-24T20:35:47Z", "closed_at": "2020-09-22T06:39:12Z", "author_association": "OWNER", "pull_request": null, "body": "And rename the first argument from `columns=` to `types=`", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/165/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 706092617, "node_id": "MDExOlB1bGxSZXF1ZXN0NDkwNzAzMTcz", "number": 166, "title": "Keyword only arguments for transform()", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2020-09-22T05:41:44Z", "updated_at": "2020-09-22T06:39:11Z", "closed_at": "2020-09-22T06:39:11Z", "author_association": "OWNER", "pull_request": "simonw/sqlite-utils/pulls/166", "body": "Refs #165", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/166/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null} {"id": 706757891, "node_id": "MDU6SXNzdWU3MDY3NTc4OTE=", "number": 169, "title": "Progress bar for \"sqlite-utils extract\"", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 5897911, "label": "2.20"}, "comments": 0, "created_at": "2020-09-22T23:40:21Z", "updated_at": "2020-09-24T20:34:50Z", "closed_at": "2020-09-23T00:02:40Z", "author_association": "OWNER", "pull_request": null, "body": "> Since these operations could take a long time against large tables, it would be neat if there was a progress bar option for the CLI command.\r\n>\r\n> The operations are full table scans so calculating progress shouldn't be too difficult.\r\n_Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/42#issuecomment-513246831_", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/169/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 716955793, "node_id": "MDExOlB1bGxSZXF1ZXN0NDk5NjAzMzU5", "number": 184, "title": "Test against Python 3.9", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2020-10-08T01:37:05Z", "updated_at": "2020-10-08T01:44:06Z", "closed_at": "2020-10-08T01:44:06Z", "author_association": "OWNER", "pull_request": "simonw/sqlite-utils/pulls/184", "body": "", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/184/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null} {"id": 718952107, "node_id": "MDU6SXNzdWU3MTg5NTIxMDc=", "number": 185, "title": "Use db[table] consistently in documentation", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2020-10-11T23:39:12Z", "updated_at": "2020-10-12T00:13:41Z", "closed_at": "2020-10-12T00:13:41Z", "author_association": "OWNER", "pull_request": null, "body": "The Python docs have a bunch of examples like this: https://sqlite-utils.readthedocs.io/en/stable/python-api.html\r\n```python\r\ndogs.enable_fts([\"name\", \"twitter\"], create_triggers=True)\r\n```\r\nThis would be easier for people to understand if it looked like this instead:\r\n```python\r\ndb[\"dogs\"].enable_fts([\"name\", \"twitter\"], create_triggers=True)\r\n```", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/185/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 730693696, "node_id": "MDExOlB1bGxSZXF1ZXN0NTEwOTU2MTM0", "number": 190, "title": "Progress bar for sqlite-utils insert command", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2020-10-27T18:08:53Z", "updated_at": "2020-10-27T18:16:03Z", "closed_at": "2020-10-27T18:16:03Z", "author_association": "OWNER", "pull_request": "simonw/sqlite-utils/pulls/190", "body": "Refs #173", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/190/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null} {"id": 735648209, "node_id": "MDU6SXNzdWU3MzU2NDgyMDk=", "number": 193, "title": "--tsv output format option", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 6079500, "label": "3.0"}, "comments": 0, "created_at": "2020-11-03T21:31:18Z", "updated_at": "2020-11-07T00:09:52Z", "closed_at": "2020-11-07T00:09:52Z", "author_association": "OWNER", "pull_request": null, "body": "We already support `--csv` for output, and the `insert` command accepts `--tsv`. The output format options should accept `--tsv` too.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/193/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 777386465, "node_id": "MDU6SXNzdWU3NzczODY0NjU=", "number": 211, "title": "table.triggers_dict introspection property", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2021-01-02T02:04:00Z", "updated_at": "2021-01-02T02:10:10Z", "closed_at": "2021-01-02T02:10:10Z", "author_association": "OWNER", "pull_request": null, "body": "`table.triggers` currently returns a list of `Trigger` values. A `table.triggers_dict` property could behave like `columns_dict`, returning a dictionary mapping trigger names to their SQL definitions for that table.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/211/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 777530107, "node_id": "MDU6SXNzdWU3Nzc1MzAxMDc=", "number": 214, "title": "sqlite-utils enable-counts command", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2021-01-02T21:45:48Z", "updated_at": "2021-01-03T04:26:44Z", "closed_at": "2021-01-03T04:26:44Z", "author_association": "OWNER", "pull_request": null, "body": "The CLI version of #212 and #213.\r\n\r\n # Enable counts for all tables:\r\n sqlite-utils enable-counts data.db\r\n\r\n # Enable counts for specific tables:\r\n sqlite-utils enable-counts data.db table1 table2", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/214/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 783910901, "node_id": "MDU6SXNzdWU3ODM5MTA5MDE=", "number": 221, "title": ".add_missing_columns() does not take case insensitivity into account", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2021-01-12T05:01:00Z", "updated_at": "2021-01-12T23:17:33Z", "closed_at": "2021-01-12T23:17:33Z", "author_association": "OWNER", "pull_request": null, "body": "SQLite columns are case insensitive - but the `.add_missing_columns()` method doesn't know that. This means that it can crash if it identifies a column that is a case-insensitive duplicate of an existing column. https://github.com/simonw/sqlite-utils/blob/4cc82fd0bccc9d2eeb3510beb4e691d7da099f84/sqlite_utils/db.py#L1974-L1980", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/221/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 787900412, "node_id": "MDU6SXNzdWU3ODc5MDA0MTI=", "number": 222, "title": ".m2m() should accept alter=True parameter", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2021-01-18T04:15:43Z", "updated_at": "2021-01-18T04:26:10Z", "closed_at": "2021-01-18T04:26:10Z", "author_association": "OWNER", "pull_request": null, "body": "Needed by https://github.com/dogsheep/swarm-to-sqlite/issues/11", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/222/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 802583450, "node_id": "MDU6SXNzdWU4MDI1ODM0NTA=", "number": 226, "title": "3.4 release is broken - includes a rogue line", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2021-02-06T02:08:01Z", "updated_at": "2021-02-06T02:10:26Z", "closed_at": "2021-02-06T02:10:26Z", "author_association": "OWNER", "pull_request": null, "body": "I started seeing weird errors, caused by this line: https://github.com/simonw/sqlite-utils/blob/f8010ca78fed8c5fca6cde19658ec09fdd468420/sqlite_utils/cli.py#L1-L3\r\n\r\nThat was added by accident in 1b666f9315d4ea6bb332b2e75e48480c26100199\r\n\r\nI'm surprised the tests didn't catch this!", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/226/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 808036774, "node_id": "MDU6SXNzdWU4MDgwMzY3NzQ=", "number": 232, "title": "Run tests against Windows in GitHub Actions", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2021-02-14T20:09:45Z", "updated_at": "2021-02-14T20:39:55Z", "closed_at": "2021-02-14T20:39:55Z", "author_association": "OWNER", "pull_request": null, "body": "> I'm going to try and get the test suite to run in Windows on GitHub Actions.\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/225#issuecomment-778834504_", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/232/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 808037010, "node_id": "MDExOlB1bGxSZXF1ZXN0NTczMTQ3MTY4", "number": 233, "title": "Run tests against Ubuntu, macOS and Windows", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2021-02-14T20:11:02Z", "updated_at": "2021-02-14T20:39:54Z", "closed_at": "2021-02-14T20:39:54Z", "author_association": "OWNER", "pull_request": "simonw/sqlite-utils/pulls/233", "body": "Refs #232", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/233/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null} {"id": 816601354, "node_id": "MDExOlB1bGxSZXF1ZXN0NTgwMjM1NDI3", "number": 241, "title": "Extract expand - work in progress", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2021-02-25T16:36:38Z", "updated_at": "2021-02-25T16:36:38Z", "closed_at": null, "author_association": "OWNER", "pull_request": "simonw/sqlite-utils/pulls/241", "body": "Refs #239. Still needs documentation and CLI implementation.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/241/reactions\", \"total_count\": 3, \"+1\": 3, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 1, "state_reason": null} {"id": 818684978, "node_id": "MDU6SXNzdWU4MTg2ODQ5Nzg=", "number": 243, "title": "How can i use this utils to deal with fts on column meta of tables ?", "user": {"value": 27874014, "label": "svjack"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2021-03-01T09:45:05Z", "updated_at": "2021-03-01T09:45:05Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "Thank you to release this bravo project.\r\nWhen i use this project on multi table db, I want to implement \r\nconvenient search on column name from different tables.\r\nI want to develop a meta table to save the meta data of different columns\r\nof different tables and search on this meta table to get rows from the\r\ndata table (which the meta table describes)\r\ndoes this project provide some simple function on it ?\r\n\r\nYou can think a have a knowledge graph about the table in the db, \r\nand i save this knowledge graph into the db with fts enabled.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/243/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 858501079, "node_id": "MDU6SXNzdWU4NTg1MDEwNzk=", "number": 255, "title": "transform --help should tell you the available types", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2021-04-15T05:24:48Z", "updated_at": "2021-05-29T03:55:52Z", "closed_at": "2021-05-29T03:55:52Z", "author_association": "OWNER", "pull_request": null, "body": "```\r\nUsage: sqlite-utils transform [OPTIONS] PATH TABLE\r\n\r\n Transform a table beyond the capabilities of ALTER TABLE\r\n\r\nOptions:\r\n --type ... Change column type to X\r\n```\r\nThis should specify that the possible types are 'INTEGER', 'TEXT', 'FLOAT', 'BLOB'.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/255/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 868188068, "node_id": "MDU6SXNzdWU4NjgxODgwNjg=", "number": 257, "title": "Insert from JSON containing strings with non-ascii characters are escaped as unicode for lists, tuples, dicts.", "user": {"value": 6586811, "label": "dylan-wu"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2021-04-26T20:46:25Z", "updated_at": "2021-05-19T02:57:05Z", "closed_at": "2021-05-19T02:57:05Z", "author_association": "CONTRIBUTOR", "pull_request": null, "body": "JSON Test File (test.json):\r\n\r\n```json\r\n[\r\n {\r\n \"id\": 123,\r\n \"text\": \"FR Th\u00e9\u00e2tre\"\r\n },\r\n {\r\n \"id\": 223,\r\n \"text\": [\r\n \"FR Th\u00e9\u00e2tre\"\r\n ]\r\n }\r\n]\r\n```\r\n\r\nCommand to import:\r\n\r\n```bash\r\nsqlite-utils insert test.db text test.json --pk=id\r\n```\r\n\r\nResulting table view from datasette:\r\n\r\n![image](https://user-images.githubusercontent.com/6586811/116147833-cdf2fb00-a6a5-11eb-8412-0aae81b6e6dd.png)\r\n\r\nOriginal, db.py line 2225:\r\n\r\n```python\r\n return json.dumps(value, default=repr)\r\n```\r\n\r\nFix, db.py line 2225:\r\n\r\n```python\r\n return json.dumps(value, default=repr, ensure_ascii=False)\r\n```", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/257/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 906355849, "node_id": "MDExOlB1bGxSZXF1ZXN0NjU3MzczNzI2", "number": 262, "title": "Ability to add descending order indexes", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2021-05-29T04:51:04Z", "updated_at": "2021-05-29T05:01:42Z", "closed_at": "2021-05-29T05:01:39Z", "author_association": "OWNER", "pull_request": "simonw/sqlite-utils/pulls/262", "body": "Refs #260", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/262/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null} {"id": 922832113, "node_id": "MDU6SXNzdWU5MjI4MzIxMTM=", "number": 274, "title": "sqlite-utils dump my.db command", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2021-06-16T16:30:14Z", "updated_at": "2021-06-16T23:51:54Z", "closed_at": "2021-06-16T23:51:54Z", "author_association": "OWNER", "pull_request": null, "body": "Inspired by the `--dump` mechanism I added to `sqlite-utils memory` here: https://github.com/simonw/sqlite-utils/issues/272#issuecomment-862018937\r\n\r\n> Can use `.iterdump()` to implement this: https://docs.python.org/3/library/sqlite3.html#sqlite3.Connection.iterdump\r\n>\r\n> Maybe instead (or as-well-as) offer `--dump` which dumps out the SQL from that.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/274/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 923602693, "node_id": "MDU6SXNzdWU5MjM2MDI2OTM=", "number": 276, "title": "support small help flag -h", "user": {"value": 601708, "label": "mcint"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2021-06-17T07:59:31Z", "updated_at": "2021-06-18T14:56:59Z", "closed_at": "2021-06-18T14:56:59Z", "author_association": "CONTRIBUTOR", "pull_request": null, "body": "", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/276/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 924991194, "node_id": "MDU6SXNzdWU5MjQ5OTExOTQ=", "number": 280, "title": "Add --encoding option to sqlite-utils memory", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2021-06-18T15:03:32Z", "updated_at": "2021-06-18T15:29:46Z", "closed_at": "2021-06-18T15:29:46Z", "author_association": "OWNER", "pull_request": null, "body": "Follow-on from #272 - this will work like `--encoding` on `sqlite-utils insert` and will affect all CSV files processed by `sqlite-utils memory`.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/280/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 925544070, "node_id": "MDU6SXNzdWU5MjU1NDQwNzA=", "number": 287, "title": "Update rowid examples in the docs", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2021-06-20T08:03:00Z", "updated_at": "2021-06-20T18:26:21Z", "closed_at": "2021-06-20T18:26:21Z", "author_association": "OWNER", "pull_request": null, "body": "Changed in #284 - a couple of examples need updating on https://github.com/simonw/sqlite-utils/blob/3.10/docs/cli.rst.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/287/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 925545468, "node_id": "MDU6SXNzdWU5MjU1NDU0Njg=", "number": 288, "title": "sqlite-utils memory blah.json --schema", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2021-06-20T08:10:40Z", "updated_at": "2021-06-20T18:26:21Z", "closed_at": "2021-06-20T18:26:21Z", "author_association": "OWNER", "pull_request": null, "body": "Like `--dump` but only outputs the schema - useful for understanding what you are about to run queries against.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/288/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 927789811, "node_id": "MDU6SXNzdWU5Mjc3ODk4MTE=", "number": 292, "title": "Add contributing documentation", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2021-06-23T02:13:05Z", "updated_at": "2021-06-25T17:53:51Z", "closed_at": "2021-06-25T17:53:51Z", "author_association": "OWNER", "pull_request": null, "body": "Like https://docs.datasette.io/en/latest/contributing.html (but simpler) - should cover how to run `black` and `flake8` and `mypy` and how to run the tests.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/292/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 931752773, "node_id": "MDU6SXNzdWU5MzE3NTI3NzM=", "number": 294, "title": "Add a `sqlite-utils memory` example to the README", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2021-06-28T16:35:59Z", "updated_at": "2021-08-18T21:40:03Z", "closed_at": "2021-08-18T21:40:03Z", "author_association": "OWNER", "pull_request": null, "body": "", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/294/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 957383814, "node_id": "MDU6SXNzdWU5NTczODM4MTQ=", "number": 301, "title": "insert-files should get a --silent option", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2021-08-01T04:11:03Z", "updated_at": "2021-08-02T19:12:21Z", "closed_at": "2021-08-02T19:12:21Z", "author_association": "OWNER", "pull_request": null, "body": "The new `sqlite-utils convert` command I'm adding in #251 will have a `--silent` option for turning off the progress bars. The only other command that has progress bars right now is `insert-files` so it should get this option too, for consistency.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/301/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 959305209, "node_id": "MDU6SXNzdWU5NTkzMDUyMDk=", "number": 307, "title": "codespell to spell check documentation", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2021-08-03T16:48:19Z", "updated_at": "2021-08-03T16:48:53Z", "closed_at": "2021-08-03T16:48:53Z", "author_association": "OWNER", "pull_request": null, "body": "As seen in https://github.com/simonw/datasette/issues/1417 and https://til.simonwillison.net/python/codespell", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/307/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 965166058, "node_id": "MDU6SXNzdWU5NjUxNjYwNTg=", "number": 313, "title": "`.add_foreign_keys()` doesn't reject being called with a View", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2021-08-10T17:22:17Z", "updated_at": "2021-08-10T17:25:34Z", "closed_at": "2021-08-10T17:25:34Z", "author_association": "OWNER", "pull_request": null, "body": "Spotted this bug using `mypy` while working on #311 / #312!\r\n\r\n```\r\n% mypy sqlite_utils\r\nsqlite_utils/db.py:725: error: Item \"View\" of \"Union[Table, View]\" has no attribute \"foreign_keys\"\r\nFound 1 error in 1 file (checked 5 source files)\r\n```\r\nRefers to this code: https://github.com/simonw/sqlite-utils/blob/c11ff89894727270d4a9eb554d3a006f5b0d8d9d/sqlite_utils/db.py#L710-L720\r\n\r\nIt's a bug! We run some checks earlier but none of them ensure that it's a view:\r\n\r\nhttps://github.com/simonw/sqlite-utils/blob/c11ff89894727270d4a9eb554d3a006f5b0d8d9d/sqlite_utils/db.py#L697-L709", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/313/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1000275035, "node_id": "PR_kwDOCGYnMM4r7n-9", "number": 327, "title": "Extract expand: Support JSON Arrays", "user": {"value": 101753, "label": "phaer"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2021-09-19T10:34:30Z", "updated_at": "2022-12-29T09:05:36Z", "closed_at": "2022-12-29T09:05:36Z", "author_association": "NONE", "pull_request": "simonw/sqlite-utils/pulls/327", "body": "Hi,\r\n\r\nI needed to extract data in JSON Arrays to normalize data imports. I've quickly hacked the following together based on #241 which refers to #239 where you, @simonw, wrote:\r\n\r\n> Could this handle lists of objects too? That would be pretty amazing - if the column has a [{...}, {...}] list in it could turn that into a many-to-many.\r\n\r\nThey way this works in my work is that many-to-many relationships are created for anything that maps to an dictionary in a list, and many-to-one relations for everything else (assumed to be scalar values). Not sure what the best approach here would be? Are many-to-one relationships are at all useful here?\r\n\r\nWhat do you think about this approach? I could try to add it to the cli interface and documentation if wanted.\r\n\r\nThanks for this awesome piece of software in any case! :sun_with_face: ", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/327/reactions\", \"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null} {"id": 1041778507, "node_id": "I_kwDOCGYnMM4-GEdL", "number": 334, "title": "Filter by datetime objects using rows_where()", "user": {"value": 11642379, "label": "viseshrp"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2021-11-02T00:44:08Z", "updated_at": "2021-11-13T19:23:21Z", "closed_at": "2021-11-13T19:23:21Z", "author_association": "NONE", "pull_request": null, "body": "Firstly, thanks for this nice utility. \r\nIt would be nice to have an example in the docs on how to filter by date range using `rows_where()`. \r\nThis doesn't seem to work:\r\n```\r\ntable.rows_where('datetime(created) between datetime(\"2021-10-31T17:29:59.277428-04:00\") AND datetime(\"2021-11-01T03:44:04.544651+00:00\")')\r\n```\r\n\r\n\r\nI could probably just use `db.query()`, which works for the above, but it would be nice if I could pass in `datetime` objects in `rows_where()`.\r\nThanks.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/334/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1053087862, "node_id": "I_kwDOCGYnMM4-xNh2", "number": 338, "title": "dict, list, tuple should all map to TEXT", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2021-11-15T00:28:01Z", "updated_at": "2021-11-15T00:36:03Z", "closed_at": "2021-11-15T00:36:03Z", "author_association": "OWNER", "pull_request": null, "body": "> This relates to the fact that dictionaries, lists and tuples get special treatment and are converted to JSON strings, using this code: https://github.com/simonw/sqlite-utils/blob/e8d958109ee290cfa1b44ef7a39629bb50ab673e/sqlite_utils/db.py#L2937-L2947\r\n>\r\n> So the `COLUMN_TYPE_MAPPING` should include those too - right now it looks like this: https://github.com/simonw/sqlite-utils/blob/e8d958109ee290cfa1b44ef7a39629bb50ab673e/sqlite_utils/db.py#L165-L188\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/322#issuecomment-968401459_", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/338/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1053136495, "node_id": "I_kwDOCGYnMM4-xZZv", "number": 341, "title": "`hash_id: Optional[Any]` should be `hash_id: Optional[str]`", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2021-11-15T02:12:39Z", "updated_at": "2021-11-15T02:19:31Z", "closed_at": "2021-11-15T02:19:31Z", "author_association": "OWNER", "pull_request": null, "body": "In a few places:\r\n\r\nhttps://github.com/simonw/sqlite-utils/blob/54a2269e91ce72b059618662ed133a85f3d42e4a/sqlite_utils/db.py#L642\r\n\r\nhttps://github.com/simonw/sqlite-utils/blob/54a2269e91ce72b059618662ed133a85f3d42e4a/sqlite_utils/db.py#L751\r\n\r\nhttps://github.com/simonw/sqlite-utils/blob/54a2269e91ce72b059618662ed133a85f3d42e4a/sqlite_utils/db.py#L1049\r\n\r\nhttps://github.com/simonw/sqlite-utils/blob/54a2269e91ce72b059618662ed133a85f3d42e4a/sqlite_utils/db.py#L1230\r\n\r\nBut it's correct here:\r\n\r\nhttps://github.com/simonw/sqlite-utils/blob/54a2269e91ce72b059618662ed133a85f3d42e4a/sqlite_utils/db.py#L2470", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/341/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1090798237, "node_id": "I_kwDOCGYnMM5BBEKd", "number": 359, "title": "Use RETURNING if available to populate last_pk", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2021-12-29T23:43:23Z", "updated_at": "2021-12-29T23:43:23Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "Inspired by this: https://news.ycombinator.com/item?id=29729283\r\n\r\n> Because SQLite is effectively serializing all the writes for us, we have zero locking in our code. We used to have to lock when inserting new items (to get the LastInsertRowId), but the newer version of SQLite supports the RETURNING keyword, so we don't even have to lock on inserts now.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/359/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1094974713, "node_id": "I_kwDOCGYnMM5BQ_z5", "number": 362, "title": "upsert --detect-types is broken", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2022-01-06T05:12:10Z", "updated_at": "2022-01-06T06:54:45Z", "closed_at": "2022-01-06T06:28:34Z", "author_association": "OWNER", "pull_request": null, "body": "Noticed this thanks to syntax highlighting in VS Code showing an unused variable - need to fix it and add a test.\r\n\r\n\r\n\r\n", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/362/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1097436959, "node_id": "I_kwDOCGYnMM5BaY8f", "number": 376, "title": "`--nl` mode should ignore blank lines", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 7558727, "label": "3.21"}, "comments": 0, "created_at": "2022-01-10T04:10:54Z", "updated_at": "2022-01-10T19:27:41Z", "closed_at": "2022-01-10T04:12:46Z", "author_association": "OWNER", "pull_request": null, "body": "Spotted this while manually testing #364 - there's no reason `--nl` should crash if you feed it an empty line in between JSON objects.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/376/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1098309897, "node_id": "I_kwDOCGYnMM5BduEJ", "number": 378, "title": "analyze=True parameter for some methods", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 7558727, "label": "3.21"}, "comments": 0, "created_at": "2022-01-10T19:54:52Z", "updated_at": "2022-01-11T01:08:11Z", "closed_at": "2022-01-11T01:08:09Z", "author_association": "OWNER", "pull_request": null, "body": "This would cause `ANALYZE` to be run against the relevant table at the end of executing the method.\r\n\r\n> Having browsed the API reference I think the methods that would benefit from an `analyze=True` parameter are:\r\n\r\n- [x] `table.create_index`\r\n- [x] `table.insert_all`\r\n- [x] `table.upsert_all`\r\n- [x] `table.delete_where`\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/366#issuecomment-1009288898_", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/378/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1098544628, "node_id": "I_kwDOCGYnMM5BenX0", "number": 379, "title": "CLI options for running ANALYZE", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 7558727, "label": "3.21"}, "comments": 0, "created_at": "2022-01-11T01:09:16Z", "updated_at": "2022-01-11T01:38:01Z", "closed_at": "2022-01-11T01:36:48Z", "author_association": "OWNER", "pull_request": null, "body": "> The Python methods are all done now, next step is the CLI options. I'll do those in a separate issue.\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/366#issuecomment-1009508865_\r\n\r\n- [x] `sqlite-utils analyze` command\r\n- [x] `sqlite-utils create-index --analyze` option (see #365)\r\n- [x] `sqlite-utils insert --analyze` option\r\n- [x] `sqlite-utils upsert --analyze` option\r\n\r\nIn #378 I also added `.delete_where(..., analyze=True)` but there isn't currently a `sqlite-utils delete-where` CLI command - deletions via CLI are expected to be handled using SQL queries.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/379/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1099897648, "node_id": "I_kwDOCGYnMM5Bjxsw", "number": 384, "title": "Add examples to every `--help`", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2022-01-12T05:31:25Z", "updated_at": "2022-01-26T03:15:02Z", "closed_at": "2022-01-26T03:15:02Z", "author_association": "OWNER", "pull_request": null, "body": "Everything on https://sqlite-utils.datasette.io/en/stable/cli-reference.html would benefit from an example.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/384/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1107557831, "node_id": "I_kwDOCGYnMM5CA_3H", "number": 386, "title": "Better \"contributing\" documentation", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2022-01-19T02:11:48Z", "updated_at": "2022-01-19T02:15:21Z", "closed_at": "2022-01-19T02:15:21Z", "author_association": "OWNER", "pull_request": null, "body": "This page jumps straight into running the tests: https://sqlite-utils.datasette.io/en/latest/contributing.html\r\n\r\nIt should add a little more about expected collaboration styles - opening an issue before filing a pull request - and probably link to https://simonwillison.net/2022/Jan/12/how-i-build-a-feature/", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/386/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1123851690, "node_id": "I_kwDOCGYnMM5C_J2q", "number": 396, "title": "mypy failure, sqlite_utils/utils.py:56", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2022-02-04T06:08:09Z", "updated_at": "2022-02-04T06:10:33Z", "closed_at": "2022-02-04T06:10:33Z", "author_association": "OWNER", "pull_request": null, "body": "https://github.com/simonw/sqlite-utils/runs/5062725880?check_suite_focus=true\r\n\r\n> `sqlite_utils/utils.py:56: error: Incompatible return value type (got \"None\", expected \"str\")`", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/396/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1178456794, "node_id": "I_kwDOCGYnMM5GPdLa", "number": 418, "title": "Add generated files to .gitignore", "user": {"value": 25778, "label": "eyeseast"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2022-03-23T17:48:12Z", "updated_at": "2022-03-24T21:01:44Z", "closed_at": "2022-03-24T21:01:44Z", "author_association": "CONTRIBUTOR", "pull_request": null, "body": "I end up with these in my local directory:\r\n\r\n\t.hypothesis/\r\n\tPipfile\r\n\tPipfile.lock\r\n\tpyproject.toml\r\n\r\nMight as well gitignore them.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/418/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1243715381, "node_id": "I_kwDOCGYnMM5KIZc1", "number": 436, "title": "Add \"copy to clipboard\" button to code examples in documentation", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2022-05-20T21:53:23Z", "updated_at": "2022-05-20T21:57:53Z", "closed_at": "2022-05-20T21:57:53Z", "author_association": "OWNER", "pull_request": null, "body": "Follows:\r\n- #435\r\n\r\nImitates:\r\n- https://github.com/simonw/datasette/issues/1748\r\n\r\nI'll use https://github.com/executablebooks/sphinx-copybutton - here's the Datasette commit: https://github.com/simonw/datasette/commit/1465fea4798599eccfe7e8f012bd8d9adfac3039", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/436/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1324659241, "node_id": "I_kwDOCGYnMM5O9LIp", "number": 459, "title": "Single quoted transform recipes on Windows do not work as expected ", "user": {"value": 19921, "label": "shakeel"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2022-08-01T16:14:54Z", "updated_at": "2022-08-01T16:14:54Z", "closed_at": null, "author_association": "CONTRIBUTOR", "pull_request": null, "body": "Trying to follow the tutorial for sqlite-utils and datasette https://datasette.io/tutorials/clean-data on Windows 11 OS `Microsoft Windows [Version 10.0.22622.440]`, with sqlite-utils and datasette installed using pipx.\r\n\r\n```\r\npipx list\r\npackage datasette 0.61.1, installed using Python 3.10.4\r\n - datasette.exe\r\npackage sqlite-utils 3.28, installed using Python 3.10.4\r\n - sqlite-utils.exe\r\n``` \r\n\r\nIn the step to transform dates into ISO dates the quoted value `'r.parsedatetime(value)'` is copied verbatim into the columns instead of applying the output of the Python recipe.\r\n\r\n```\r\nsqlite-utils convert manatees.db locations \\\r\n REPDATE created_date last_edited_date \\\r\n 'r.parsedatetime(value)' --dry-run\r\n\r\n1975/01/31 00:00:00+00\r\n --- becomes:\r\nr.parsedatetime(value)\r\n\r\nWould affect 13568 rows\r\n```\r\n\r\nHowever, if I change the code from single quotes to double quotes, it works as expected.\r\n\r\n```\r\nsqlite-utils convert manatees.db locations \\\r\n REPDATE created_date last_edited_date \\\r\n \"r.parsedatetime(value)\" --dry-run\r\n\r\n1975/01/31 00:00:00+00\r\n --- becomes:\r\n1975-01-31T00:00:00+00:00\r\n\r\nWould affect 13568 rows\r\n```\r\n\r\nSpecifying the transform code recipe should work with single quotes on Windows.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/459/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1342374388, "node_id": "PR_kwDOCGYnMM49Wv9T", "number": 466, "title": "Use Read the Docs action v1 (#463)", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2022-08-17T23:11:50Z", "updated_at": "2022-08-17T23:11:54Z", "closed_at": "2022-08-17T23:11:54Z", "author_association": "OWNER", "pull_request": "simonw/sqlite-utils/pulls/466", "body": "Read the Docs repository was renamed from `readthedocs/readthedocs-preview` to `readthedocs/actions/`. Now, the `preview` action is under `readthedocs/actions/preview` and is tagged as `v1`", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/466/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null} {"id": 1355193529, "node_id": "I_kwDOCGYnMM5Qxpy5", "number": 479, "title": "OperationalError: cannot VACUUM from within a transaction", "user": {"value": 7908073, "label": "chapmanjacobd"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2022-08-30T05:34:24Z", "updated_at": "2022-08-30T05:34:24Z", "closed_at": null, "author_association": "CONTRIBUTOR", "pull_request": null, "body": "Maybe when calling `.vacuum()` and other DB-level write-lock operations `sqlite_utils` could guard against this error message by automatically committing first?\r\n\r\n```\r\n 46 db[\"media\"].optimize() # type: ignore\r\n---> 47 db.vacuum()\r\n\r\nFile ~/.local/lib/python3.10/site-packages/sqlite_utils/db.py:1047, in Database.vacuum(self)\r\n 1045 def vacuum(self):\r\n 1046 \"Run a SQLite ``VACUUM`` against the database.\"\r\n-> 1047 self.execute(\"VACUUM;\")\r\n\r\nFile ~/.local/lib/python3.10/site-packages/sqlite_utils/db.py:470, in Database.execute(self, sql, parameters)\r\n 468 return self.conn.execute(sql, parameters)\r\n 469 else:\r\n--> 470 return self.conn.execute(sql)\r\n\r\nOperationalError: cannot VACUUM from within a transaction\r\n```\r\n\r\nIt might also be nice to add a sentence or two about how transactions are committed on the [docs page](https://sqlite-utils.datasette.io/en/latest/python-api.html#detect-fts). When I was swapping out my sqlite3 code for this library it was nice that everything was pretty much drop-in but I was/am unsure what to do about the places I explicitly call `.commit()` in my code\r\n\r\nRelated to https://github.com/simonw/sqlite-utils/issues/121", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/479/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1359604075, "node_id": "I_kwDOCGYnMM5RCelr", "number": 481, "title": "Idea: `sqlite-utils create-table tablename --sql \"select ...\"`", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2022-09-02T01:41:24Z", "updated_at": "2022-09-02T01:42:08Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "Could offer syntactic sugar for:\r\n\r\n```sql\r\ncreate table foo as select * from bar\r\n```\r\n\r\n```\r\nsqlite-utils create-table data.db foo --sql \"select * from bar\"\r\n```\r\nhttps://sqlite-utils.datasette.io/en/stable/cli-reference.html#create-table", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/481/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1432377191, "node_id": "I_kwDOCGYnMM5VYFdn", "number": 509, "title": "`sqlite-utils transform` breaks DEFAULT string values and STRFTIME()", "user": {"value": 2199875, "label": "kennysong"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2022-11-02T02:32:23Z", "updated_at": "2023-05-08T21:13:38Z", "closed_at": "2023-05-08T21:13:38Z", "author_association": "NONE", "pull_request": null, "body": "Very nice library! Our team found sqlite-utils through @simonw's [comment on the \"Simple declarative schema migration for SQLite\" article](https://news.ycombinator.com/item?id=31249823), and we were excited to use it, but unfortunately `sqlite-utils transform` seems to break our DB. \r\n\r\nRunning `sqlite-utils transform` to modify a column mangles their DEFAULT values:\r\n\r\n- Default string values are wrapped in extra single quotes\r\n- Function expressions such as [`STRFTIME()`](https://www.sqlite.org/lang_datefunc.html) are turned into strings!\r\n\r\n------\r\n\r\nHere are steps to reproduce:\r\n\r\n**Original database**\r\n\r\n```\r\n$ sqlite3 test.db << EOF\r\nCREATE TABLE mytable (\r\n col1 TEXT DEFAULT 'foo',\r\n col2 TEXT DEFAULT (STRFTIME('%Y-%m-%d %H:%M:%f', 'NOW'))\r\n)\r\nEOF\r\n\r\n$ sqlite3 test.db \"SELECT sql FROM sqlite_master WHERE name = 'mytable';\"\r\nCREATE TABLE mytable (\r\n col1 TEXT DEFAULT 'foo',\r\n col2 TEXT DEFAULT (STRFTIME('%Y-%m-%d %H:%M:%f', 'NOW'))\r\n)\r\n```\r\n\r\n**Modified database after sqlite-utils**\r\n\r\n```\r\n$ sqlite3 test.db \"INSERT INTO mytable DEFAULT VALUES; SELECT * FROM mytable;\"\r\nfoo|2022-11-02 02:26:58.038\r\n\r\n$ sqlite-utils transform test.db mytable --rename col1 renamedcol1\r\n\r\n$ sqlite3 test.db \"SELECT sql FROM sqlite_master WHERE name = 'mytable';\"\r\nCREATE TABLE \"mytable\" (\r\n [renamedcol1] TEXT DEFAULT '''foo''',\r\n [col2] TEXT DEFAULT 'STRFTIME(''%Y-%m-%d %H:%M:%f'', ''NOW'')'\r\n)\r\n\r\n$ sqlite3 test.db \"INSERT INTO mytable DEFAULT VALUES; SELECT * FROM mytable;\"\r\nfoo|2022-11-02 02:26:58.038\r\n'foo'|STRFTIME('%Y-%m-%d %H:%M:%f', 'NOW')\r\n```\r\n\r\n(Related: #336)", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/509/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1453134846, "node_id": "I_kwDOCGYnMM5WnRP-", "number": 513, "title": "Add or document streamlined workflow for importing Datasette csv / json exports", "user": {"value": 19328961, "label": "henry501"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2022-11-17T10:54:47Z", "updated_at": "2022-11-17T10:54:47Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "I'm working on some small front-end enhancements to the laion-aesthetic-datasette project, and I wanted to partially populate a database directly using exports from the existing Datasette instance instead of downloading the parquet files and creating my own multi-GB database.\r\n\r\nThere have been a number of small issues that are certainly related to my relative lack of familiarity with the toolkit, but that are still surprising. \r\n\r\nFor example: a CSV export of the images table (http://laion-aesthetic.datasette.io/laion-aesthetic-6pls.csv?sql=select+rowid%2C+url%2C+text%2C+domain_id%2C+width%2C+height%2C+similarity%2C+punsafe%2C+pwatermark%2C+aesthetic%2C+hash%2C+__index_level_0__+from+images+order+by+random%28%29+limit+100) has nested single quotes, double quotes, and commas that aren't handled by rows_from_file. Similarly, the json output has to be manually transformed to add the column names and remove extraneous information before sqlite_utils can import it.\r\n\r\nI was able to work through these issues, but as an enhancement it would be really helpful to create or document a clear workflow that avoids the friction of this data transformation.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/513/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1487764628, "node_id": "I_kwDOCGYnMM5YrXyU", "number": 518, "title": "flake8 ValueError: Error code '#' supplied to 'extend-ignore' option...", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2022-12-10T01:30:24Z", "updated_at": "2022-12-10T01:36:46Z", "closed_at": "2022-12-10T01:36:46Z", "author_association": "OWNER", "pull_request": null, "body": "> `Error code '#' supplied to 'extend-ignore' option does not match '^[A-Z]{1,3}[0-9]{0,3}$'`\r\n\r\nhttps://github.com/simonw/sqlite-utils/actions/runs/3662011265/jobs/6190770361\r\n\r\nI think from this:\r\n\r\nhttps://github.com/simonw/sqlite-utils/blob/e660635cea6c32f4022818380b1e1ee88e7c93a6/setup.cfg#L1-L3\r\n", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/518/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1550536442, "node_id": "I_kwDOCGYnMM5ca076", "number": 521, "title": "Custom JSON encoder", "user": {"value": 31504, "label": "janrito"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2023-01-20T09:19:40Z", "updated_at": "2023-01-20T09:19:40Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "It would be nice if we could specify a custom encoder (and decoder) for types that will need extra deserialisation \u2013 e.g., sets, enums or sparse matrices \u2013 or even project-specific types", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/521/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1553425465, "node_id": "I_kwDOCGYnMM5cl2Q5", "number": 522, "title": "Add COLUMN_TYPE_MAPPING for timedelta", "user": {"value": 81377, "label": "maport"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2023-01-23T16:49:54Z", "updated_at": "2023-11-04T00:49:51Z", "closed_at": "2023-11-04T00:49:51Z", "author_association": "NONE", "pull_request": null, "body": "Currently trying to create a column with Python type `datetime.timedelta` results in an error:\r\n\r\n```\r\n>>> from sqlite_utils import Database\r\n>>> db = Database(\"test.db\")\r\n>>> test_tbl = db['test']\r\n>>> test_tbl.insert({'col1': datetime.timedelta()})\r\nTraceback (most recent call last):\r\n File \"\", line 1, in \r\n File \"/usr/local/lib/python3.10/dist-packages/sqlite_utils/db.py\", line 2979, in insert\r\n return self.insert_all(\r\n File \"/usr/local/lib/python3.10/dist-packages/sqlite_utils/db.py\", line 3082, in insert_all\r\n self.create(\r\n File \"/usr/local/lib/python3.10/dist-packages/sqlite_utils/db.py\", line 1574, in create\r\n self.db.create_table(\r\n File \"/usr/local/lib/python3.10/dist-packages/sqlite_utils/db.py\", line 961, in create_table\r\n sql = self.create_table_sql(\r\n File \"/usr/local/lib/python3.10/dist-packages/sqlite_utils/db.py\", line 852, in create_table_sql\r\n column_type=COLUMN_TYPE_MAPPING[column_type],\r\nKeyError: \r\n```\r\n\r\nThe reason this would be useful is that `MySQLdb` uses `timedelta` for MySQL `TIME` columns:\r\n\r\n```\r\n>>> import MySQLdb\r\n>>> conn = MySQLdb.connect(host='database', user='user', passwd='pw')\r\n>>> csr = conn.cursor()\r\n>>> csr.execute(\"SELECT CAST('11:20' AS TIME)\")\r\n>>> tuple(csr)\r\n((datetime.timedelta(seconds=40800),),)\r\n```\r\n\r\nSo currently any attempt to convert a MySQL DB with a `TIME` column using `db-to-sqlite` will result in the above error.\r\n\r\nI was rather surprised that `MySQLdb` uses `timedelta` for `TIME` columns but I see that [this column type](https://dev.mysql.com/doc/refman/8.0/en/time.html) is intended for time intervals as well as the time of day so it makes sense. \r\n\r\n", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/522/reactions\", \"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1576990618, "node_id": "PR_kwDOCGYnMM5JkkED", "number": 526, "title": "Fix repeated calls to `Table.convert()`", "user": {"value": 167893, "label": "mcarpenter"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2023-02-09T00:14:49Z", "updated_at": "2023-05-08T21:56:05Z", "closed_at": "2023-05-08T21:53:58Z", "author_association": "CONTRIBUTOR", "pull_request": "simonw/sqlite-utils/pulls/526", "body": "Fixes #525. All tests pass.\r\n\r\nThere's perhaps a better way to name lambdas? There could be a collision if a caller passes a function with name like `lambda_123456`.\r\n\r\nSQLite [documentation](https://www.sqlite.org/appfunc.html) is a little, ah, lite on function name specs. If there is a character that can be used in place of underscore in a SQLite function name that is not permitted in a Python function identifier then that could be a good way to prevent accidental collisions. (I tried dash, colon, dot, no joy).\r\n\r\nOtherwise, there is little chance of this happening and if it should happen the risk is mitigated by now throwing an exception in the case of a (name, arity) collision without `replace=True`.\r\n\r\n\r\n----\r\n:books: Documentation preview :books:: https://sqlite-utils--526.org.readthedocs.build/en/526/\r\n\r\n", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/526/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null} {"id": 1701018909, "node_id": "I_kwDOCGYnMM5lY30d", "number": 543, "title": "Tests broken on Windows due to new convert() lambda names", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2023-05-08T22:11:29Z", "updated_at": "2023-05-08T22:19:04Z", "closed_at": "2023-05-08T22:19:04Z", "author_association": "OWNER", "pull_request": null, "body": "https://github.com/simonw/sqlite-utils/actions/runs/4920084038/jobs/8788501314\r\n```python\r\nsql = 'update [example] set [dt] = lambda_-9223371942137158589([dt]);'\r\n```\r\nFrom:\r\n- #526", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/543/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1754174496, "node_id": "I_kwDOCGYnMM5ojpQg", "number": 558, "title": "Ability to define unique columns when creating a table", "user": {"value": 1910303, "label": "aguinane"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2023-06-13T06:56:19Z", "updated_at": "2023-08-18T01:06:03Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "When creating a new table, it would be good to have an option to set unique columns similar to how not_null is set.\r\n\r\n```python\r\nfrom sqlite_utils import Database\r\n\r\ncolumns = {\"mRID\": str, \"name\": str}\r\ndb = Database(\"example.db\")\r\ndb[\"ExampleTable\"].create(columns, pk=\"mRID\", not_null=[\"mRID\"], if_not_exists=True)\r\ndb[\"ExampleTable\"].create_index([\"mRID\"], unique=True, if_not_exists=True)\r\n```\r\n\r\nSo something like this would add the UNIQUE flag to the table definition. \r\n\r\n```python\r\ndb[\"ExampleTable\"].create(columns, pk=\"mRID\", not_null=[\"mRID\"], unique=[\"mRID\"], if_not_exists=True)\r\n```\r\n\r\n```sql\r\nCREATE TABLE ExampleTable (\r\n mRID TEXT PRIMARY KEY\r\n NOT NULL\r\n UNIQUE,\r\n name TEXT\r\n);\r\n```", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/558/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1773450152, "node_id": "I_kwDOCGYnMM5ptLOo", "number": 559, "title": "sqlean support", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2023-06-25T19:27:26Z", "updated_at": "2023-06-25T23:25:53Z", "closed_at": "2023-06-25T23:25:53Z", "author_association": "OWNER", "pull_request": null, "body": "If sqlean is available, use that.\r\n\r\nRefs:\r\n- https://github.com/nalgeon/sqlean.py/issues/1#issuecomment-1605707788\r\n\r\nThis will provide a good workaround for:\r\n- #235 ", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/559/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1786243905, "node_id": "I_kwDOCGYnMM5qd-tB", "number": 564, "title": "Document that running `db.transform()` tidies up the schema indentation", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2023-07-03T13:59:28Z", "updated_at": "2023-07-22T22:15:34Z", "closed_at": "2023-07-22T22:15:34Z", "author_association": "OWNER", "pull_request": null, "body": "> ... and it turns out running `.transform()` with no arguments still fixes the format of the schema!\r\n\r\n```pycon\r\n>>> db[\"log\"].add_column(\"foo\", str)\r\n\r\n>>> db[\"log\"].add_column(\"bar\", str)\r\n
\r\n>>> db[\"log\"].add_column(\"baz\", str)\r\n
\r\n>>> print(db[\"log\"].schema)\r\nCREATE TABLE \"log\" (\r\n [id] INTEGER PRIMARY KEY,\r\n [name2] TEXT,\r\n [age] INTEGER,\r\n [weight] FLOAT\r\n, [foo] TEXT, [bar] TEXT, [baz] TEXT)\r\n>>> db[\"log\"].transform()\r\n
\r\n>>> print(db[\"log\"].schema)\r\nCREATE TABLE \"log\" (\r\n [id] INTEGER PRIMARY KEY,\r\n [name2] TEXT,\r\n [age] INTEGER,\r\n [weight] FLOAT,\r\n [foo] TEXT,\r\n [bar] TEXT,\r\n [baz] TEXT\r\n)\r\n```\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/llm/issues/65#issuecomment-1618347727_\r\n ", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/564/reactions\", \"total_count\": 1, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 1}", "draft": null, "state_reason": "completed"} {"id": 1816857105, "node_id": "I_kwDOCGYnMM5sSwoR", "number": 570, "title": "`sqlite-utils install -e` option", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2023-07-22T18:32:23Z", "updated_at": "2023-07-22T18:55:59Z", "closed_at": "2023-07-22T18:32:56Z", "author_association": "OWNER", "pull_request": null, "body": "As seen in LLM.\r\n\r\nNeeded while working on:\r\n- #567", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/570/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1821108702, "node_id": "I_kwDOCGYnMM5si-ne", "number": 579, "title": "Special handling for SQLite column of type `JSON`", "user": {"value": 15178711, "label": "asg017"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2023-07-25T20:37:23Z", "updated_at": "2023-07-25T20:37:23Z", "closed_at": null, "author_association": "CONTRIBUTOR", "pull_request": null, "body": "`sqlite-utils` should detect and have specially handling for column with a `JSON` column. For example:\r\n\r\n```sql\r\nCREATE TABLE \"dogs\" (\r\n id INTEGER PRIMARY KEY,\r\n name TEXT,\r\n friends JSON \r\n);\r\n```\r\n\r\n## Automatic Nesting\r\n\r\nAccording to [\"Nested JSON Values\"](https://sqlite-utils.datasette.io/en/stable/cli.html#nested-json-values), sqlite-utils will only expand JSON if the `--json-cols` flag is passed. It looks like it'll try to `json.load` all text column to test if its JSON, which can get expensive on non-json columns. \r\n\r\nInstead, `sqlite-utils` should be default (ie without the `--json-cols` flags) do the `maybe_json()` operation on columns with a declared `JSON` type. So the above table would expand the `\"friends\"` column as expected, withoutthe `--json-cols` flag:\r\n\r\n```bash\r\nsqlite-utils dogs.db \"select * from dogs\" | python -mjson.tool\r\n```\r\n\r\n```\r\n[\r\n {\r\n \"id\": 1,\r\n \"name\": \"Cleo\",\r\n \"friends\": [\r\n {\r\n \"name\": \"Pancakes\"\r\n },\r\n {\r\n \"name\": \"Bailey\"\r\n }\r\n ]\r\n }\r\n]\r\n```\r\n\r\n---\r\n\r\nI'm sure there's other ways `sqlite-utils` can specially handle JSON columns, so keeping this open while I think of more", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/579/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1822918995, "node_id": "I_kwDOCGYnMM5sp4lT", "number": 580, "title": "Add way to export to a csv file using the Python library", "user": {"value": 44324811, "label": "kevinlinxc"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2023-07-26T18:09:26Z", "updated_at": "2023-07-26T18:09:26Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "According to the documentation, we can make a csv output using the CLI tool, but not the Python library. Could we have the latter?", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/580/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1839344979, "node_id": "I_kwDOCGYnMM5toi1T", "number": 582, "title": "Handling CSV/file input that contains NUL bytes", "user": {"value": 1448859, "label": "betatim"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2023-08-07T12:24:14Z", "updated_at": "2023-08-07T12:24:14Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "I was using sqlite-utils to create a DB from a CSV and it turns out the CSV contains a NUL byte.\r\n\r\nWhen the processing reaches the line that contains the NUL an exception is raised.\r\n\r\nI'm wondering if there is something that can be done in `sqlite-utils` to say \"skip lines with encoding errors\" or some such. I think it isn't super straightforward though as the exception comes from inside the `csv` module that does all the parsing.\r\n\r\nConcretely the file is the `KernelVersions.csv` from https://www.kaggle.com/datasets/kaggle/meta-kaggle\r\n\r\nThis is the command and output:\r\n```\r\n$ sqlite-utils insert --csv kaggle.db kaggle KernelVersions.csv\r\n [------------------------------------] 0%\r\n [#####################---------------] 60% 00:04:24Traceback (most recent call last):\r\n File \"/home/foobar/miniconda/envs/meta-kaggle/bin/sqlite-utils\", line 10, in \r\n sys.exit(cli())\r\n File \"/home/foobar/miniconda/envs/meta-kaggle/lib/python3.10/site-packages/click/core.py\", line 1128, in __call__\r\n return self.main(*args, **kwargs)\r\n File \"/home/foobar/miniconda/envs/meta-kaggle/lib/python3.10/site-packages/click/core.py\", line 1053, in main\r\n rv = self.invoke(ctx)\r\n File \"/home/foobar/miniconda/envs/meta-kaggle/lib/python3.10/site-packages/click/core.py\", line 1659, in invoke\r\n return _process_result(sub_ctx.command.invoke(sub_ctx))\r\n File \"/home/foobar/miniconda/envs/meta-kaggle/lib/python3.10/site-packages/click/core.py\", line 1395, in invoke\r\n return ctx.invoke(self.callback, **ctx.params)\r\n File \"/home/foobar/miniconda/envs/meta-kaggle/lib/python3.10/site-packages/click/core.py\", line 754, in invoke\r\n return __callback(*args, **kwargs)\r\n File \"/home/foobar/miniconda/envs/meta-kaggle/lib/python3.10/site-packages/sqlite_utils/cli.py\", line 1223, in insert\r\n insert_upsert_implementation(\r\n File \"/home/foobar/miniconda/envs/meta-kaggle/lib/python3.10/site-packages/sqlite_utils/cli.py\", line 1085, in insert_upsert_implementation\r\n db[table].insert_all(\r\n File \"/home/foobar/miniconda/envs/meta-kaggle/lib/python3.10/site-packages/sqlite_utils/db.py\", line 3198, in insert_all\r\n chunk = list(chunk)\r\n File \"/home/foobar/miniconda/envs/meta-kaggle/lib/python3.10/site-packages/sqlite_utils/db.py\", line 3742, in fix_square_braces\r\n for record in records:\r\n File \"/home/foobar/miniconda/envs/meta-kaggle/lib/python3.10/site-packages/sqlite_utils/cli.py\", line 1071, in \r\n docs = (decode_base64_values(doc) for doc in docs)\r\n File \"/home/foobar/miniconda/envs/meta-kaggle/lib/python3.10/site-packages/sqlite_utils/cli.py\", line 1068, in \r\n docs = (verify_is_dict(doc) for doc in docs)\r\n File \"/home/foobar/miniconda/envs/meta-kaggle/lib/python3.10/site-packages/sqlite_utils/cli.py\", line 1003, in \r\n docs = (dict(zip(headers, row)) for row in reader)\r\n_csv.Error: line contains NUL\r\n```", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/582/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1920416843, "node_id": "I_kwDOCGYnMM5ydzxL", "number": 597, "title": "sqlite-utils insert-files should be able to convert fields", "user": {"value": 1737541, "label": "grimnight"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2023-09-30T22:20:47Z", "updated_at": "2023-09-30T22:20:47Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "Currently using both `insert-files` and `convert` is needed in order to create sqlar files, it would be more convenient if it could be done with just one command.\r\n\r\n```shell\r\n~\r\n\u276f cat test.py\r\nimport os\r\n\r\nclass Example:\r\n def __init__(self, arg1, arg2):\r\n self.arg1 = arg1\r\n\r\n~\r\n\u276f sqlite-utils insert-files test.sqlar sqlar test.py -c name:name -c data:content -c mode:mode -c mtime:mtime -c sz:size --pk=name\r\n [####################################] 100%\r\n\r\n~\r\n\u276f sqlite-utils convert test.sqlar sqlar data \"zlib.compress(value)\" --import=zlib --where \"name = 'test.py'\"\r\n[####################################] 100%\r\n\r\n~\r\n\u276f cat test.py | sqlite-utils convert test.sqlar sqlar data \"zlib.compress(sys.stdin.buffer.read())\" --import=zlib --import=sys --where \"name = 'test.py'\" # Alternative way\r\n [####################################] 100%\r\n\r\n~\r\n\u276f sqlite3 test.sqlar \"SELECT hex(data) FROM sqlar WHERE name = 'test.py';\" | python3 -c \"import sys, zlib; sys.stdout.buffer.write(zlib.decompress(bytes.fromhex(sys.stdin.read())))\"\r\nimport os\r\n\r\nclass Example:\r\n def __init__(self, arg1, arg2):\r\n self.arg1 = arg1\r\n\r\n~\r\n\u276f rm test.py\r\n\r\n~\r\n\u276f sqlar -l test.sqlar\r\ntest.py\r\n\r\n~\r\n\u276f sqlar -x test.sqlar\r\n\r\n~\r\n\u276f cat test.py\r\nimport os\r\n\r\nclass Example:\r\n def __init__(self, arg1, arg2):\r\n self.arg1 = arg1\r\n\r\n```", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/597/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1977155641, "node_id": "I_kwDOCGYnMM512QA5", "number": 601, "title": "Move plugin directory into documentation", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2023-11-04T04:07:52Z", "updated_at": "2023-11-04T04:07:52Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "https://github.com/simonw/sqlite-utils-plugins should be in the official documentation.\r\n\r\nI can use the same pattern as https://llm.datasette.io/en/stable/plugins/directory.html\r\n\r\nhttps://til.simonwillison.net/readthedocs/stable-docs", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/601/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1978603203, "node_id": "I_kwDOCGYnMM517xbD", "number": 602, "title": "`sqlite-utils transform` removes the `AUTOINCREMENT` keyword", "user": {"value": 4472046, "label": "ArsTapatun"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2023-11-06T08:48:43Z", "updated_at": "2023-11-06T08:48:43Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "### Context\r\n\r\nWe ran into this bug randomly, noticing that deleted `ROWID` would get reused after migrating the DB. Using `transform` to change any column in the table will also unexpectedly strip away the `AUTOINCREMENT` keyword from the primary key definition, even if it was not the transformation target.\r\n\r\n### Reproducible example\r\n\r\n**Original database**\r\n\r\n```sql\r\n$ sqlite3 test.db << EOF\r\nCREATE TABLE mytable (\r\n col1 INTEGER PRIMARY KEY AUTOINCREMENT,\r\n col2 TEXT NOT NULL\r\n)\r\nEOF\r\n\r\n$ sqlite3 test.db \".schema mytable\"\r\nCREATE TABLE mytable (\r\n col1 INTEGER PRIMARY KEY AUTOINCREMENT,\r\n col2 TEXT NOT NULL\r\n);\r\n```\r\n\r\n**Modified database after sqlite-utils**\r\n\r\n```sql\r\n$ sqlite-utils transform test.db mytable --rename col2 renamedcol2\r\n\r\n$ sqlite3 test.db \"SELECT sql FROM sqlite_master WHERE name = 'mytable';\"\r\nCREATE TABLE IF NOT EXISTS \"mytable\" (\r\n [col1] INTEGER PRIMARY KEY,\r\n [renamedcol2] TEXT NOT NULL\r\n);\r\n```", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/602/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 349850687, "node_id": "MDU6SXNzdWUzNDk4NTA2ODc=", "number": 2, "title": "Mechanism for adding foreign keys to an existing table", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2018-08-12T22:50:56Z", "updated_at": "2019-02-24T21:34:41Z", "closed_at": "2019-02-24T21:34:41Z", "author_association": "OWNER", "pull_request": null, "body": "SQLite does not have ALTER TABLE support for adding new foreign keys... but it turns out it's possible to make these changes without having to duplicate the entire table by carefully running `UPDATE sqlite_master SET sql=... WHERE type='table' AND name='X';`\r\n\r\nHere's how Django does it: https://github.com/django/django/blob/d3449faaa915a08c275b35de01e66a7ef6bdb2dc/django/db/backends/sqlite3/schema.py#L103-L125\r\n\r\nAnd here's the official documentation about this: https://sqlite.org/lang_altertable.html#otheralter (scroll to the very bottom of the page)", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/2/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 351845423, "node_id": "MDU6SXNzdWUzNTE4NDU0MjM=", "number": 3, "title": "Experiment with contentless FTS tables", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2018-08-18T19:31:01Z", "updated_at": "2019-07-22T20:58:55Z", "closed_at": "2019-07-22T20:58:55Z", "author_association": "OWNER", "pull_request": null, "body": "Could greatly reduce size of resulting database for large datasets: http://cocoamine.net/blog/2015/09/07/contentless-fts4-for-large-immutable-documents/", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/3/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}