{"html_url": "https://github.com/simonw/sqlite-utils/issues/66#issuecomment-553528386", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/66", "id": 553528386, "node_id": "MDEyOklzc3VlQ29tbWVudDU1MzUyODM4Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-11-13T18:03:10Z", "updated_at": "2019-11-13T18:03:54Z", "author_association": "OWNER", "body": "Maybe `inplace()` (combining \"insert\" and \"replace\")?\r\n\r\nIt could be an alias for `.insert(..., replace=True)`", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 521868864, "label": "The \".upsert()\" method is misnamed"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/66#issuecomment-553528850", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/66", "id": 553528850, "node_id": "MDEyOklzc3VlQ29tbWVudDU1MzUyODg1MA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-11-13T18:04:20Z", "updated_at": "2019-11-13T18:04:20Z", "author_association": "OWNER", "body": "This is going to affect the design of the CLI subcommands as well.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 521868864, "label": "The \".upsert()\" method is misnamed"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/66#issuecomment-553540146", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/66", "id": 553540146, "node_id": "MDEyOklzc3VlQ29tbWVudDU1MzU0MDE0Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-11-13T18:33:30Z", "updated_at": "2019-11-13T18:33:30Z", "author_association": "OWNER", "body": "Maybe instead of inventing a new term I should tell people to use `.insert(..., replace=True)` directly. That matches `ignore=True`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 521868864, "label": "The \".upsert()\" method is misnamed"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/66#issuecomment-553574011", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/66", "id": 553574011, "node_id": "MDEyOklzc3VlQ29tbWVudDU1MzU3NDAxMQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-11-13T19:53:45Z", "updated_at": "2019-11-13T19:53:45Z", "author_association": "OWNER", "body": "First step: add a `replace=True` argument to `insert()` and `insert_all()` that does the same thing as the current `upsert=True`\r\n\r\nhttps://github.com/simonw/sqlite-utils/blob/8dab9fd1ccf571e188eec9ccf606a0c50fccf200/sqlite_utils/db.py#L938-L946", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 521868864, "label": "The \".upsert()\" method is misnamed"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/66#issuecomment-554565198", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/66", "id": 554565198, "node_id": "MDEyOklzc3VlQ29tbWVudDU1NDU2NTE5OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-11-15T23:12:28Z", "updated_at": "2019-11-15T23:12:28Z", "author_association": "OWNER", "body": "Urgh this is going to be quite a bit of work, especially in the CLI module which shares an implementation for `upsert` and `insert` in a way that looks like it will have to be unwrapped.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 521868864, "label": "The \".upsert()\" method is misnamed"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/66#issuecomment-555690319", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/66", "id": 555690319, "node_id": "MDEyOklzc3VlQ29tbWVudDU1NTY5MDMxOQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-11-19T20:10:17Z", "updated_at": "2019-11-19T20:10:17Z", "author_association": "OWNER", "body": "Thinking about this further: I believe every time I've personally used `upsert` in the past (either with the Python library or the CLI tool) I've actually wanted the new behaviour, where \"upsert\" means \"update existing record with these changes, or insert a new record if one does not exist\".\r\n\r\nSo I'm happy with `upsert` doing that, and `insert --replace` being added as an option that does what `upsert` does ta the moment.\r\n\r\nI'll still ship it as version 2.0 since it's technically a breaking change.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 521868864, "label": "The \".upsert()\" method is misnamed"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/66#issuecomment-569131397", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/66", "id": 569131397, "node_id": "MDEyOklzc3VlQ29tbWVudDU2OTEzMTM5Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-12-26T20:49:11Z", "updated_at": "2019-12-26T20:49:11Z", "author_association": "OWNER", "body": "Don't forget to update the documentation. This will be quite an involved task.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 521868864, "label": "The \".upsert()\" method is misnamed"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/66#issuecomment-569226620", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/66", "id": 569226620, "node_id": "MDEyOklzc3VlQ29tbWVudDU2OTIyNjYyMA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-12-27T09:05:29Z", "updated_at": "2019-12-27T09:05:36Z", "author_association": "OWNER", "body": "I'm going to start by ignoring the existing `upsert` entirely and implementing `.insert(..., replace=True)` and `$ sqlite-utils insert --replace`. Including updating the tests.\r\n\r\nThen I'll figure out how to implement the new `.upsert()` / `$ sqlite-utils upsert`.\r\n\r\nThen I'll update the documentation, and ship `sqlite-utils` 2.0.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 521868864, "label": "The \".upsert()\" method is misnamed"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/66#issuecomment-569588216", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/66", "id": 569588216, "node_id": "MDEyOklzc3VlQ29tbWVudDU2OTU4ODIxNg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-12-30T05:31:45Z", "updated_at": "2019-12-30T05:31:45Z", "author_association": "OWNER", "body": "Last step: update changelog and ship 2.0. Then I can close this issue.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 521868864, "label": "The \".upsert()\" method is misnamed"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/66#issuecomment-569844426", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/66", "id": 569844426, "node_id": "MDEyOklzc3VlQ29tbWVudDU2OTg0NDQyNg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-12-31T01:30:20Z", "updated_at": "2019-12-31T01:30:20Z", "author_association": "OWNER", "body": "I shipped 2.0 - release notes here: https://sqlite-utils.readthedocs.io/en/stable/changelog.html#v2\r\n\r\nI also wrote about it on my blog: https://simonwillison.net/2019/Dec/30/sqlite-utils-2/", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 521868864, "label": "The \".upsert()\" method is misnamed"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/63#issuecomment-549430429", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/63", "id": 549430429, "node_id": "MDEyOklzc3VlQ29tbWVudDU0OTQzMDQyOQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-11-04T16:20:35Z", "updated_at": "2019-11-04T16:20:35Z", "author_association": "OWNER", "body": "I don't think we need this. We already have a `dogs.create_index([\"name\"], if_not_exists=True)` option.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 517241040, "label": "ensure_index() method"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/62#issuecomment-549418134", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/62", "id": 549418134, "node_id": "MDEyOklzc3VlQ29tbWVudDU0OTQxODEzNA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-11-04T15:54:18Z", "updated_at": "2019-11-04T15:54:18Z", "author_association": "OWNER", "body": "This is a good idea. The individual row version of it can work like the `.update(pk, ...)` method.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 500783373, "label": "[enhancement] Method to delete a row in python"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/62#issuecomment-549425012", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/62", "id": 549425012, "node_id": "MDEyOklzc3VlQ29tbWVudDU0OTQyNTAxMg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-11-04T16:09:08Z", "updated_at": "2019-11-04T16:09:08Z", "author_association": "OWNER", "body": "Documentation for `table.delete()`: https://github.com/simonw/sqlite-utils/blob/19073d6d972fad9d68dd74c28544cd29083f1c12/docs/python-api.rst#deleting-a-specific-record", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 500783373, "label": "[enhancement] Method to delete a row in python"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/62#issuecomment-549425364", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/62", "id": 549425364, "node_id": "MDEyOklzc3VlQ29tbWVudDU0OTQyNTM2NA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-11-04T16:09:56Z", "updated_at": "2019-11-04T16:09:56Z", "author_association": "OWNER", "body": "Since we have `table.rows_where(where, where_args)` it makes sense to me to also support `table.delete_where(where, where_args)`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 500783373, "label": "[enhancement] Method to delete a row in python"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/62#issuecomment-549429512", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/62", "id": 549429512, "node_id": "MDEyOklzc3VlQ29tbWVudDU0OTQyOTUxMg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-11-04T16:18:48Z", "updated_at": "2019-11-04T16:18:48Z", "author_association": "OWNER", "body": "Documentation for `.delete_where()`: https://github.com/simonw/sqlite-utils/blob/169ea455fc1f1d5e5b6e44cb339ba7ffa9d49c31/docs/python-api.rst#deleting-multiple-records", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 500783373, "label": "[enhancement] Method to delete a row in python"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/62#issuecomment-549435364", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/62", "id": 549435364, "node_id": "MDEyOklzc3VlQ29tbWVudDU0OTQzNTM2NA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-11-04T16:30:34Z", "updated_at": "2019-11-04T16:30:34Z", "author_association": "OWNER", "body": "Released as 1.12.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 500783373, "label": "[enhancement] Method to delete a row in python"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/61#issuecomment-533818697", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/61", "id": 533818697, "node_id": "MDEyOklzc3VlQ29tbWVudDUzMzgxODY5Nw==", "user": {"value": 49260, "label": "amjith"}, "created_at": "2019-09-21T18:09:01Z", "updated_at": "2019-09-21T18:09:28Z", "author_association": "CONTRIBUTOR", "body": "@witeshadow The library version doesn't have helpers around CSV (at least not from what I can see in the code). \r\n\r\nBut here's a snippet that makes it easy to insert from CSV using the library. \r\n\r\n```\r\nimport csv\r\nfrom sqlite_utils import Database\r\n\r\n# CSV Reader\r\n\r\ncsv_file = open(\"filename.csv\") # open the csv file.\r\nreader = csv.reader(csv_file) # Create a CSV reader\r\nheaders = next(reader) # First line is the header\r\ndocs = (dict(zip(headers, row)) for row in reader)\r\n\r\n# Now you can use the `sqlite_utils` library. \r\n\r\ndb = Database(\"my_database.db\")\r\ndb[\"table_name\"].insert_all(docs)\r\n```\r\n\r\nThis snippet is adapted from reading the CLI source code on how it implements the csv option.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 491219910, "label": "importing CSV to SQLite as library"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/61#issuecomment-549432592", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/61", "id": 549432592, "node_id": "MDEyOklzc3VlQ29tbWVudDU0OTQzMjU5Mg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-11-04T16:25:01Z", "updated_at": "2019-11-04T16:25:01Z", "author_association": "OWNER", "body": "Yeah I've thought about this a bit and I'm OK leaving it out.\r\n\r\nThe core idea of `sqlite-utils` is that if you can create a list, iterator or generator of Python dictionaries you can efficiently insert those into a SQLite table.\r\n\r\nThe `--csv` function is actually implemented as just a few lines of code which turn that incoming CSV into a generator of dictionaries:\r\n\r\nhttps://github.com/simonw/sqlite-utils/blob/169ea455fc1f1d5e5b6e44cb339ba7ffa9d49c31/sqlite_utils/cli.py#L364-L368\r\n\r\nI could turn this into a reusable helper function but since it wouldn't have anything to do with database inserts (it would just be a helper that turns a CSV into a generator of dictionaries) it doesn't feel like it fits well in the Python library section of this package.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 491219910, "label": "importing CSV to SQLite as library"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/606#issuecomment-1843465748", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/606", "id": 1843465748, "node_id": "IC_kwDOCGYnMM5t4Q4U", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-12-06T18:36:51Z", "updated_at": "2023-12-06T18:36:51Z", "author_association": "OWNER", "body": "I'll add `bytes` too - `float` already works. This makes sense because when you are working with the Python API you use `str` and `float` and `bytes` and `int` to specify column types.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 2029161033, "label": "str and int as aliases for text and integer"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/606#issuecomment-1843579184", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/606", "id": 1843579184, "node_id": "IC_kwDOCGYnMM5t4skw", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-12-06T19:43:55Z", "updated_at": "2023-12-06T19:43:55Z", "author_association": "OWNER", "body": "Updated documentation:\r\n- https://sqlite-utils.datasette.io/en/latest/cli.html#cli-add-column\r\n- https://sqlite-utils.datasette.io/en/latest/cli-reference.html#add-column", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 2029161033, "label": "str and int as aliases for text and integer"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/605#issuecomment-1846554637", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/605", "id": 1846554637, "node_id": "IC_kwDOCGYnMM5uEDAN", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-12-08T05:07:54Z", "updated_at": "2023-12-08T05:07:54Z", "author_association": "OWNER", "body": "Thanks for opening an issue - this should help future Google searchers figure out what's going on here.\r\n\r\nAnother approach here could be to store large integers as `TEXT` in SQLite (or even as `BLOB`).\r\n\r\nBoth storing as `REAL` and storing as `TEXT/BLOB` feel nasty to me, but it looks like SQLite has a hard upper limit of 9223372036854775807 for integers.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 2007893839, "label": "Insert fails with `Error: Python int too large to convert to SQLite INTEGER`; can we use `NUMERIC` here?"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/604#issuecomment-1843585454", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/604", "id": 1843585454, "node_id": "IC_kwDOCGYnMM5t4uGu", "user": {"value": 22429695, "label": "codecov[bot]"}, "created_at": "2023-12-06T19:48:26Z", "updated_at": "2023-12-08T05:05:03Z", "author_association": "NONE", "body": "## [Codecov](https://app.codecov.io/gh/simonw/sqlite-utils/pull/604?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report\nAll modified and coverable lines are covered by tests :white_check_mark:\n> Comparison is base [(`9286c1b`)](https://app.codecov.io/gh/simonw/sqlite-utils/commit/9286c1ba432e890b1bb4b2a1f847b15364c1fa18?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) 95.77% compared to head [(`1698a9d`)](https://app.codecov.io/gh/simonw/sqlite-utils/pull/604?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) 95.72%.\n> Report is 1 commits behind head on main.\n\n> :exclamation: Current head 1698a9d differs from pull request most recent head 61c6e26. Consider uploading reports for the commit 61c6e26 to get more accurate results\n\n\n
Additional details and impacted files\n\n\n```diff\n@@ Coverage Diff @@\n## main #604 +/- ##\n==========================================\n- Coverage 95.77% 95.72% -0.06% \n==========================================\n Files 8 8 \n Lines 2842 2852 +10 \n==========================================\n+ Hits 2722 2730 +8 \n- Misses 120 122 +2 \n```\n\n\n\n
\n\n[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/simonw/sqlite-utils/pull/604?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). \n:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 2001006157, "label": "Add more STRICT table support"}, "performed_via_github_app": "{\"id\": 254, \"slug\": \"codecov\", \"node_id\": \"MDM6QXBwMjU0\", \"owner\": {\"login\": \"codecov\", \"id\": 8226205, \"node_id\": \"MDEyOk9yZ2FuaXphdGlvbjgyMjYyMDU=\", \"avatar_url\": \"https://avatars.githubusercontent.com/u/8226205?v=4\", \"gravatar_id\": \"\", \"url\": \"https://api.github.com/users/codecov\", \"html_url\": \"https://github.com/codecov\", \"followers_url\": \"https://api.github.com/users/codecov/followers\", \"following_url\": \"https://api.github.com/users/codecov/following{/other_user}\", \"gists_url\": \"https://api.github.com/users/codecov/gists{/gist_id}\", \"starred_url\": \"https://api.github.com/users/codecov/starred{/owner}{/repo}\", \"subscriptions_url\": \"https://api.github.com/users/codecov/subscriptions\", \"organizations_url\": \"https://api.github.com/users/codecov/orgs\", \"repos_url\": \"https://api.github.com/users/codecov/repos\", \"events_url\": \"https://api.github.com/users/codecov/events{/privacy}\", \"received_events_url\": \"https://api.github.com/users/codecov/received_events\", \"type\": \"Organization\", \"site_admin\": false}, \"name\": \"Codecov\", \"description\": \"Codecov provides highly integrated tools to group, merge, archive and compare coverage reports. Whether your team is comparing changes in a pull request or reviewing a single commit, Codecov will improve the code review workflow and quality.\\r\\n\\r\\n## Code coverage done right.\u00ae\\r\\n\\r\\n1. Upload coverage reports from your CI builds.\\r\\n2. Codecov merges all builds and languages into one beautiful coherent report.\\r\\n3. Get commit statuses, pull request comments and coverage overlay via our browser extension.\\r\\n\\r\\nWhen Codecov merges your uploads it keeps track of the CI provider (inc. build details) and user specified context, e.g. `#unittest` ~ `#smoketest` or `#oldcode` ~ `#newcode`. You can track the `#unittest` coverage independently of other groups. [Learn more here](\\r\\nhttp://docs.codecov.io/docs/flags)\\r\\n\\r\\nThrough **Codecov's Browser Extension** reports overlay directly in GitHub UI to assist in code review in [Chrome](https://chrome.google.com/webstore/detail/codecov/gedikamndpbemklijjkncpnolildpbgo) or Firefox (https://addons.mozilla.org/en-US/firefox/addon/codecov/)\\r\\n\\r\\n*Highly detailed* **pull request comments** and *customizable* **commit statuses** will improve your team's workflow and code coverage incrementally.\\r\\n\\r\\n**File backed configuration** all through the `codecov.yml`. \\r\\n\\r\\n## FAQ\\r\\n- Do you **merge multiple uploads** to the same commit? **Yes**\\r\\n- Do you **support multiple languages** in the same project? **Yes**\\r\\n- Can you **group coverage reports** by project and/or test type? **Yes**\\r\\n- How does **pricing** work? Only paid users can view reports and post statuses/comments. \", \"external_url\": \"https://codecov.io\", \"html_url\": \"https://github.com/apps/codecov\", \"created_at\": \"2016-09-25T14:18:27Z\", \"updated_at\": \"2023-09-08T15:29:16Z\", \"permissions\": {\"administration\": \"read\", \"checks\": \"write\", \"contents\": \"read\", \"emails\": \"read\", \"issues\": \"read\", \"members\": \"read\", \"metadata\": \"read\", \"pull_requests\": \"write\", \"statuses\": \"write\"}, \"events\": [\"check_run\", \"check_suite\", \"create\", \"delete\", \"fork\", \"member\", \"membership\", \"organization\", \"public\", \"pull_request\", \"push\", \"release\", \"repository\", \"status\", \"team_add\"]}"} {"html_url": "https://github.com/simonw/sqlite-utils/pull/604#issuecomment-1843586503", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/604", "id": 1843586503, "node_id": "IC_kwDOCGYnMM5t4uXH", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-12-06T19:49:10Z", "updated_at": "2023-12-06T19:49:29Z", "author_association": "OWNER", "body": "This looks really great on first glance - design is good, implementation is solid, tests and documentation look great.\r\n\r\nLooks like a couple of `mypy` failures in the tests at the moment:\r\n\r\n```\r\n mypy sqlite_utils tests\r\n\r\nsqlite_utils/db.py:543: error: Incompatible types in assignment (expression has type \"type[Table]\", variable has type \"type[View]\") [assignment]\r\ntests/test_lookup.py:156: error: Name \"test_lookup_new_table\" already defined on line 5 [no-redef]\r\nFound 2 errors in 2 files (checked 54 source files)\r\nError: Process completed with exit code 1.\r\n```\r\n\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 2001006157, "label": "Add more STRICT table support"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/604#issuecomment-1843975536", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/604", "id": 1843975536, "node_id": "IC_kwDOCGYnMM5t6NVw", "user": {"value": 16437338, "label": "tkhattra"}, "created_at": "2023-12-07T01:17:05Z", "updated_at": "2023-12-07T01:17:05Z", "author_association": "CONTRIBUTOR", "body": "Apologies - I pushed a fix that addresses the mypy failures.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 2001006157, "label": "Add more STRICT table support"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/604#issuecomment-1846560096", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/604", "id": 1846560096, "node_id": "IC_kwDOCGYnMM5uEEVg", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-12-08T05:16:44Z", "updated_at": "2023-12-08T05:17:20Z", "author_association": "OWNER", "body": "Also tested this manually like so:\r\n\r\n```bash\r\nsqlite-utils create-table strict.db strictint id integer size integer --strict\r\nsqlite-utils create-table strict.db notstrictint id integer size integer \r\nsqlite-utils install sqlite-utils-shell\r\nsqlite-utils shell strict.db \r\n```\r\n```\r\nAttached to strict.db\r\nType 'exit' to exit.\r\nsqlite-utils> insert into strictint (size) values (4);\r\n1 row affected\r\nsqlite-utils> insert into strictint (size) values ('four');\r\nAn error occurred: cannot store TEXT value in INTEGER column strictint.size\r\nsqlite-utils> insert into notstrictint (size) values ('four');\r\n1 row affected\r\nsqlite-utils> commit;\r\nDone\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 2001006157, "label": "Add more STRICT table support"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/603#issuecomment-1846555822", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/603", "id": 1846555822, "node_id": "IC_kwDOCGYnMM5uEDSu", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-12-08T05:09:55Z", "updated_at": "2023-12-08T05:10:31Z", "author_association": "OWNER", "body": "I'm unable to replicate this issue. This is with a fresh install of `sqlite-utils==3.35.2`:\r\n```\r\n(base) ~ python3.12\r\nPython 3.12.0 (v3.12.0:0fb18b02c8, Oct 2 2023, 09:45:56) [Clang 13.0.0 (clang-1300.0.29.30)] on darwin\r\nType \"help\", \"copyright\", \"credits\" or \"license\" for more information.\r\n>>> import sqlite_utils\r\n>>> db = sqlite_utils.Database(memory=True)\r\n>>> db[\"foo\"].insert({\"bar\": 1})\r\n\r\n>>> import sys\r\n>>> sys.version\r\n'3.12.0 (v3.12.0:0fb18b02c8, Oct 2 2023, 09:45:56) [Clang 13.0.0 (clang-1300.0.29.30)]'\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1988525411, "label": "Pyhton 3.12 Bug report"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/600#issuecomment-1793263638", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/600", "id": 1793263638, "node_id": "IC_kwDOCGYnMM5q4wgW", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-11-04T00:19:58Z", "updated_at": "2023-11-04T00:19:58Z", "author_association": "OWNER", "body": "Thanks for this!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1977004379, "label": "Add spatialite arm64 linux path"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/600#issuecomment-1793264654", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/600", "id": 1793264654, "node_id": "IC_kwDOCGYnMM5q4wwO", "user": {"value": 22429695, "label": "codecov[bot]"}, "created_at": "2023-11-04T00:22:07Z", "updated_at": "2023-11-04T00:27:29Z", "author_association": "NONE", "body": "## [Codecov](https://app.codecov.io/gh/simonw/sqlite-utils/pull/600?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report\nAll modified and coverable lines are covered by tests :white_check_mark:\n> Comparison is base [(`622c3a5`)](https://app.codecov.io/gh/simonw/sqlite-utils/commit/622c3a5a7dd53a09c029e2af40c2643fe7579340?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) 95.77% compared to head [(`b1a6076`)](https://app.codecov.io/gh/simonw/sqlite-utils/pull/600?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) 95.77%.\n\n
Additional details and impacted files\n\n\n```diff\n@@ Coverage Diff @@\n## main #600 +/- ##\n=======================================\n Coverage 95.77% 95.77% \n=======================================\n Files 8 8 \n Lines 2840 2840 \n=======================================\n Hits 2720 2720 \n Misses 120 120 \n```\n\n\n| [Files](https://app.codecov.io/gh/simonw/sqlite-utils/pull/600?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage \u0394 | |\n|---|---|---|\n| [sqlite\\_utils/db.py](https://app.codecov.io/gh/simonw/sqlite-utils/pull/600?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-c3FsaXRlX3V0aWxzL2RiLnB5) | `97.22% <\u00f8> (\u00f8)` | |\n| [sqlite\\_utils/utils.py](https://app.codecov.io/gh/simonw/sqlite-utils/pull/600?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-c3FsaXRlX3V0aWxzL3V0aWxzLnB5) | `94.56% <\u00f8> (\u00f8)` | |\n\n\n
\n\n[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/simonw/sqlite-utils/pull/600?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). \n:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1977004379, "label": "Add spatialite arm64 linux path"}, "performed_via_github_app": "{\"id\": 254, \"slug\": \"codecov\", \"node_id\": \"MDM6QXBwMjU0\", \"owner\": {\"login\": \"codecov\", \"id\": 8226205, \"node_id\": \"MDEyOk9yZ2FuaXphdGlvbjgyMjYyMDU=\", \"avatar_url\": \"https://avatars.githubusercontent.com/u/8226205?v=4\", \"gravatar_id\": \"\", \"url\": \"https://api.github.com/users/codecov\", \"html_url\": \"https://github.com/codecov\", \"followers_url\": \"https://api.github.com/users/codecov/followers\", \"following_url\": \"https://api.github.com/users/codecov/following{/other_user}\", \"gists_url\": \"https://api.github.com/users/codecov/gists{/gist_id}\", \"starred_url\": \"https://api.github.com/users/codecov/starred{/owner}{/repo}\", \"subscriptions_url\": \"https://api.github.com/users/codecov/subscriptions\", \"organizations_url\": \"https://api.github.com/users/codecov/orgs\", \"repos_url\": \"https://api.github.com/users/codecov/repos\", \"events_url\": \"https://api.github.com/users/codecov/events{/privacy}\", \"received_events_url\": \"https://api.github.com/users/codecov/received_events\", \"type\": \"Organization\", \"site_admin\": false}, \"name\": \"Codecov\", \"description\": \"Codecov provides highly integrated tools to group, merge, archive and compare coverage reports. Whether your team is comparing changes in a pull request or reviewing a single commit, Codecov will improve the code review workflow and quality.\\r\\n\\r\\n## Code coverage done right.\u00ae\\r\\n\\r\\n1. Upload coverage reports from your CI builds.\\r\\n2. Codecov merges all builds and languages into one beautiful coherent report.\\r\\n3. Get commit statuses, pull request comments and coverage overlay via our browser extension.\\r\\n\\r\\nWhen Codecov merges your uploads it keeps track of the CI provider (inc. build details) and user specified context, e.g. `#unittest` ~ `#smoketest` or `#oldcode` ~ `#newcode`. You can track the `#unittest` coverage independently of other groups. [Learn more here](\\r\\nhttp://docs.codecov.io/docs/flags)\\r\\n\\r\\nThrough **Codecov's Browser Extension** reports overlay directly in GitHub UI to assist in code review in [Chrome](https://chrome.google.com/webstore/detail/codecov/gedikamndpbemklijjkncpnolildpbgo) or Firefox (https://addons.mozilla.org/en-US/firefox/addon/codecov/)\\r\\n\\r\\n*Highly detailed* **pull request comments** and *customizable* **commit statuses** will improve your team's workflow and code coverage incrementally.\\r\\n\\r\\n**File backed configuration** all through the `codecov.yml`. \\r\\n\\r\\n## FAQ\\r\\n- Do you **merge multiple uploads** to the same commit? **Yes**\\r\\n- Do you **support multiple languages** in the same project? **Yes**\\r\\n- Can you **group coverage reports** by project and/or test type? **Yes**\\r\\n- How does **pricing** work? Only paid users can view reports and post statuses/comments. \", \"external_url\": \"https://codecov.io\", \"html_url\": \"https://github.com/apps/codecov\", \"created_at\": \"2016-09-25T14:18:27Z\", \"updated_at\": \"2023-09-08T15:29:16Z\", \"permissions\": {\"administration\": \"read\", \"checks\": \"write\", \"contents\": \"read\", \"emails\": \"read\", \"issues\": \"read\", \"members\": \"read\", \"metadata\": \"read\", \"pull_requests\": \"write\", \"statuses\": \"write\"}, \"events\": [\"check_run\", \"check_suite\", \"create\", \"delete\", \"fork\", \"member\", \"membership\", \"organization\", \"public\", \"pull_request\", \"push\", \"release\", \"repository\", \"status\", \"team_add\"]}"} {"html_url": "https://github.com/simonw/sqlite-utils/pull/600#issuecomment-1793265952", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/600", "id": 1793265952, "node_id": "IC_kwDOCGYnMM5q4xEg", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-11-04T00:25:34Z", "updated_at": "2023-11-04T00:25:34Z", "author_association": "OWNER", "body": "The tests failed because they found a spelling mistake in a completely unrelated area of the code - not sure why that had not been caught before.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1977004379, "label": "Add spatialite arm64 linux path"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/600#issuecomment-1793268126", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/600", "id": 1793268126, "node_id": "IC_kwDOCGYnMM5q4xme", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-11-04T00:31:34Z", "updated_at": "2023-11-04T00:31:34Z", "author_association": "OWNER", "body": "Testing this manually on macOS using Docker Desk top like this:\r\n```bash\r\ndocker run -it --rm arm64v8/ubuntu /bin/bash\r\n```\r\nThen inside the container:\r\n```bash\r\nuname -m\r\n```\r\nOutputs: `aarch64`\r\n\r\nThen:\r\n```bash\r\napt install spatialite-bin libsqlite3-mod-spatialite git python3 python3-venv -y\r\ncd /tmp\r\ngit clone https://github.com/simonw/sqlite-utils\r\ncd sqlite-utils\r\npython3 -m venv venv\r\nsource venv/bin/activate\r\npip install -e '.[test]'\r\nsqlite-utils memory \"select spatialite_version()\" --load-extension=spatialite\r\n```\r\nWhich output:\r\n```\r\nTraceback (most recent call last):\r\n File \"/tmp/sqlite-utils/venv/bin/sqlite-utils\", line 33, in \r\n sys.exit(load_entry_point('sqlite-utils', 'console_scripts', 'sqlite-utils')())\r\n File \"/tmp/sqlite-utils/venv/lib/python3.10/site-packages/click/core.py\", line 1157, in __call__\r\n return self.main(*args, **kwargs)\r\n File \"/tmp/sqlite-utils/venv/lib/python3.10/site-packages/click/core.py\", line 1078, in main\r\n rv = self.invoke(ctx)\r\n File \"/tmp/sqlite-utils/venv/lib/python3.10/site-packages/click/core.py\", line 1688, in invoke\r\n return _process_result(sub_ctx.command.invoke(sub_ctx))\r\n File \"/tmp/sqlite-utils/venv/lib/python3.10/site-packages/click/core.py\", line 1434, in invoke\r\n return ctx.invoke(self.callback, **ctx.params)\r\n File \"/tmp/sqlite-utils/venv/lib/python3.10/site-packages/click/core.py\", line 783, in invoke\r\n return __callback(*args, **kwargs)\r\n File \"/tmp/sqlite-utils/sqlite_utils/cli.py\", line 1959, in memory\r\n _load_extensions(db, load_extension)\r\n File \"/tmp/sqlite-utils/sqlite_utils/cli.py\", line 3232, in _load_extensions\r\n if \":\" in ext:\r\nTypeError: argument of type 'NoneType' is not iterable\r\n```\r\nThen I ran this:\r\n```bash\r\ngit checkout -b MikeCoats-spatialite-paths-linux-arm main\r\ngit pull https://github.com/MikeCoats/sqlite-utils.git spatialite-paths-linux-arm\r\n```\r\nAnd now:\r\n```bash\r\nsqlite-utils memory \"select spatialite_version()\" --load-extension=spatialite\r\n```\r\nOutputs:\r\n```json\r\n[{\"spatialite_version()\": \"5.0.1\"}]\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1977004379, "label": "Add spatialite arm64 linux path"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/600#issuecomment-1793269219", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/600", "id": 1793269219, "node_id": "IC_kwDOCGYnMM5q4x3j", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-11-04T00:34:33Z", "updated_at": "2023-11-04T00:34:33Z", "author_association": "OWNER", "body": "The GIS tests now pass in that container too:\r\n```bash\r\npytest tests/test_gis.py\r\n```\r\n```\r\n======================== test session starts =========================\r\nplatform linux -- Python 3.10.12, pytest-7.4.3, pluggy-1.3.0\r\nrootdir: /tmp/sqlite-utils\r\nplugins: hypothesis-6.88.1\r\ncollected 12 items \r\n\r\ntests/test_gis.py ............ [100%]\r\n\r\n========================= 12 passed in 0.48s =========================\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1977004379, "label": "Add spatialite arm64 linux path"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/6#issuecomment-457978729", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/6", "id": 457978729, "node_id": "MDEyOklzc3VlQ29tbWVudDQ1Nzk3ODcyOQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-01-28T02:12:19Z", "updated_at": "2019-01-28T02:12:19Z", "author_association": "OWNER", "body": "Will need to solve #7 for this to become truly efficient.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 403624090, "label": "\"sqlite-utils insert\" should support newline-delimited JSON"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/599#issuecomment-1793268750", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/599", "id": 1793268750, "node_id": "IC_kwDOCGYnMM5q4xwO", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-11-04T00:33:25Z", "updated_at": "2023-11-04T00:33:25Z", "author_association": "OWNER", "body": "See details of how I tested this here:\r\n- https://github.com/simonw/sqlite-utils/pull/600#issuecomment-1793268126\r\n\r\nShort version: having applied this fix, the following command (on simulated `aarch64`):\r\n```bash\r\nsqlite-utils memory \"select spatialite_version()\" --load-extension=spatialite\r\n```\r\nOutputs:\r\n```json\r\n[{\"spatialite_version()\": \"5.0.1\"}]\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1976986318, "label": "Cannot find spatialite on arm64 linux"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/598#issuecomment-1793272429", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/598", "id": 1793272429, "node_id": "IC_kwDOCGYnMM5q4ypt", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-11-04T00:40:34Z", "updated_at": "2023-11-04T00:40:34Z", "author_association": "OWNER", "body": "Thanks!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1926729132, "label": "Fixed issue #433 - CLI eats cursor"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/598#issuecomment-1793274485", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/598", "id": 1793274485, "node_id": "IC_kwDOCGYnMM5q4zJ1", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-11-04T00:46:55Z", "updated_at": "2023-11-04T00:46:55Z", "author_association": "OWNER", "body": "Manually tested. Before:\r\n\r\n![cursor-bug](https://github.com/simonw/sqlite-utils/assets/9599/3bdd30ea-1a54-4fec-b42d-793130a17bc1)\r\n\r\nAfter:\r\n\r\n![cursor-fix](https://github.com/simonw/sqlite-utils/assets/9599/015d4e4e-b40c-4a93-81f5-1a4adef69b11)\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1926729132, "label": "Fixed issue #433 - CLI eats cursor"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/596#issuecomment-1793274869", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/596", "id": 1793274869, "node_id": "IC_kwDOCGYnMM5q4zP1", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-11-04T00:47:55Z", "updated_at": "2023-11-04T00:47:55Z", "author_association": "OWNER", "body": "Thanks!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1919296686, "label": "Fixes mapping for time fields related to mysql, closes #522"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/595#issuecomment-1733312349", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/595", "id": 1733312349, "node_id": "IC_kwDOCGYnMM5nUD9d", "user": {"value": 123451970, "label": "cycle-data"}, "created_at": "2023-09-25T09:38:13Z", "updated_at": "2023-09-25T09:38:57Z", "author_association": "NONE", "body": "Never mind\r\n\r\nWhen I created the connection using \r\n`sqlite_utils.Database(path)`\r\n\r\nI just needed to add the following statement right after and it did the trick\r\n\r\n`self.db.conn.execute(\"PRAGMA foreign_keys = ON\")`\r\n\r\nHope this helps people in the future \ud83d\udc4d ", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1907281675, "label": "Cascading DELETE not working with Table.delete(pk)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/594#issuecomment-1714919806", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/594", "id": 1714919806, "node_id": "IC_kwDOCGYnMM5mN5l-", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-12T03:49:41Z", "updated_at": "2023-09-12T03:49:41Z", "author_association": "OWNER", "body": "Digging in a bit more:\r\n```pycon\r\n>>> pprint(list(db.query('PRAGMA foreign_key_list(courses)')))\r\n[{'from': 'campus_name',\r\n 'id': 0,\r\n 'match': 'NONE',\r\n 'on_delete': 'NO ACTION',\r\n 'on_update': 'NO ACTION',\r\n 'seq': 0,\r\n 'table': 'departments',\r\n 'to': 'campus_name'},\r\n {'from': 'dept_code',\r\n 'id': 0,\r\n 'match': 'NONE',\r\n 'on_delete': 'NO ACTION',\r\n 'on_update': 'NO ACTION',\r\n 'seq': 1,\r\n 'table': 'departments',\r\n 'to': 'dept_code'}]\r\n```\r\nI think the way you tell it's a compound foreign key is that both of those have the same `id` value - of `0` - but they then have two different `seq` values of `0` and `1`.\r\n\r\nRight now I ignore those columns entirely: https://github.com/simonw/sqlite-utils/blob/622c3a5a7dd53a09c029e2af40c2643fe7579340/sqlite_utils/db.py#L1523-L1540", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1891614971, "label": "Represent compound foreign keys in table.foreign_keys output"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/594#issuecomment-1714920708", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/594", "id": 1714920708, "node_id": "IC_kwDOCGYnMM5mN50E", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-12T03:51:13Z", "updated_at": "2023-09-12T03:51:13Z", "author_association": "OWNER", "body": "Changing this without breaking backwards compatibility (and forcing a 4.0 release) will be tricky, because `ForeignKey()` is a `namedtuple`:\r\n\r\nhttps://github.com/simonw/sqlite-utils/blob/622c3a5a7dd53a09c029e2af40c2643fe7579340/sqlite_utils/db.py#L148-L150\r\n\r\nI could swap it out for a `dataclass` and add those extra columns, but I need to make sure that code like this still works:\r\n```python\r\nfor table, column, other_table, other_column in table.foreign_keys:\r\n # ...\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1891614971, "label": "Represent compound foreign keys in table.foreign_keys output"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/593#issuecomment-1710939868", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/593", "id": 1710939868, "node_id": "IC_kwDOCGYnMM5l-t7c", "user": {"value": 22429695, "label": "codecov[bot]"}, "created_at": "2023-09-08T01:03:40Z", "updated_at": "2023-09-09T00:44:52Z", "author_association": "NONE", "body": "## [Codecov](https://app.codecov.io/gh/simonw/sqlite-utils/pull/593?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report\nPatch coverage: **`100.00%`** and no project coverage change.\n> Comparison is base [(`5d123f0`)](https://app.codecov.io/gh/simonw/sqlite-utils/commit/5d123f031fc4fadc98f508e0ef6b7b6671e86155?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) 95.77% compared to head [(`b86374f`)](https://app.codecov.io/gh/simonw/sqlite-utils/pull/593?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) 95.77%.\n\n
Additional details and impacted files\n\n\n```diff\n@@ Coverage Diff @@\n## main #593 +/- ##\n=======================================\n Coverage 95.77% 95.77% \n=======================================\n Files 8 8 \n Lines 2837 2840 +3 \n=======================================\n+ Hits 2717 2720 +3 \n Misses 120 120 \n```\n\n\n| [Files Changed](https://app.codecov.io/gh/simonw/sqlite-utils/pull/593?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage \u0394 | |\n|---|---|---|\n| [sqlite\\_utils/db.py](https://app.codecov.io/gh/simonw/sqlite-utils/pull/593?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-c3FsaXRlX3V0aWxzL2RiLnB5) | `97.22% <100.00%> (+<0.01%)` | :arrow_up: |\n\n\n
\n\n[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/simonw/sqlite-utils/pull/593?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). \n:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1886783150, "label": ".transform() now preserves rowid values, refs #592"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/592#issuecomment-1710930934", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/592", "id": 1710930934, "node_id": "IC_kwDOCGYnMM5l-rv2", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-08T00:47:57Z", "updated_at": "2023-09-08T00:47:57Z", "author_association": "OWNER", "body": "That's odd, I wrote a test for this just now and it passes already:\r\n```python\r\ndef test_transform_preserves_rowids(fresh_db):\r\n # Create a rowid table\r\n fresh_db[\"places\"].insert_all(\r\n (\r\n {\"name\": \"Paris\", \"country\": \"France\"},\r\n {\"name\": \"London\", \"country\": \"UK\"},\r\n {\"name\": \"New York\", \"country\": \"USA\"},\r\n ),\r\n )\r\n assert fresh_db[\"places\"].use_rowid\r\n previous_rows = list(\r\n tuple(row) for row in fresh_db.execute(\"select rowid, name from places\")\r\n )\r\n # Transform it\r\n fresh_db[\"places\"].transform(column_order=(\"country\", \"name\"))\r\n # Should be the same\r\n next_rows = list(\r\n tuple(row) for row in fresh_db.execute(\"select rowid, name from places\")\r\n )\r\n assert previous_rows == next_rows\r\n```\r\nSo maybe I'm wrong about the cause of that bug?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1886771493, "label": "`table.transform()` should preserve `rowid` values"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/592#issuecomment-1710931605", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/592", "id": 1710931605, "node_id": "IC_kwDOCGYnMM5l-r6V", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-08T00:49:02Z", "updated_at": "2023-09-08T00:49:02Z", "author_association": "OWNER", "body": "I tried bumping that up to 10,000 rows instead of just 3 but the test still passed.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1886771493, "label": "`table.transform()` should preserve `rowid` values"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/592#issuecomment-1710933716", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/592", "id": 1710933716, "node_id": "IC_kwDOCGYnMM5l-sbU", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-08T00:52:42Z", "updated_at": "2023-09-08T00:52:42Z", "author_association": "OWNER", "body": "I just noticed that the table where I encountered this bug wasn't actually a `rowid` table after all - it had an `id` column that was a text primary key.\r\n\r\nThe reason the `rowid` was important is that's how the FTS mechanism in Datasette relates FTS entries to their rows.\r\n\r\nBut I tried this test and it passed, too:\r\n```python\r\ndef test_transform_preserves_rowids(fresh_db):\r\n fresh_db[\"places\"].insert_all(\r\n [\r\n {\"id\": \"1\", \"name\": \"Paris\", \"country\": \"France\"},\r\n {\"id\": \"2\", \"name\": \"London\", \"country\": \"UK\"},\r\n {\"id\": \"3\", \"name\": \"New York\", \"country\": \"USA\"},\r\n ],\r\n pk=\"id\",\r\n )\r\n previous_rows = list(\r\n tuple(row) for row in fresh_db.execute(\"select rowid, id, name from places\")\r\n )\r\n # Transform it\r\n fresh_db[\"places\"].transform(column_order=(\"country\", \"name\"))\r\n # Should be the same\r\n next_rows = list(\r\n tuple(row) for row in fresh_db.execute(\"select rowid, id, name from places\")\r\n )\r\n assert previous_rows == next_rows\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1886771493, "label": "`table.transform()` should preserve `rowid` values"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/592#issuecomment-1710934448", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/592", "id": 1710934448, "node_id": "IC_kwDOCGYnMM5l-smw", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-08T00:54:03Z", "updated_at": "2023-09-08T00:54:03Z", "author_association": "OWNER", "body": "Oh! Maybe the row ID preservation here is a coincidence because the tables are created from scratch and count 1, 2, 3.\r\n\r\nIf I delete a row from the table and then insert some more - breaking the `rowid` sequence - it might show the bug.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1886771493, "label": "`table.transform()` should preserve `rowid` values"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/592#issuecomment-1710935270", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/592", "id": 1710935270, "node_id": "IC_kwDOCGYnMM5l-szm", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-08T00:55:30Z", "updated_at": "2023-09-08T00:55:30Z", "author_association": "OWNER", "body": "Yes! That recreated the bug:\r\n```\r\n> assert previous_rows == next_rows\r\nE AssertionError: assert equals failed\r\nE [ [ \r\nE (1, '1', 'Paris'), (1, '1', 'Paris'), \r\nE (3, '3', 'New York'), (2, '3', 'New York'), \r\nE (4, '4', 'London'), (3, '4', 'London'), \r\nE ] ...\r\nE \r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1886771493, "label": "`table.transform()` should preserve `rowid` values"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/592#issuecomment-1712895580", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/592", "id": 1712895580, "node_id": "IC_kwDOCGYnMM5mGLZc", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-10T17:46:41Z", "updated_at": "2023-09-10T17:46:41Z", "author_association": "OWNER", "body": "In working on this I learned that `rowid` values in SQLite are way less stable than I had thought - in particular, they are often entirely rewritten on a `VACUUM`:\r\n\r\nhttps://www.sqlite.org/lang_vacuum.html#how_vacuum_works\r\n\r\n> The VACUUM command may change the [ROWIDs](https://www.sqlite.org/lang_createtable.html#rowid) of entries in any tables that do not have an explicit [INTEGER PRIMARY KEY](https://www.sqlite.org/lang_createtable.html#rowid).\r\n\r\nSo this fix wasn't as valuable as I thought. I need to move away from ever assuming that a `rowid` is a useful foreign key for anything.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1886771493, "label": "`table.transform()` should preserve `rowid` values"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/591#issuecomment-1708693020", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/591", "id": 1708693020, "node_id": "IC_kwDOCGYnMM5l2JYc", "user": {"value": 22429695, "label": "codecov[bot]"}, "created_at": "2023-09-06T16:14:03Z", "updated_at": "2023-11-04T00:54:25Z", "author_association": "NONE", "body": "## [Codecov](https://app.codecov.io/gh/simonw/sqlite-utils/pull/591?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report\nAll modified and coverable lines are covered by tests :white_check_mark:\n> Comparison is base [(`347fdc8`)](https://app.codecov.io/gh/simonw/sqlite-utils/commit/347fdc865e91b8d3410f49a5c9d5b499fbb594c1?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) 95.74% compared to head [(`1f14df1`)](https://app.codecov.io/gh/simonw/sqlite-utils/pull/591?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) 95.74%.\n\n
Additional details and impacted files\n\n\n```diff\n@@ Coverage Diff @@\n## main #591 +/- ##\n=======================================\n Coverage 95.74% 95.74% \n=======================================\n Files 8 8 \n Lines 2842 2842 \n=======================================\n Hits 2721 2721 \n Misses 121 121 \n```\n\n\n\n\n
\n\n[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/simonw/sqlite-utils/pull/591?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). \n:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1884335789, "label": "Test against Python 3.12 preview"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/591#issuecomment-1708695907", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/591", "id": 1708695907, "node_id": "IC_kwDOCGYnMM5l2KFj", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-06T16:15:59Z", "updated_at": "2023-09-06T16:19:14Z", "author_association": "OWNER", "body": "The test failure was while installing `numpy`, relating to importing `distutils` - maybe relevant:\r\n- https://github.com/pypa/setuptools/issues/3661\r\n\r\n```\r\n25h Installing build dependencies: started\r\n Installing build dependencies: finished with status 'done'\r\n Getting requirements to build wheel: started\r\n Getting requirements to build wheel: finished with status 'done'\r\nERROR: Exception:\r\nTraceback (most recent call last):\r\n...\r\n File \"/opt/hostedtoolcache/Python/3.12.0-rc.2/x64/lib/python3.12/site-packages/pip/_internal/utils/misc.py\", line 697, in get_requires_for_build_wheel\r\n return super().get_requires_for_build_wheel(config_settings=cs)\r\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\r\n File \"/opt/hostedtoolcache/Python/3.12.0-rc.2/x64/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_impl.py\", line 166, in get_requires_for_build_wheel\r\n return self._call_hook('get_requires_for_build_wheel', {\r\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\r\n File \"/opt/hostedtoolcache/Python/3.12.0-rc.2/x64/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_impl.py\", line 321, in _call_hook\r\n raise BackendUnavailable(data.get('traceback', ''))\r\npip._vendor.pyproject_hooks._impl.BackendUnavailable: Traceback (most recent call last):\r\n File \"/opt/hostedtoolcache/Python/3.12.0-rc.2/x64/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py\", line 77, in _build_backend\r\n obj = import_module(mod_path)\r\n ^^^^^^^^^^^^^^^^^^^^^^^\r\n File \"/opt/hostedtoolcache/Python/3.12.0-rc.2/x64/lib/python3.12/importlib/__init__.py\", line 90, in import_module\r\n return _bootstrap._gcd_import(name[level:], package, level)\r\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\r\n File \"\", line 1381, in _gcd_import\r\n File \"\", line 1354, in _find_and_load\r\n File \"\", line 1304, in _find_and_load_unlocked\r\n File \"\", line 488, in _call_with_frames_removed\r\n File \"\", line 1381, in _gcd_import\r\n File \"\", line 1354, in _find_and_load\r\n File \"\", line 1325, in _find_and_load_unlocked\r\n File \"\", line 929, in _load_unlocked\r\n File \"\", line 994, in exec_module\r\n File \"\", line 488, in _call_with_frames_removed\r\n File \"/tmp/pip-build-env-x9nyg3kd/overlay/lib/python3.12/site-packages/setuptools/__init__.py\", line 10, in \r\n import distutils.core\r\nModuleNotFoundError: No module named 'distutils'\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1884335789, "label": "Test against Python 3.12 preview"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/591#issuecomment-1793278279", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/591", "id": 1793278279, "node_id": "IC_kwDOCGYnMM5q40FH", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-11-04T00:58:03Z", "updated_at": "2023-11-04T00:58:03Z", "author_association": "OWNER", "body": "I'm going to abandon this PR and ship the 3.12 testing change directly to `main`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1884335789, "label": "Test against Python 3.12 preview"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/590#issuecomment-1704387161", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/590", "id": 1704387161, "node_id": "IC_kwDOCGYnMM5lluJZ", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-03T19:50:36Z", "updated_at": "2023-09-03T19:50:36Z", "author_association": "OWNER", "body": "Maybe just populate `db.memory: bool` and `db.memory_name: Optional[str]` for this, then document them.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1879214365, "label": "Ability to tell if a Database is an in-memory one"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/589#issuecomment-1704383901", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/589", "id": 1704383901, "node_id": "IC_kwDOCGYnMM5lltWd", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-03T19:34:05Z", "updated_at": "2023-09-03T19:34:05Z", "author_association": "OWNER", "body": "For that particular case I realized I'd quite like to have a mechanism for applying functions for a block of code and then de-registering them at the end - a context manager.\r\n\r\nI played with this idea a bit:\r\n\r\n```python\r\nwith db.register_functions(md5, md5_random):\r\n db.query(...)\r\n```\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1879209560, "label": "Mechanism for de-registering registered SQL functions"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/589#issuecomment-1704384111", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/589", "id": 1704384111, "node_id": "IC_kwDOCGYnMM5lltZv", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-03T19:35:03Z", "updated_at": "2023-09-03T19:35:03Z", "author_association": "OWNER", "body": "Normally in Python/`sqlite3` you de-register a function by passing `None` to it.\r\n\r\nYou can't do that with `db.register_function()` at the moment because a `fn` of `None` does something else:\r\n\r\nhttps://github.com/simonw/sqlite-utils/blob/1260bdc7bfe31c36c272572c6389125f8de6ef71/sqlite_utils/db.py#L461-L464", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1879209560, "label": "Mechanism for de-registering registered SQL functions"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/589#issuecomment-1704384393", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/589", "id": 1704384393, "node_id": "IC_kwDOCGYnMM5llteJ", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-03T19:36:34Z", "updated_at": "2023-09-03T19:36:34Z", "author_association": "OWNER", "body": "Here's a prototype: https://github.com/simonw/sqlite-utils/commit/62f673835c4a66f87cf6f949eaff43c8b014619b\r\n\r\nStill needs tests and documentation (and some more thought to make sure it's doing the right thing).", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1879209560, "label": "Mechanism for de-registering registered SQL functions"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/588#issuecomment-1694823972", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/588", "id": 1694823972, "node_id": "IC_kwDOCGYnMM5lBPYk", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-28T00:41:54Z", "updated_at": "2023-08-28T00:41:54Z", "author_association": "OWNER", "body": "Tips on typing `**kwargs`: https://adamj.eu/tech/2021/05/11/python-type-hints-args-and-kwargs/", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1868713944, "label": "`table.get(column=value)` option for retrieving things not by their primary key"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/587#issuecomment-1685096129", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/587", "id": 1685096129, "node_id": "IC_kwDOCGYnMM5kcIbB", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-19T20:03:00Z", "updated_at": "2023-08-19T20:03:00Z", "author_association": "OWNER", "body": "Simplest possible recreation of the bug:\r\n```bash\r\npython -c '\r\nimport sqlite_utils\r\ndb = sqlite_utils.Database(memory=True)\r\ndb.execute(\"\"\"\r\nCREATE TABLE \"logs\" (\r\n [id] INTEGER PRIMARY KEY,\r\n [chat_id] INTEGER REFERENCES [log]([id]),\r\n [reply_to_id] INTEGER\r\n);\r\n\"\"\")\r\ndb[\"logs\"].add_foreign_key(\"reply_to_id\", \"logs\", \"id\")\r\n'\r\n```\r\nThat `chat_id` line is the line that causes the problem - because it is defining a reference to a table that no longer exists!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1857851384, "label": "New .add_foreign_key() can break if PRAGMA legacy_alter_table=ON and there's an invalid foreign key reference"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/587#issuecomment-1685096284", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/587", "id": 1685096284, "node_id": "IC_kwDOCGYnMM5kcIdc", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-19T20:03:59Z", "updated_at": "2023-08-19T20:03:59Z", "author_association": "OWNER", "body": "Although this is revealing a problem in the underlying code (that schema is invalid), it also represents a regression: `sqlite-utils 3.34` ran this just fine, but it fails on `sqlite-utils 3.35` due to the change made in:\r\n- #577", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1857851384, "label": "New .add_foreign_key() can break if PRAGMA legacy_alter_table=ON and there's an invalid foreign key reference"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/587#issuecomment-1685096381", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/587", "id": 1685096381, "node_id": "IC_kwDOCGYnMM5kcIe9", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-19T20:04:32Z", "updated_at": "2023-08-19T20:04:32Z", "author_association": "OWNER", "body": "I'm inclined to say this isn't a bug in `sqlite-utils` though - it's a bug in the code that calls it. So I'm not going to fix it here.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1857851384, "label": "New .add_foreign_key() can break if PRAGMA legacy_alter_table=ON and there's an invalid foreign key reference"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/586#issuecomment-1683396150", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/586", "id": 1683396150, "node_id": "IC_kwDOCGYnMM5kVpY2", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-18T06:02:18Z", "updated_at": "2023-08-18T06:06:31Z", "author_association": "OWNER", "body": "More notes in here:\r\n- https://github.com/simonw/datasette-edit-schema/issues/35#issuecomment-1683392873\r\n\r\nNot all Python/SQLite installations exhibit this problem by default!\r\n\r\nIt turns out this is controlled by the `legacy_alter_table` pragma: https://sqlite.org/pragma.html#pragma_legacy_alter_table\r\n\r\nIf that PRAGMA is turned on (default in newer SQLites) then `alter table` will error if you try to rename a table that is referenced in a view.\r\n\r\nHere's a one-liner to test if it is on or not:\r\n\r\n```bash\r\npython -c 'import sqlite3; print(sqlite3.connect(\":memory:\").execute(\"PRAGMA legacy_alter_table\").fetchall())'\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1856075668, "label": ".transform() fails to drop column if table is part of a view"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/586#issuecomment-1683398866", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/586", "id": 1683398866, "node_id": "IC_kwDOCGYnMM5kVqDS", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-18T06:05:50Z", "updated_at": "2023-08-18T06:06:42Z", "author_association": "OWNER", "body": "Options:\r\n- Provide a `recreate_views: bool` parameter to `table.transform()` controlling if views that might reference this table are stashed and dropped and recreated within a transaction as part of the operation. But should that be `True` or `False` by default?\r\n- Read that `PRAGMA` and automatically do that view workaround if it's turned on\r\n- Toggle that `PRAGMA` off for the duration of the `.transform()` operation and on again at the end. Does it only affect the current connection?\r\n- Try the `transform()` in a transaction, detect the `\"error in view\"`, `\"no such table\"`error, if spotted then do the VIEW workaround and try again\r\n\r\nI'm on the fence as to which of these I like the most. I'm tempted to go with the one which just drops VIEWS and recreates them all the time, because it feels simpler.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1856075668, "label": ".transform() fails to drop column if table is part of a view"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/586#issuecomment-1683404978", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/586", "id": 1683404978, "node_id": "IC_kwDOCGYnMM5kVriy", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-18T06:13:46Z", "updated_at": "2023-08-18T06:13:46Z", "author_association": "OWNER", "body": "I shipped the view recreating fix in `datasette-edit-schema`, so at least I can start exercising that fix and see if it has any weird issues.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1856075668, "label": ".transform() fails to drop column if table is part of a view"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/585#issuecomment-1683195669", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/585", "id": 1683195669, "node_id": "IC_kwDOCGYnMM5kU4cV", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-18T01:24:57Z", "updated_at": "2023-08-18T01:24:57Z", "author_association": "OWNER", "body": "Currently:\r\n```bash\r\nsqlite-utils transform --help\r\n```\r\n```\r\nUsage: sqlite-utils transform [OPTIONS] PATH TABLE\r\n\r\n Transform a table beyond the capabilities of ALTER TABLE\r\n\r\n Example:\r\n\r\n sqlite-utils transform mydb.db mytable \\\r\n --drop column1 \\\r\n --rename column2 column_renamed\r\n\r\nOptions:\r\n --type ... Change column type to INTEGER, TEXT, FLOAT or BLOB\r\n --drop TEXT Drop this column\r\n --rename ... Rename this column to X\r\n -o, --column-order TEXT Reorder columns\r\n --not-null TEXT Set this column to NOT NULL\r\n --not-null-false TEXT Remove NOT NULL from this column\r\n --pk TEXT Make this column the primary key\r\n --pk-none Remove primary key (convert to rowid table)\r\n --default ... Set default value for this column\r\n --default-none TEXT Remove default from this column\r\n --drop-foreign-key TEXT Drop foreign key constraint for this column\r\n --sql Output SQL without executing it\r\n --load-extension TEXT Path to SQLite extension, with optional\r\n :entrypoint\r\n -h, --help Show this message and exit.\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1855894222, "label": "CLI equivalents to `transform(add_foreign_keys=)`"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/585#issuecomment-1683197882", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/585", "id": 1683197882, "node_id": "IC_kwDOCGYnMM5kU4-6", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-18T01:25:53Z", "updated_at": "2023-08-18T01:25:53Z", "author_association": "OWNER", "body": "Probably most relevant here is this snippet from:\r\n```bash\r\nsqlite-utils create-table --help\r\n```\r\n```\r\n --default ... Default value that should be set for a column\r\n --fk ... Column, other table, other column to set as a\r\n foreign key\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1855894222, "label": "CLI equivalents to `transform(add_foreign_keys=)`"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/585#issuecomment-1683198740", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/585", "id": 1683198740, "node_id": "IC_kwDOCGYnMM5kU5MU", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-18T01:26:47Z", "updated_at": "2023-08-18T01:26:47Z", "author_association": "OWNER", "body": "The only CLI feature that supports providing just the column name appears to be this:\r\n```bash\r\nsqlite-utils add-foreign-key --help\r\n```\r\n```\r\nUsage: sqlite-utils add-foreign-key [OPTIONS] PATH TABLE COLUMN [OTHER_TABLE]\r\n [OTHER_COLUMN]\r\n\r\n Add a new foreign key constraint to an existing table\r\n\r\n Example:\r\n\r\n sqlite-utils add-foreign-key my.db books author_id authors id\r\n\r\n WARNING: Could corrupt your database! Back up your database file first.\r\n```\r\nI can drop that WARNING now since I'm not writing to `sqlite_master` any more.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1855894222, "label": "CLI equivalents to `transform(add_foreign_keys=)`"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/585#issuecomment-1683200128", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/585", "id": 1683200128, "node_id": "IC_kwDOCGYnMM5kU5iA", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-18T01:29:00Z", "updated_at": "2023-08-18T01:29:00Z", "author_association": "OWNER", "body": "I'm not going to implement the `foreign_keys=` option that entirely replaces existing foreign keys - I'll just do a `--add-foreign-key` multi-option.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1855894222, "label": "CLI equivalents to `transform(add_foreign_keys=)`"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/585#issuecomment-1683201239", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/585", "id": 1683201239, "node_id": "IC_kwDOCGYnMM5kU5zX", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-18T01:30:46Z", "updated_at": "2023-08-18T01:30:46Z", "author_association": "OWNER", "body": "Help can now look like this:\r\n```\r\n --drop-foreign-key TEXT Drop foreign key constraint for this column\r\n --add-foreign-key ...\r\n Add a foreign key constraint from a column\r\n to another table with another column\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1855894222, "label": "CLI equivalents to `transform(add_foreign_keys=)`"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/585#issuecomment-1683212074", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/585", "id": 1683212074, "node_id": "IC_kwDOCGYnMM5kU8cq", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-18T01:43:54Z", "updated_at": "2023-08-18T01:43:54Z", "author_association": "OWNER", "body": "Some manual testing:\r\n```bash\r\nsqlite-utils create-table /tmp/t.db places id integer name text country integer city integer continent integer --pk id\r\nsqlite-utils schema /tmp/t.db\r\n```\r\n```sql\r\nCREATE TABLE [places] (\r\n [id] INTEGER PRIMARY KEY,\r\n [name] TEXT,\r\n [country] INTEGER,\r\n [city] INTEGER,\r\n [continent] INTEGER\r\n);\r\n```\r\n```bash\r\nsqlite-utils create-table /tmp/t.db country id integer name text\r\nsqlite-utils create-table /tmp/t.db city id integer name text\r\nsqlite-utils create-table /tmp/t.db continent id integer name text\r\nsqlite-utils schema /tmp/t.db\r\n```\r\n```sql\r\nCREATE TABLE [places] (\r\n [id] INTEGER PRIMARY KEY,\r\n [name] TEXT,\r\n [country] INTEGER,\r\n [city] INTEGER,\r\n [continent] INTEGER\r\n);\r\nCREATE TABLE [country] (\r\n [id] INTEGER,\r\n [name] TEXT\r\n);\r\nCREATE TABLE [city] (\r\n [id] INTEGER,\r\n [name] TEXT\r\n);\r\nCREATE TABLE [continent] (\r\n [id] INTEGER,\r\n [name] TEXT\r\n);\r\n```\r\n```bash\r\nsqlite-utils transform /tmp/t.db places --add-foreign-key country country id --add-foreign-key continent continent id\r\nsqlite-utils schema /tmp/t.db\r\n```\r\n```sql\r\nCREATE TABLE [country] (\r\n [id] INTEGER,\r\n [name] TEXT\r\n);\r\nCREATE TABLE [city] (\r\n [id] INTEGER,\r\n [name] TEXT\r\n);\r\nCREATE TABLE [continent] (\r\n [id] INTEGER,\r\n [name] TEXT\r\n);\r\nCREATE TABLE \"places\" (\r\n [id] INTEGER PRIMARY KEY,\r\n [name] TEXT,\r\n [country] INTEGER REFERENCES [country]([id]),\r\n [city] INTEGER,\r\n [continent] INTEGER REFERENCES [continent]([id])\r\n);\r\n```\r\n```bash\r\nsqlite-utils transform /tmp/t.db places --drop-foreign-key country\r\nsqlite-utils schema /tmp/t.db places\r\n```\r\n```sql\r\nCREATE TABLE \"places\" (\r\n [id] INTEGER PRIMARY KEY,\r\n [name] TEXT,\r\n [country] INTEGER,\r\n [city] INTEGER,\r\n [continent] INTEGER REFERENCES [continent]([id])\r\n)\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1855894222, "label": "CLI equivalents to `transform(add_foreign_keys=)`"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/585#issuecomment-1683217284", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/585", "id": 1683217284, "node_id": "IC_kwDOCGYnMM5kU9uE", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-18T01:50:21Z", "updated_at": "2023-08-18T01:50:21Z", "author_association": "OWNER", "body": "And a test of the `--sql` option:\r\n```bash\r\nsqlite-utils create-table /tmp/t.db places id integer name text country integer city integer continent integer --pk id\r\nsqlite-utils create-table /tmp/t.db country id integer name text\r\nsqlite-utils create-table /tmp/t.db city id integer name text\r\nsqlite-utils create-table /tmp/t.db continent id integer name text\r\nsqlite-utils transform /tmp/t.db places --add-foreign-key country country id --add-foreign-key continent continent id --sql\r\n```\r\nOutputs:\r\n```sql\r\nCREATE TABLE [places_new_6a705d2f5a13] (\r\n [id] INTEGER PRIMARY KEY,\r\n [name] TEXT,\r\n [country] INTEGER REFERENCES [country]([id]),\r\n [city] INTEGER,\r\n [continent] INTEGER REFERENCES [continent]([id])\r\n);\r\nINSERT INTO [places_new_6a705d2f5a13] ([id], [name], [country], [city], [continent])\r\n SELECT [id], [name], [country], [city], [continent] FROM [places];\r\nDROP TABLE [places];\r\nALTER TABLE [places_new_6a705d2f5a13] RENAME TO [places];\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1855894222, "label": "CLI equivalents to `transform(add_foreign_keys=)`"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/584#issuecomment-1683112298", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/584", "id": 1683112298, "node_id": "IC_kwDOCGYnMM5kUkFq", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-17T23:33:14Z", "updated_at": "2023-08-17T23:33:14Z", "author_association": "OWNER", "body": "Just one failing test left:\r\n\r\n```\r\n # Soundness check foreign_keys point to existing tables\r\n for fk in foreign_keys:\r\n if fk.other_table == name and columns.get(fk.other_column):\r\n continue\r\n if not any(\r\n c for c in self[fk.other_table].columns if c.name == fk.other_column\r\n ):\r\n> raise AlterError(\r\n \"No such column: {}.{}\".format(fk.other_table, fk.other_column)\r\n )\r\nE sqlite_utils.db.AlterError: No such column: breeds.rowid\r\n\r\nsqlite_utils/db.py:882: AlterError\r\n==== short test summary info ====\r\nFAILED tests/test_create.py::test_add_column_foreign_key - sqlite_utils.db.AlterError: No such column: breeds.rowid\r\n==== 1 failed, 378 deselected in 0.49s ====\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1855838223, "label": ".transform() instead of modifying sqlite_master for add_foreign_keys"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/584#issuecomment-1683112857", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/584", "id": 1683112857, "node_id": "IC_kwDOCGYnMM5kUkOZ", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-17T23:33:58Z", "updated_at": "2023-08-17T23:33:58Z", "author_association": "OWNER", "body": "Full test:\r\n\r\nhttps://github.com/simonw/sqlite-utils/blob/842b61321fc6a9f0bdb913ab138e39d71bf42e00/tests/test_create.py#L468-L484", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1855838223, "label": ".transform() instead of modifying sqlite_master for add_foreign_keys"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/584#issuecomment-1683114719", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/584", "id": 1683114719, "node_id": "IC_kwDOCGYnMM5kUkrf", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-17T23:36:02Z", "updated_at": "2023-08-17T23:36:02Z", "author_association": "OWNER", "body": "Just these three lines recreate the problem:\r\n```python\r\nfrom sqlite_utils import Database\r\n\r\nfresh_db = Database(memory=True)\r\n\r\nfresh_db.create_table(\"dogs\", {\"name\": str})\r\nfresh_db.create_table(\"breeds\", {\"name\": str})\r\nfresh_db[\"dogs\"].add_column(\"breed_id\", fk=\"breeds\")\r\n```\r\nTraceback:\r\n```\r\nTraceback (most recent call last):\r\n File \"\", line 1, in \r\n File \"/Users/simon/Dropbox/Development/sqlite-utils/sqlite_utils/db.py\", line 2170, in add_column\r\n self.add_foreign_key(col_name, fk, fk_col)\r\n File \"/Users/simon/Dropbox/Development/sqlite-utils/sqlite_utils/db.py\", line 2273, in add_foreign_key\r\n self.db.add_foreign_keys([(self.name, column, other_table, other_column)])\r\n File \"/Users/simon/Dropbox/Development/sqlite-utils/sqlite_utils/db.py\", line 1169, in add_foreign_keys\r\n self[table].transform(add_foreign_keys=fks)\r\n File \"/Users/simon/Dropbox/Development/sqlite-utils/sqlite_utils/db.py\", line 1728, in transform\r\n sqls = self.transform_sql(\r\n ^^^^^^^^^^^^^^^^^^^\r\n File \"/Users/simon/Dropbox/Development/sqlite-utils/sqlite_utils/db.py\", line 1896, in transform_sql\r\n self.db.create_table_sql(\r\n File \"/Users/simon/Dropbox/Development/sqlite-utils/sqlite_utils/db.py\", line 882, in create_table_sql\r\n raise AlterError(\r\nsqlite_utils.db.AlterError: No such column: breeds.rowid\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1855838223, "label": ".transform() instead of modifying sqlite_master for add_foreign_keys"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/584#issuecomment-1683118376", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/584", "id": 1683118376, "node_id": "IC_kwDOCGYnMM5kUlko", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-17T23:41:10Z", "updated_at": "2023-08-17T23:41:10Z", "author_association": "OWNER", "body": "The problem here is that the table created by this line:\r\n```python\r\nfresh_db.create_table(\"breeds\", {\"name\": str})\r\n```\r\nHas this schema:\r\n```sql\r\nCREATE TABLE [breeds] (\r\n [name] TEXT\r\n);\r\n```\r\nSQLite creates an invisible `rowid` column for it automatically.\r\n\r\nOn the `main` branch with the old implementation that table ends up looking like this after the foreign key has been added:\r\n```sql\r\n(Pdb) print(fresh_db.schema)\r\nCREATE TABLE [dogs] (\r\n [name] TEXT\r\n, [breed_id] INTEGER,\r\n FOREIGN KEY([breed_id]) REFERENCES [breeds]([rowid])\r\n);\r\nCREATE TABLE [breeds] (\r\n [name] TEXT\r\n);\r\n```\r\nBut I think this validation check is failing now: https://github.com/simonw/sqlite-utils/blob/842b61321fc6a9f0bdb913ab138e39d71bf42e00/sqlite_utils/db.py#L875-L884\r\n\r\nHere's what the debugger reveals about this code:\r\n```python\r\n for fk in foreign_keys:\r\n if fk.other_table == name and columns.get(fk.other_column):\r\n continue\r\n if not any(\r\n c for c in self[fk.other_table].columns if c.name == fk.other_column\r\n ):\r\n raise AlterError(\r\n \"No such column: {}.{}\".format(fk.other_table, fk.other_column)\r\n )\r\n```\r\n```\r\n(Pdb) fk\r\nForeignKey(table='dogs', column='breed_id', other_table='breeds', other_column='rowid')\r\n(Pdb) self[fk.other_table].columns\r\n[Column(cid=0, name='name', type='TEXT', notnull=0, default_value=None, is_pk=0)]\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1855838223, "label": ".transform() instead of modifying sqlite_master for add_foreign_keys"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/584#issuecomment-1683122490", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/584", "id": 1683122490, "node_id": "IC_kwDOCGYnMM5kUmk6", "user": {"value": 22429695, "label": "codecov[bot]"}, "created_at": "2023-08-17T23:45:44Z", "updated_at": "2023-08-18T00:46:07Z", "author_association": "NONE", "body": "## [Codecov](https://app.codecov.io/gh/simonw/sqlite-utils/pull/584?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report\nPatch coverage: **`92.85%`** and project coverage change: **`-0.07%`** :warning:\n> Comparison is base [(`1dc6b5a`)](https://app.codecov.io/gh/simonw/sqlite-utils/commit/1dc6b5aa644a92d3654f7068110ed7930989ce71?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) 95.82% compared to head [(`2915050`)](https://app.codecov.io/gh/simonw/sqlite-utils/pull/584?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) 95.76%.\n> Report is 1 commits behind head on main.\n\n
Additional details and impacted files\n\n\n```diff\n@@ Coverage Diff @@\n## main #584 +/- ##\n==========================================\n- Coverage 95.82% 95.76% -0.07% \n==========================================\n Files 8 8 \n Lines 2829 2834 +5 \n==========================================\n+ Hits 2711 2714 +3 \n- Misses 118 120 +2 \n```\n\n\n| [Files Changed](https://app.codecov.io/gh/simonw/sqlite-utils/pull/584?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage \u0394 | |\n|---|---|---|\n| [sqlite\\_utils/db.py](https://app.codecov.io/gh/simonw/sqlite-utils/pull/584?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-c3FsaXRlX3V0aWxzL2RiLnB5) | `97.22% <92.85%> (-0.15%)` | :arrow_down: |\n\n\n
\n\n[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/simonw/sqlite-utils/pull/584?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). \n:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1855838223, "label": ".transform() instead of modifying sqlite_master for add_foreign_keys"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/584#issuecomment-1683122767", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/584", "id": 1683122767, "node_id": "IC_kwDOCGYnMM5kUmpP", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-17T23:46:09Z", "updated_at": "2023-08-17T23:46:09Z", "author_association": "OWNER", "body": "Oops, `mypy` failures:\r\n\r\n```\r\nsqlite_utils/db.py:781: error: Incompatible types in assignment (expression has type \"Tuple[Any, ...]\", variable has type \"Union[str, ForeignKey, Tuple[str, str], Tuple[str, str, str], Tuple[str, str, str, str]]\") [assignment]\r\nsqlite_utils/db.py:1164: error: Need type annotation for \"by_table\" (hint: \"by_table: Dict[, ] = ...\") [var-annotated]\r\nsqlite_utils/db.py:1169: error: Item \"View\" of \"Union[Table, View]\" has no attribute \"transform\" [union-attr]\r\nsqlite_utils/db.py:1813: error: Argument 1 to \"append\" of \"list\" has incompatible type \"ForeignKey\"; expected [arg-type]\r\nsqlite_utils/db.py:1824: error: Argument 1 to \"append\" of \"list\" has incompatible type \"ForeignKey\"; expected [arg-type]\r\nFound 5 errors in 1 file (checked 56 source files)\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1855838223, "label": ".transform() instead of modifying sqlite_master for add_foreign_keys"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/584#issuecomment-1683137259", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/584", "id": 1683137259, "node_id": "IC_kwDOCGYnMM5kUqLr", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-18T00:06:59Z", "updated_at": "2023-08-18T00:06:59Z", "author_association": "OWNER", "body": "The docs still describe the old trick, I need to update that: https://sqlite-utils.datasette.io/en/3.34/python-api.html#adding-foreign-key-constraints", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1855838223, "label": ".transform() instead of modifying sqlite_master for add_foreign_keys"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/584#issuecomment-1683138953", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/584", "id": 1683138953, "node_id": "IC_kwDOCGYnMM5kUqmJ", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-18T00:09:20Z", "updated_at": "2023-08-18T00:09:20Z", "author_association": "OWNER", "body": "Weird, I'm getting a `flake8` problem in CI which doesn't occur on my laptop:\r\n```\r\n./tests/test_recipes.py:99:9: F811 redefinition of unused 'fn' from line 96\r\n./tests/test_recipes.py:127:9: F811 redefinition of unused 'fn' from line 124\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1855838223, "label": ".transform() instead of modifying sqlite_master for add_foreign_keys"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/584#issuecomment-1683139304", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/584", "id": 1683139304, "node_id": "IC_kwDOCGYnMM5kUqro", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-18T00:09:56Z", "updated_at": "2023-08-18T00:09:56Z", "author_association": "OWNER", "body": "Upgrading `flake8` locally replicated the error:\r\n```\r\npip install -U flake8\r\nflake8\r\n```\r\n```\r\n./tests/test_recipes.py:99:9: F811 redefinition of unused 'fn' from line 96\r\n./tests/test_recipes.py:127:9: F811 redefinition of unused 'fn' from line 124\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1855838223, "label": ".transform() instead of modifying sqlite_master for add_foreign_keys"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/584#issuecomment-1683143723", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/584", "id": 1683143723, "node_id": "IC_kwDOCGYnMM5kUrwr", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-18T00:14:52Z", "updated_at": "2023-08-18T00:14:52Z", "author_association": "OWNER", "body": "Another docs update: this bit in here https://sqlite-utils.datasette.io/en/3.34/python-api.html#adding-multiple-foreign-key-constraints-at-once talks about how `.add_foreign_keys()` is a performance optimization to avoid having to run VACUUM a bunch of separate times:\r\n\r\n> The final step in adding a new foreign key to a SQLite database is to run `VACUUM`, to ensure the new foreign key is available in future introspection queries.\r\n> \r\n> `VACUUM` against a large (multi-GB) database can take several minutes or longer. If you are adding multiple foreign keys using `table.add_foreign_key(...)` these can quickly add up.\r\n> \r\n> Instead, you can use `db.add_foreign_keys(...)` to add multiple foreign keys within a single transaction. This method takes a list of four-tuples, each one specifying a `table`, `column`, `other_table` and `other_column`.\r\n\r\nThat doesn't apply any more - the new mechanism using `.transform()` works completely differently, so this issue around running VACUUM no longer applies.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1855838223, "label": ".transform() instead of modifying sqlite_master for add_foreign_keys"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/584#issuecomment-1683145110", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/584", "id": 1683145110, "node_id": "IC_kwDOCGYnMM5kUsGW", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-18T00:16:28Z", "updated_at": "2023-08-18T00:16:48Z", "author_association": "OWNER", "body": "One last piece of documentation: need to document the new option to `table.transform()` and `table.transform_sql()`:\r\n\r\nhttps://github.com/simonw/sqlite-utils/blob/0771ac61fe5c2aca74075b20b1a99b9bd4c65661/sqlite_utils/db.py#L1706-L1708\r\n\r\nI should write tests for them too.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1855838223, "label": ".transform() instead of modifying sqlite_master for add_foreign_keys"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/584#issuecomment-1683145819", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/584", "id": 1683145819, "node_id": "IC_kwDOCGYnMM5kUsRb", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-18T00:17:26Z", "updated_at": "2023-08-18T00:17:26Z", "author_association": "OWNER", "body": "Updated documentation: https://sqlite-utils--584.org.readthedocs.build/en/584/python-api.html#adding-foreign-key-constraints", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1855838223, "label": ".transform() instead of modifying sqlite_master for add_foreign_keys"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/584#issuecomment-1683164661", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/584", "id": 1683164661, "node_id": "IC_kwDOCGYnMM5kUw31", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-18T00:45:53Z", "updated_at": "2023-08-18T00:45:53Z", "author_association": "OWNER", "body": "More updated documentation:\r\n- https://sqlite-utils--584.org.readthedocs.build/en/584/reference.html#sqlite_utils.db.Table.transform\r\n- https://sqlite-utils--584.org.readthedocs.build/en/584/python-api.html#python-api-transform-add-foreign-key-constraints", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1855838223, "label": ".transform() instead of modifying sqlite_master for add_foreign_keys"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/583#issuecomment-1683110636", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/583", "id": 1683110636, "node_id": "IC_kwDOCGYnMM5kUjrs", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-17T23:31:27Z", "updated_at": "2023-08-17T23:31:27Z", "author_association": "OWNER", "body": "Spotted this while working on:\r\n- #577 ", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1855836914, "label": "Get rid of test.utils.collapse_whitespace"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/581#issuecomment-1652496702", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/581", "id": 1652496702, "node_id": "IC_kwDOCGYnMM5ifxk-", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-07-26T21:07:45Z", "updated_at": "2023-07-26T21:07:45Z", "author_association": "OWNER", "body": "Docs: https://sqlite-utils.datasette.io/en/latest/cli.html#using-the-debugger", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1823160748, "label": "`sqlite-utils convert --pdb` option"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/58#issuecomment-710399593", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/58", "id": 710399593, "node_id": "MDEyOklzc3VlQ29tbWVudDcxMDM5OTU5Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-16T18:39:31Z", "updated_at": "2020-10-16T18:39:31Z", "author_association": "OWNER", "body": "I don't think this is valuable enough to justify adding to the library - especially since you can execute FTS search against views by joining to an FTS table built against an underlying table.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 488293926, "label": "Support enabling FTS on views"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/578#issuecomment-1648323482", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/578", "id": 1648323482, "node_id": "IC_kwDOCGYnMM5iP2ua", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-07-24T17:31:56Z", "updated_at": "2023-07-24T17:31:56Z", "author_association": "OWNER", "body": "The main blocker here is coming up with a design. The challenge is cleanly integrating it with the existing format options:\r\n\r\nhttps://github.com/simonw/sqlite-utils/blob/8bee14588687b66c54c7a3dfae5de2f9cc7cac3f/docs/cli-reference.rst#L114-L132\r\n\r\nI'd like to avoid breaking backwards compatibility.\r\n\r\nThe most obvious solution is to add a `--format x` option. This is slightly confusing as `--fmt` already exists.\r\n\r\nOr... `--fmt` could be enhanced to handle plugins too - and the existing set of formats (from the `tabulate` integration) could be refactored to use the new hook.\r\n\r\nIf I ever do release `sqlite-utils 4` I'm tempted to move that tabulate stuff into a plugin.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1818838294, "label": "Plugin hook for adding new output formats"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/578#issuecomment-1648324312", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/578", "id": 1648324312, "node_id": "IC_kwDOCGYnMM5iP27Y", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-07-24T17:32:21Z", "updated_at": "2023-07-24T17:32:21Z", "author_association": "OWNER", "body": "I'm currently leaning very slightly towards `--format geojson` - and explaining in the docs that `--format` is for formats added by plugins.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1818838294, "label": "Plugin hook for adding new output formats"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/578#issuecomment-1648325682", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/578", "id": 1648325682, "node_id": "IC_kwDOCGYnMM5iP3Qy", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-07-24T17:33:10Z", "updated_at": "2023-07-24T17:33:10Z", "author_association": "OWNER", "body": "A related feature would be support for plugins to add new ways of _ingesting_ data - currently `sqlite-utils insert` works against JSON, newline-JSON, CSV and TSV.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1818838294, "label": "Plugin hook for adding new output formats"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/578#issuecomment-1648339661", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/578", "id": 1648339661, "node_id": "IC_kwDOCGYnMM5iP6rN", "user": {"value": 25778, "label": "eyeseast"}, "created_at": "2023-07-24T17:44:30Z", "updated_at": "2023-07-24T17:44:30Z", "author_association": "CONTRIBUTOR", "body": "> A related feature would be support for plugins to add new ways of ingesting data - currently sqlite-utils insert works against JSON, newline-JSON, CSV and TSV.\r\n\r\nThis is my goal, to have one plugin that handles input and output symmetrically. I'd like to be able to do something like this:\r\n\r\n```sh\r\nsqlite-utils insert data.db table file.geojson --format geojson\r\n# ... explore and manipulate in Datasette\r\nsqlite-utils query data.db ... --format geojson > output.geojson\r\n```\r\n\r\nThis would work especially well with [datasette-query-files](https://github.com/eyeseast/datasette-query-files), since I already have the queries I need saved in standalone SQL files.\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1818838294, "label": "Plugin hook for adding new output formats"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/578#issuecomment-1668113177", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/578", "id": 1668113177, "node_id": "IC_kwDOCGYnMM5jbWMZ", "user": {"value": 25778, "label": "eyeseast"}, "created_at": "2023-08-07T15:41:49Z", "updated_at": "2023-08-07T15:41:49Z", "author_association": "CONTRIBUTOR", "body": "I wonder if this should be two hooks: input and output. The current `--csv` (and `--tsv`) options apply to both. Haven't looked at how it's implemented. Or maybe it's one hook that returns a format for reading and for writing.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1818838294, "label": "Plugin hook for adding new output formats"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/577#issuecomment-1683066934", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/577", "id": 1683066934, "node_id": "IC_kwDOCGYnMM5kUZA2", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-17T22:37:18Z", "updated_at": "2023-08-17T22:37:18Z", "author_association": "OWNER", "body": "I'm certain this could work.\r\n\r\nIt turns out the `.transform()` method already has code that creates the new table with a copy of foreign keys from the old one - dropping any foreign keys that were specified in the `drop_foreign_keys=` parameter:\r\n\r\nhttps://github.com/simonw/sqlite-utils/blob/1dc6b5aa644a92d3654f7068110ed7930989ce71/sqlite_utils/db.py#L1850-L1872\r\n\r\nImproving this code to support adding foreign keys as well would be pretty simple.\r\n\r\nAnd then the `.add_foreign_keys()` and `.add_foreign_key()` methods could be updated to use `.transform(...)` under the hood instead.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1817289521, "label": "Get `add_foreign_keys()` to work without modifying `sqlite_master`"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/577#issuecomment-1683068505", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/577", "id": 1683068505, "node_id": "IC_kwDOCGYnMM5kUZZZ", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-17T22:39:17Z", "updated_at": "2023-08-17T22:39:38Z", "author_association": "OWNER", "body": "This would help address these issues, among potentially many others:\r\n- https://github.com/simonw/llm/issues/60\r\n- https://github.com/simonw/llm/issues/116\r\n- https://github.com/simonw/llm/issues/123", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1817289521, "label": "Get `add_foreign_keys()` to work without modifying `sqlite_master`"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/577#issuecomment-1683071519", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/577", "id": 1683071519, "node_id": "IC_kwDOCGYnMM5kUaIf", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-17T22:42:28Z", "updated_at": "2023-08-17T22:42:28Z", "author_association": "OWNER", "body": "Looking at the whole of the `.add_foreign_keys()` method, the first section of it can remain unchanged - it's just a bunch of validation:\r\n\r\nhttps://github.com/simonw/sqlite-utils/blob/13ebcc575d2547c45e8d31288b71a3242c16b886/sqlite_utils/db.py#L1106-L1149\r\n\r\nAt that point we have `foreign_keys_to_create` as the ones that are new, but we should instead try to build up a `foreign_keys` which is both new and old, ready to be passed to `.transform()`.\r\n\r\nHere's the rest of that function, which will be replaced by a called to `.transform(foreign_keys=foreign_keys)`:\r\n\r\nhttps://github.com/simonw/sqlite-utils/blob/13ebcc575d2547c45e8d31288b71a3242c16b886/sqlite_utils/db.py#L1151-L1177", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1817289521, "label": "Get `add_foreign_keys()` to work without modifying `sqlite_master`"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/577#issuecomment-1683074009", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/577", "id": 1683074009, "node_id": "IC_kwDOCGYnMM5kUavZ", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-17T22:45:29Z", "updated_at": "2023-08-17T22:47:08Z", "author_association": "OWNER", "body": "Actually I think `table.transform()` might get the following optional arguments:\r\n```python\r\n def transform(\r\n self,\r\n *,\r\n # ...\r\n # This one exists already:\r\n drop_foreign_keys: Optional[Iterable] = None,\r\n # These two are new. This one specifies keys to add:\r\n add_foreign_keys: Optional[ForeignKeysType] = None,\r\n # Or this one causes them all to be replaced with the new definitions:\r\n foreign_keys: Optional[ForeignKeysType] = None,\r\n```\r\nThere should be validation that forbids you from using `foreign_keys=` at the same time as either `drop_foreign_keys=` or `add_foreign_keys=` because the point of `foreign_keys=` is to define the keys for the new table all in one go.\r\n\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1817289521, "label": "Get `add_foreign_keys()` to work without modifying `sqlite_master`"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/577#issuecomment-1683074546", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/577", "id": 1683074546, "node_id": "IC_kwDOCGYnMM5kUa3y", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-17T22:46:18Z", "updated_at": "2023-08-17T22:46:18Z", "author_association": "OWNER", "body": "Maybe this:\r\n```python\r\n drop_foreign_keys: Optional[Iterable] = None,\r\n```\r\nShould be this:\r\n```python\r\n drop_foreign_keys: Optional[Iterable[str]] = None,\r\n```\r\nBecause it takes a list of column names that should have their foreign keys dropped.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1817289521, "label": "Get `add_foreign_keys()` to work without modifying `sqlite_master`"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/577#issuecomment-1683074857", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/577", "id": 1683074857, "node_id": "IC_kwDOCGYnMM5kUa8p", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-17T22:46:40Z", "updated_at": "2023-08-17T22:46:40Z", "author_association": "OWNER", "body": "As a reminder:\r\nhttps://github.com/simonw/sqlite-utils/blob/1dc6b5aa644a92d3654f7068110ed7930989ce71/sqlite_utils/db.py#L159-L165", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1817289521, "label": "Get `add_foreign_keys()` to work without modifying `sqlite_master`"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/577#issuecomment-1683076325", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/577", "id": 1683076325, "node_id": "IC_kwDOCGYnMM5kUbTl", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-17T22:48:36Z", "updated_at": "2023-08-17T22:48:36Z", "author_association": "OWNER", "body": "I'm inclined to just go with the `.transform()` method and not attempt to keep around the method that involves updating `sqlite_master` and then add code to detect if that's possible (or catch if it fails) and fall back on the other mechanism.\r\n\r\nIt would be nice to drop some code complexity, plus I don't yet have a way of running automated tests against Python + SQLite versions that exhibit the problem.", "reactions": "{\"total_count\": 1, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 1, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1817289521, "label": "Get `add_foreign_keys()` to work without modifying `sqlite_master`"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/577#issuecomment-1683098094", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/577", "id": 1683098094, "node_id": "IC_kwDOCGYnMM5kUgnu", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-17T23:15:36Z", "updated_at": "2023-08-17T23:15:36Z", "author_association": "OWNER", "body": "An interesting side-effect of this change is that it does result in a slightly different schema - e.g. this test: https://github.com/simonw/sqlite-utils/blob/1dc6b5aa644a92d3654f7068110ed7930989ce71/tests/test_extract.py#L118-L133\r\n\r\nNeeds updating like so:\r\n```diff\r\ndiff --git a/tests/test_extract.py b/tests/test_extract.py\r\nindex 70ad0cf..fd52534 100644\r\n--- a/tests/test_extract.py\r\n+++ b/tests/test_extract.py\r\n@@ -127,8 +127,7 @@ def test_extract_rowid_table(fresh_db):\r\n assert fresh_db[\"tree\"].schema == (\r\n 'CREATE TABLE \"tree\" (\\n'\r\n \" [name] TEXT,\\n\"\r\n- \" [common_name_latin_name_id] INTEGER,\\n\"\r\n- \" FOREIGN KEY([common_name_latin_name_id]) REFERENCES [common_name_latin_name]([id])\\n\"\r\n+ \" [common_name_latin_name_id] INTEGER REFERENCES [common_name_latin_name]([id])\\n\"\r\n \")\"\r\n )\r\n assert (\r\n```\r\nUnfortunately this means it may break other test suites that depend on `sqlite-utils` that have schema tests like this baked in.\r\n\r\nI don't think this should count as a breaking change release though, but it's still worth noting.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1817289521, "label": "Get `add_foreign_keys()` to work without modifying `sqlite_master`"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/577#issuecomment-1684235760", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/577", "id": 1684235760, "node_id": "IC_kwDOCGYnMM5kY2Xw", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-18T17:43:11Z", "updated_at": "2023-08-18T17:43:11Z", "author_association": "OWNER", "body": "Here's a new plugin that brings back the `sqlite_master` modifying version, for those that can use it:\r\n\r\nhttps://github.com/simonw/sqlite-utils-fast-fks", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1817289521, "label": "Get `add_foreign_keys()` to work without modifying `sqlite_master`"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/576#issuecomment-1646752814", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/576", "id": 1646752814, "node_id": "IC_kwDOCGYnMM5iJ3Qu", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-07-23T05:43:13Z", "updated_at": "2023-07-23T05:43:13Z", "author_association": "OWNER", "body": "Using `llm` in my `sqlite-utils` local checkout:\r\n```bash\r\ngit diff 0.2 0.3 | llm -s 'release notes as markdown bullet points'\r\n```\r\n> - Added a new `Table` class in the `db` module which represents a table in a SQLite database\r\n> - The `Table` class has properties that provide information about the table, such as column names, foreign keys, and schema\r\n> - The `Table` class also has a `count` property that returns the number of rows in the table\r\n> - The `Table` class has a `create` method that can be used to create a new table with specified columns, primary key, and foreign keys\r\n> - Updated the `Database` class in the `__init__` module to include the `Table` class and expose it via the `__all__` variable\r\n> - Added new tests for creating tables and introspecting tables\r\n\r\n```bash\r\ngit diff 0.3 0.3.1 | llm -s 'release notes as markdown bullet points'\r\n```\r\n\r\n> - Updated PyPI version badge to display the latest version\r\n> - Updated Travis CI badge to link to the master branch\r\n> - Added a badge for documentation status\r\n> - Added a license badge\r\n> - Added a description of the package\r\n> - Added installation instructions\r\n> - Added a link to the documentation\r\n> - Added a \"Related projects\" section and linked to Datasette and csvs-to-sqlite\r\n> - Updated the version in setup.py to 0.3.1\r\n\r\n```bash\r\ngit diff 0.3.1 0.4 | llm -s 'release notes as markdown bullet points'\r\n```\r\n\r\n> - Added an `enable_fts` method to the `Table` class that enables full-text search on specified columns\r\n> - Added a `populate_fts` method to the `Table` class that refreshes the search index for full-text search\r\n> - Added a `search` method to the `Table` class that allows searching the table using full-text search\r\n> - Updated the `setup.py` script to version 0.4\r\n> - Updated the `db.py` file to include the `enable_fts`, `populate_fts`, and `search` methods\r\n> - Added test cases for the `enable_fts`, `populate_fts`, and `search` methods in the `test_enable_fts.py` file\r\n> - Added test cases for the `count`, `columns`, and `schema` properties in the `test_introspect.py` file\r\n> ", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1816997390, "label": "Backfill the release notes prior to 0.4"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/576#issuecomment-1646753678", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/576", "id": 1646753678, "node_id": "IC_kwDOCGYnMM5iJ3eO", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-07-23T05:49:51Z", "updated_at": "2023-07-23T05:49:51Z", "author_association": "OWNER", "body": "Done - bottom of https://sqlite-utils.datasette.io/en/latest/changelog.html#id116", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1816997390, "label": "Backfill the release notes prior to 0.4"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/575#issuecomment-1646687461", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/575", "id": 1646687461, "node_id": "IC_kwDOCGYnMM5iJnTl", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-07-22T23:01:44Z", "updated_at": "2023-07-22T23:01:44Z", "author_association": "OWNER", "body": "Relevant code: https://github.com/simonw/sqlite-utils/blob/3f80a026983d3e634f05a46f2a6da162b5139dd9/sqlite_utils/db.py#L346", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1816919568, "label": "Python API ability to opt-out of connection plugins"}, "performed_via_github_app": null}