{"html_url": "https://github.com/simonw/sqlite-utils/issues/577#issuecomment-1683066934", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/577", "id": 1683066934, "node_id": "IC_kwDOCGYnMM5kUZA2", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-17T22:37:18Z", "updated_at": "2023-08-17T22:37:18Z", "author_association": "OWNER", "body": "I'm certain this could work.\r\n\r\nIt turns out the `.transform()` method already has code that creates the new table with a copy of foreign keys from the old one - dropping any foreign keys that were specified in the `drop_foreign_keys=` parameter:\r\n\r\nhttps://github.com/simonw/sqlite-utils/blob/1dc6b5aa644a92d3654f7068110ed7930989ce71/sqlite_utils/db.py#L1850-L1872\r\n\r\nImproving this code to support adding foreign keys as well would be pretty simple.\r\n\r\nAnd then the `.add_foreign_keys()` and `.add_foreign_key()` methods could be updated to use `.transform(...)` under the hood instead.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1817289521, "label": "Get `add_foreign_keys()` to work without modifying `sqlite_master`"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/577#issuecomment-1683068505", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/577", "id": 1683068505, "node_id": "IC_kwDOCGYnMM5kUZZZ", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-17T22:39:17Z", "updated_at": "2023-08-17T22:39:38Z", "author_association": "OWNER", "body": "This would help address these issues, among potentially many others:\r\n- https://github.com/simonw/llm/issues/60\r\n- https://github.com/simonw/llm/issues/116\r\n- https://github.com/simonw/llm/issues/123", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1817289521, "label": "Get `add_foreign_keys()` to work without modifying `sqlite_master`"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/577#issuecomment-1683071519", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/577", "id": 1683071519, "node_id": "IC_kwDOCGYnMM5kUaIf", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-17T22:42:28Z", "updated_at": "2023-08-17T22:42:28Z", "author_association": "OWNER", "body": "Looking at the whole of the `.add_foreign_keys()` method, the first section of it can remain unchanged - it's just a bunch of validation:\r\n\r\nhttps://github.com/simonw/sqlite-utils/blob/13ebcc575d2547c45e8d31288b71a3242c16b886/sqlite_utils/db.py#L1106-L1149\r\n\r\nAt that point we have `foreign_keys_to_create` as the ones that are new, but we should instead try to build up a `foreign_keys` which is both new and old, ready to be passed to `.transform()`.\r\n\r\nHere's the rest of that function, which will be replaced by a called to `.transform(foreign_keys=foreign_keys)`:\r\n\r\nhttps://github.com/simonw/sqlite-utils/blob/13ebcc575d2547c45e8d31288b71a3242c16b886/sqlite_utils/db.py#L1151-L1177", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1817289521, "label": "Get `add_foreign_keys()` to work without modifying `sqlite_master`"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/577#issuecomment-1683074009", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/577", "id": 1683074009, "node_id": "IC_kwDOCGYnMM5kUavZ", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-17T22:45:29Z", "updated_at": "2023-08-17T22:47:08Z", "author_association": "OWNER", "body": "Actually I think `table.transform()` might get the following optional arguments:\r\n```python\r\n    def transform(\r\n        self,\r\n        *,\r\n        # ...\r\n        # This one exists already:\r\n        drop_foreign_keys: Optional[Iterable] = None,\r\n        # These two are new. This one specifies keys to add:\r\n        add_foreign_keys: Optional[ForeignKeysType] = None,\r\n        # Or this one causes them all to be replaced with the new definitions:\r\n        foreign_keys: Optional[ForeignKeysType] = None,\r\n```\r\nThere should be validation that forbids you from using `foreign_keys=` at the same time as either `drop_foreign_keys=` or `add_foreign_keys=` because the point of `foreign_keys=` is to define the keys for the new table all in one go.\r\n\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1817289521, "label": "Get `add_foreign_keys()` to work without modifying `sqlite_master`"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/577#issuecomment-1683074546", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/577", "id": 1683074546, "node_id": "IC_kwDOCGYnMM5kUa3y", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-17T22:46:18Z", "updated_at": "2023-08-17T22:46:18Z", "author_association": "OWNER", "body": "Maybe this:\r\n```python\r\n        drop_foreign_keys: Optional[Iterable] = None,\r\n```\r\nShould be this:\r\n```python\r\n        drop_foreign_keys: Optional[Iterable[str]] = None,\r\n```\r\nBecause it takes a list of column names that should have their foreign keys dropped.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1817289521, "label": "Get `add_foreign_keys()` to work without modifying `sqlite_master`"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/577#issuecomment-1683074857", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/577", "id": 1683074857, "node_id": "IC_kwDOCGYnMM5kUa8p", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-17T22:46:40Z", "updated_at": "2023-08-17T22:46:40Z", "author_association": "OWNER", "body": "As a reminder:\r\nhttps://github.com/simonw/sqlite-utils/blob/1dc6b5aa644a92d3654f7068110ed7930989ce71/sqlite_utils/db.py#L159-L165", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1817289521, "label": "Get `add_foreign_keys()` to work without modifying `sqlite_master`"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/577#issuecomment-1683076325", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/577", "id": 1683076325, "node_id": "IC_kwDOCGYnMM5kUbTl", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-17T22:48:36Z", "updated_at": "2023-08-17T22:48:36Z", "author_association": "OWNER", "body": "I'm inclined to just go with the `.transform()` method and not attempt to keep around the method that involves updating `sqlite_master` and then add code to detect if that's possible (or catch if it fails) and fall back on the other mechanism.\r\n\r\nIt would be nice to drop some code complexity, plus I don't yet have a way of running automated tests against Python + SQLite versions that exhibit the problem.", "reactions": "{\"total_count\": 1, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 1, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1817289521, "label": "Get `add_foreign_keys()` to work without modifying `sqlite_master`"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/577#issuecomment-1683098094", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/577", "id": 1683098094, "node_id": "IC_kwDOCGYnMM5kUgnu", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-17T23:15:36Z", "updated_at": "2023-08-17T23:15:36Z", "author_association": "OWNER", "body": "An interesting side-effect of this change is that it does result in a slightly different schema - e.g. this test: https://github.com/simonw/sqlite-utils/blob/1dc6b5aa644a92d3654f7068110ed7930989ce71/tests/test_extract.py#L118-L133\r\n\r\nNeeds updating like so:\r\n```diff\r\ndiff --git a/tests/test_extract.py b/tests/test_extract.py\r\nindex 70ad0cf..fd52534 100644\r\n--- a/tests/test_extract.py\r\n+++ b/tests/test_extract.py\r\n@@ -127,8 +127,7 @@ def test_extract_rowid_table(fresh_db):\r\n     assert fresh_db[\"tree\"].schema == (\r\n         'CREATE TABLE \"tree\" (\\n'\r\n         \"   [name] TEXT,\\n\"\r\n-        \"   [common_name_latin_name_id] INTEGER,\\n\"\r\n-        \"   FOREIGN KEY([common_name_latin_name_id]) REFERENCES [common_name_latin_name]([id])\\n\"\r\n+        \"   [common_name_latin_name_id] INTEGER REFERENCES [common_name_latin_name]([id])\\n\"\r\n         \")\"\r\n     )\r\n     assert (\r\n```\r\nUnfortunately this means it may break other test suites that depend on `sqlite-utils` that have schema tests like this baked in.\r\n\r\nI don't think this should count as a breaking change release though, but it's still worth noting.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1817289521, "label": "Get `add_foreign_keys()` to work without modifying `sqlite_master`"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/577#issuecomment-1684235760", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/577", "id": 1684235760, "node_id": "IC_kwDOCGYnMM5kY2Xw", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-18T17:43:11Z", "updated_at": "2023-08-18T17:43:11Z", "author_association": "OWNER", "body": "Here's a new plugin that brings back the `sqlite_master` modifying version, for those that can use it:\r\n\r\nhttps://github.com/simonw/sqlite-utils-fast-fks", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1817289521, "label": "Get `add_foreign_keys()` to work without modifying `sqlite_master`"}, "performed_via_github_app": null}