{"html_url": "https://github.com/simonw/sqlite-utils/issues/235#issuecomment-1223378004", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/235", "id": 1223378004, "node_id": "IC_kwDOCGYnMM5I60RU", "user": {"value": 1558033, "label": "wpears"}, "created_at": "2022-08-23T00:44:11Z", "updated_at": "2022-08-23T00:44:11Z", "author_association": "NONE", "body": "This bug affects me as well. Env:\r\n\r\n```\r\nPython 3.8.12\r\nsqlite-utils, version 3.28\r\nsqlite3 3.32.3\r\nMacOS Big Sur 11.6.7\r\nIntel\r\n```\r\n\r\nSimilar to @mdrovdahl, I was able to work around this bug by piping the SQL string constructed in `add_foreign_keys` to the `sqlite3` command itself. Specifically, if you're trying to patch this yourself, replace [lines 1026-1039 of db.py in your site packages](https://github.com/simonw/sqlite-utils/blob/main/sqlite_utils/db.py#L1026-L1039) with something similar to the following:\r\n\r\n```\r\nprint(\"PRAGMA writable_schema = 1;\")\r\nfor table_name, new_sql in table_sql.items():\r\n    print(\"UPDATE sqlite_master SET sql = '{}' WHERE name = '{}';\".format(\r\n         new_sql, table_name)\r\n    )\r\nprint(\"PRAGMA writable_schema = 0;\")\r\nprint(\"VACUUM;\")\r\n```\r\n \r\nThen from your terminal:\r\n`db-to-sqlite \"<connection string>\"  your.db --all > output.sql && sqlite3 your.db < output.sql`\r\n\r\nIf you want to run this with `-p`, you'll have to actually open a file in code to write to instead of redirecting the output.", "reactions": "{\"total_count\": 3, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 3, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 810618495, "label": "Extract columns cannot create foreign key relation: sqlite3.OperationalError: table sqlite_master may not be modified"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/235#issuecomment-1206241356", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/235", "id": 1206241356, "node_id": "IC_kwDOCGYnMM5H5chM", "user": {"value": 503614, "label": "lfdebrux"}, "created_at": "2022-08-05T09:26:15Z", "updated_at": "2022-08-05T09:29:42Z", "author_association": "NONE", "body": "I am getting the same error when using github-to-sqlite (which uses sqlite-utils internally). I am also using an M1 MacBook Pro, with macOS Monterey 12.5, with Python 3.10.6 for arm64 installed using pyenv. I have sqlite-utils 3.28 installed.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 810618495, "label": "Extract columns cannot create foreign key relation: sqlite3.OperationalError: table sqlite_master may not be modified"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/235#issuecomment-1198414383", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/235", "id": 1198414383, "node_id": "IC_kwDOCGYnMM5Hblov", "user": {"value": 474467, "label": "mdrovdahl"}, "created_at": "2022-07-28T17:10:06Z", "updated_at": "2022-07-28T17:10:06Z", "author_association": "NONE", "body": "I was able to fight through this by capturing the SQL commands from the `add_foreign_keys()` function in sqlite-utils and then executing them manually via the sqlite3 client, first setting PRAGMA writable_schema on and then updating the sqlite_master table. Still no clue why they were failing when run in context...", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 810618495, "label": "Extract columns cannot create foreign key relation: sqlite3.OperationalError: table sqlite_master may not be modified"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/235#issuecomment-1172766270", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/235", "id": 1172766270, "node_id": "IC_kwDOCGYnMM5F5v4-", "user": {"value": 474467, "label": "mdrovdahl"}, "created_at": "2022-07-01T22:40:26Z", "updated_at": "2022-07-01T22:40:26Z", "author_association": "NONE", "body": "Note, I do not get this issue using my Intel MacBook Pro =/\r\n\r\nEnvironment\r\n```\r\nmarkd@Marks-MBP metabase % python3 --version\r\nPython 3.9.13\r\nmarkd@Marks-MBP metabase % sqlite3 --version\r\n3.37.0 2021-12-09 01:34:53 9ff244ce0739f8ee52a3e9671adb4ee54c83c640b02e3f9d185fd2f9a179aapl\r\nmarkd@Marks-MBP metabase % sqlite-utils --version\r\nsqlite-utils, version 3.27\r\nmarkd@Marks-MBP metabase % sqlite3 github.db \r\nSQLite version 3.37.0 2021-12-09 01:34:53\r\nEnter \".help\" for usage hints.\r\nsqlite> .dbconfig\r\n          defensive off\r\n            dqs_ddl on\r\n            dqs_dml on\r\n        enable_fkey off\r\n        enable_qpsg off\r\n     enable_trigger on\r\n        enable_view on\r\n     fts3_tokenizer off\r\n legacy_alter_table on\r\n legacy_file_format off\r\n     load_extension off\r\n   no_ckpt_on_close off\r\n     reset_database off\r\n        trigger_eqp off\r\n     trusted_schema on\r\n    writable_schema off\r\n```\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 810618495, "label": "Extract columns cannot create foreign key relation: sqlite3.OperationalError: table sqlite_master may not be modified"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/235#issuecomment-1172697090", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/235", "id": 1172697090, "node_id": "IC_kwDOCGYnMM5F5fAC", "user": {"value": 474467, "label": "mdrovdahl"}, "created_at": "2022-07-01T20:37:40Z", "updated_at": "2022-07-01T20:37:55Z", "author_association": "NONE", "body": "I just ran into what appears to be the same issue on a MacBook Pro, M1 Pro. \r\n\r\nEnvironment:\r\n```\r\nmarkd@Marks-MacBook-Pro metabase % python --version                                                             \r\nPython 3.8.9\r\nmarkd@Marks-MacBook-Pro metabase % sqlite3 --version                                                            \r\n3.37.0 2021-12-09 01:34:53 9ff244ce0739f8ee52a3e9671adb4ee54c83c640b02e3f9d185fd2f9a179aapl\r\nmarkd@Marks-MacBook-Pro metabase % sqlite-utils --version                                                        \r\nsqlite-utils, version 3.27\r\nmarkd@Marks-MacBook-Pro metabase % sqlite3 gh.db \r\nSQLite version 3.37.0 2021-12-09 01:34:53\r\nEnter \".help\" for usage hints.\r\nsqlite> .dbconfig\r\n          defensive off\r\n            dqs_ddl on\r\n            dqs_dml on\r\n        enable_fkey off\r\n        enable_qpsg off\r\n     enable_trigger on\r\n        enable_view on\r\n     fts3_tokenizer off\r\n legacy_alter_table on\r\n legacy_file_format off\r\n     load_extension off\r\n   no_ckpt_on_close off\r\n     reset_database off\r\n        trigger_eqp off\r\n     trusted_schema on\r\n    writable_schema off\r\n``` \r\n\r\nError\r\n```\r\nmarkd@Marks-MacBook-Pro metabase % github-to-sqlite repos gh.db a8cteam51                                        \r\nTraceback (most recent call last):\r\n  File \"/Users/markd/Library/Python/3.8/bin/github-to-sqlite\", line 8, in <module>\r\n    sys.exit(cli())\r\n  File \"/Users/markd/Library/Python/3.8/lib/python/site-packages/click/core.py\", line 1130, in __call__\r\n    return self.main(*args, **kwargs)\r\n  File \"/Users/markd/Library/Python/3.8/lib/python/site-packages/click/core.py\", line 1055, in main\r\n    rv = self.invoke(ctx)\r\n  File \"/Users/markd/Library/Python/3.8/lib/python/site-packages/click/core.py\", line 1657, in invoke\r\n    return _process_result(sub_ctx.command.invoke(sub_ctx))\r\n  File \"/Users/markd/Library/Python/3.8/lib/python/site-packages/click/core.py\", line 1404, in invoke\r\n    return ctx.invoke(self.callback, **ctx.params)\r\n  File \"/Users/markd/Library/Python/3.8/lib/python/site-packages/click/core.py\", line 760, in invoke\r\n    return __callback(*args, **kwargs)\r\n  File \"/Users/markd/Library/Python/3.8/lib/python/site-packages/github_to_sqlite/cli.py\", line 268, in repos\r\n    utils.ensure_db_shape(db)\r\n  File \"/Users/markd/Library/Python/3.8/lib/python/site-packages/github_to_sqlite/utils.py\", line 688, in ensure_db_shape\r\n    ensure_foreign_keys(db)\r\n  File \"/Users/markd/Library/Python/3.8/lib/python/site-packages/github_to_sqlite/utils.py\", line 682, in ensure_foreign_keys\r\n    db[table].add_foreign_key(column, table2, column2)\r\n  File \"/Users/markd/Library/Python/3.8/lib/python/site-packages/sqlite_utils/db.py\", line 2004, in add_foreign_key\r\n    self.db.add_foreign_keys([(self.name, column, other_table, other_column)])\r\n  File \"/Users/markd/Library/Python/3.8/lib/python/site-packages/sqlite_utils/db.py\", line 1019, in add_foreign_keys\r\n    cursor.execute(\r\nsqlite3.OperationalError: table sqlite_master may not be modified\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 810618495, "label": "Extract columns cannot create foreign key relation: sqlite3.OperationalError: table sqlite_master may not be modified"}, "performed_via_github_app": null}