html_url,issue_url,id,node_id,user,user_label,created_at,updated_at,author_association,body,reactions,issue,issue_label,performed_via_github_app
https://github.com/simonw/sqlite-utils/issues/235#issuecomment-1606415188,https://api.github.com/repos/simonw/sqlite-utils/issues/235,1606415188,IC_kwDOCGYnMM5fv_NU,9599,simonw,2023-06-26T01:46:47Z,2023-06-26T01:47:01Z,OWNER,"I just tested this in a brand new virtual environment using the macOS Python 3:
```bash
pipenv shell --python /Applications/Xcode.app/Contents/Developer/usr/bin/python3
```
Then in that virtual environment I ran:
```bash
pip install sqlite-utils
# Confirm the right one is on the path:
which sqlite-utils
curl ""https://data.nasa.gov/resource/y77d-th95.json"" | \
sqlite-utils insert meteorites.db meteorites - --pk=id
sqlite-utils extract meteorites.db meteorites recclass
```
This threw the same error reported above. Then I did this:
```bash
rm meteorites.db
pip install sqlean.py
curl ""https://data.nasa.gov/resource/y77d-th95.json"" | \
sqlite-utils insert meteorites.db meteorites - --pk=id
sqlite-utils extract meteorites.db meteorites recclass
```
And that second time it worked correctly.","{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 1, ""rocket"": 0, ""eyes"": 0}",810618495,Extract columns cannot create foreign key relation: sqlite3.OperationalError: table sqlite_master may not be modified,
https://github.com/simonw/sqlite-utils/issues/235#issuecomment-1606411508,https://api.github.com/repos/simonw/sqlite-utils/issues/235,1606411508,IC_kwDOCGYnMM5fv-T0,9599,simonw,2023-06-26T01:42:10Z,2023-06-26T01:42:22Z,OWNER,https://sqlite-utils.datasette.io/en/stable/changelog.html#v3-33 - upgrading to `sqlite-utils>=3.33` and then installing both `sqlean.py` and `sqlite-dump` in the same virtual environment as `sqlite-utils` should fix this issue.,"{""total_count"": 2, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 2, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",810618495,Extract columns cannot create foreign key relation: sqlite3.OperationalError: table sqlite_master may not be modified,
https://github.com/simonw/sqlite-utils/issues/235#issuecomment-1604379952,https://api.github.com/repos/simonw/sqlite-utils/issues/235,1604379952,IC_kwDOCGYnMM5foOUw,9599,simonw,2023-06-23T14:39:55Z,2023-06-23T15:39:32Z,OWNER,"Ideally a workaround for this right now would be to install `pysqlite3` in the same virtual environment:
sqlite-utils install pysqlite3-binary
But `pysqlite3-binary` doesn't yet ship a wheel for macOS so this probably won't work for most people.
The ""easiest"" fix at the moment is to use Python from Homebrew - so `brew install sqlite-utils` for example won't suffer from this problem. Not a great solution for people who aren't using Homebrew though!","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",810618495,Extract columns cannot create foreign key relation: sqlite3.OperationalError: table sqlite_master may not be modified,
https://github.com/simonw/sqlite-utils/issues/235#issuecomment-1504288134,https://api.github.com/repos/simonw/sqlite-utils/issues/235,1504288134,IC_kwDOCGYnMM5ZqZ2G,9599,simonw,2023-04-11T23:55:06Z,2023-04-12T03:34:32Z,OWNER,"Also checked the official Datasette Docker image - I had to run that in Codespaces because it doesn't currently work on my M2 Mac:
```
codespace@codespaces-112c61:/workspaces/sqlite-utils$ docker pull datasetteproject/datasette
Using default tag: latest
...
codespace@codespaces-112c61:/workspaces/sqlite-utils$ docker run -it datasetteproject/datasette /
bin/bash
root@75ba34f501ec:/# python
Python 3.11.0 (main, Dec 6 2022, 13:31:55) [GCC 10.2.1 20210110] on linux
Type ""help"", ""copyright"", ""credits"" or ""license"" for more information.
>>> import sqlite3
.executescript(""""""
PRAGMA writable_schema = 1;
UPDATE sqlite_master SET sql = 'CREATE TABLE [foos] (id integer primary key)';
PRAGMA writable_schema = 0;
"""""")>>> db = sqlite3.connect("":memory:"")
>>> db.executescript(""""""
... PRAGMA writable_schema = 1;
... UPDATE sqlite_master SET sql = 'CREATE TABLE [foos] (id integer primary key)';
... PRAGMA writable_schema = 0;
... """""")
>>>
```
So that confirms that the official image also has a Python with a SQLite that's not in defensive mode.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",810618495,Extract columns cannot create foreign key relation: sqlite3.OperationalError: table sqlite_master may not be modified,
https://github.com/simonw/sqlite-utils/issues/235#issuecomment-1504245029,https://api.github.com/repos/simonw/sqlite-utils/issues/235,1504245029,IC_kwDOCGYnMM5ZqPUl,9599,simonw,2023-04-11T23:13:41Z,2023-04-11T23:14:39Z,OWNER,"I also tested this against the current `ubuntu:latest` Docker image (on an M2 Mac), in Python 3.10 and 3.11:
```
docker run -it ubuntu:latest /bin/bash
```
Then in the container:
```
apt-get update
apt-get install python3
python3
# pasted in the above recipe
apt install software-properties-common
add-apt-repository ppa:deadsnakes/ppa
apt install python3.11
python3.11
# pasted it in again
```
In both cases the Python code did not raise an exception, which suggests that on Ubuntu those two Python versions do not have the defensive mode set.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",810618495,Extract columns cannot create foreign key relation: sqlite3.OperationalError: table sqlite_master may not be modified,
https://github.com/simonw/sqlite-utils/issues/235#issuecomment-1502559442,https://api.github.com/repos/simonw/sqlite-utils/issues/235,1502559442,IC_kwDOCGYnMM5ZjzzS,9599,simonw,2023-04-11T01:32:30Z,2023-04-11T01:33:27Z,OWNER,"This seems to work:
```python
import sqlite3
db = sqlite3.connect("":memory:"")
db.executescript(""""""
PRAGMA writable_schema = 1;
UPDATE sqlite_master SET sql = 'CREATE TABLE [foos] (id integer primary key)';
PRAGMA writable_schema = 0;
"""""")
```
It succeeds on my Python 3.11 and raises the following exception on my broken Python 3.9:
```
sqlite3.OperationalError: table sqlite_master may not be modified
```
Removing the `PRAGMA writable_schema = 1;` causes the same exception to be raised on both Pythons.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",810618495,Extract columns cannot create foreign key relation: sqlite3.OperationalError: table sqlite_master may not be modified,
https://github.com/simonw/sqlite-utils/issues/235#issuecomment-1502557629,https://api.github.com/repos/simonw/sqlite-utils/issues/235,1502557629,IC_kwDOCGYnMM5ZjzW9,9599,simonw,2023-04-11T01:30:12Z,2023-04-11T01:30:12Z,OWNER,"I'll ask on the SQLite forum if it's possible to toggle that mode on and off using regular SQL. My hunch is that it isn't.
In which case `sqlite-utils` should at least know how to catch this error and display a much more readable error message, maybe with a link to further documentation.
A utility function that can detect this mode would be really useful too. I'd probably have to do a test that tries to modify `sqlite_master` on a new in-memory database to catch if it's possible or not.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",810618495,Extract columns cannot create foreign key relation: sqlite3.OperationalError: table sqlite_master may not be modified,
https://github.com/simonw/sqlite-utils/issues/235#issuecomment-1502556111,https://api.github.com/repos/simonw/sqlite-utils/issues/235,1502556111,IC_kwDOCGYnMM5Zjy_P,9599,simonw,2023-04-11T01:28:41Z,2023-04-11T01:28:41Z,OWNER,"Investigating this one now.
The `sqlite-utils` test suite passes without errors on my Python 3.11.2 installation... but it fails with this error on a Python 3.9.6 installation.
In the broken version's virtual environment directory I ran this:
```
cat pyvenv.cfg
home = /Applications/Xcode.app/Contents/Developer/usr/bin
implementation = CPython
version_info = 3.9.6.final.0
virtualenv = 20.17.1
include-system-site-packages = false
base-prefix = /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9
base-exec-prefix = /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9
base-executable = /Applications/Xcode.app/Contents/Developer/usr/bin/python3
```
So it looks like the Xcode `python3` has ""defensive"" mode turned on for SQLite.
As far as I can tell there's no way to turn it OFF again in Python.
My virtual environment that DOES work has this:
```
home = /opt/homebrew/opt/python@3.11/bin
implementation = CPython
version_info = 3.11.2.final.0
virtualenv = 20.17.1
include-system-site-packages = false
base-prefix = /opt/homebrew/opt/python@3.11/Frameworks/Python.framework/Versions/3.11
base-exec-prefix = /opt/homebrew/opt/python@3.11/Frameworks/Python.framework/Versions/3.11
base-executable = /opt/homebrew/opt/python@3.11/bin/python3.11
```
So the Python 3.11 I installed through Homebrew doesn't exhibit this bug.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",810618495,Extract columns cannot create foreign key relation: sqlite3.OperationalError: table sqlite_master may not be modified,