html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,issue,performed_via_github_app https://github.com/simonw/sqlite-utils/issues/538#issuecomment-1538975545,https://api.github.com/repos/simonw/sqlite-utils/issues/538,1538975545,IC_kwDOCGYnMM5buuc5,1231935,2023-05-08T20:06:35Z,2023-05-08T20:06:35Z,NONE,"perfect, thank you!","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1695428235, https://github.com/simonw/sqlite-utils/issues/538#issuecomment-1538921774,https://api.github.com/repos/simonw/sqlite-utils/issues/538,1538921774,IC_kwDOCGYnMM5buhUu,9599,2023-05-08T19:24:41Z,2023-05-08T19:24:41Z,OWNER,That fix seems to work!,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1695428235, https://github.com/simonw/sqlite-utils/issues/538#issuecomment-1538910894,https://api.github.com/repos/simonw/sqlite-utils/issues/538,1538910894,IC_kwDOCGYnMM5buequ,9599,2023-05-08T19:16:52Z,2023-05-08T19:17:00Z,OWNER,"How about if I had logic which checked that all not-null columns were provided in the call to `upsert_all()` - and if they were, modified the `INSERT OR IGNORE INTO` to include a placeholder value for those columns that would then be fixed by the later `UPDATE`? Something like this: ```python [ ('INSERT OR IGNORE INTO [comments]([id], name) VALUES(?, ?);', [1, '']), ('UPDATE [comments] SET [name] = ? WHERE [id] = ?', ['Cleo', 1]) ] ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1695428235, https://github.com/simonw/sqlite-utils/issues/538#issuecomment-1538903556,https://api.github.com/repos/simonw/sqlite-utils/issues/538,1538903556,IC_kwDOCGYnMM5buc4E,9599,2023-05-08T19:11:24Z,2023-05-08T19:13:23Z,OWNER,"I could detect if this happens using `cursor.rowcount` - not sure how I would recover from it though. This would also require some major re-engineering, since currently it all works by generating a list of SQL queries in advance and applying them inside a loop in `.insert_chunk()`: https://github.com/simonw/sqlite-utils/blob/80763edaa2bdaf1113717378b8d62075c4dcbcfb/sqlite_utils/db.py#L2839-L2878 ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1695428235, https://github.com/simonw/sqlite-utils/issues/538#issuecomment-1538893329,https://api.github.com/repos/simonw/sqlite-utils/issues/538,1538893329,IC_kwDOCGYnMM5buaYR,9599,2023-05-08T19:04:47Z,2023-05-08T19:04:47Z,OWNER,"This feels like a fundamental flaw in the way upserts are implemented by `sqlite-utils`. One fix would be to switch to using the `UPSERT` feature in SQLite: https://www.sqlite.org/lang_UPSERT.html But... > UPSERT syntax was added to SQLite with version 3.24.0 (2018-06-04). I still want to support SQLite versions earlier than that.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1695428235, https://github.com/simonw/sqlite-utils/issues/538#issuecomment-1538889482,https://api.github.com/repos/simonw/sqlite-utils/issues/538,1538889482,IC_kwDOCGYnMM5buZcK,9599,2023-05-08T19:02:38Z,2023-05-08T19:02:38Z,OWNER,"Here's the code at fault: https://github.com/simonw/sqlite-utils/blob/80763edaa2bdaf1113717378b8d62075c4dcbcfb/sqlite_utils/db.py#L2774-L2788","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1695428235, https://github.com/simonw/sqlite-utils/issues/538#issuecomment-1538887361,https://api.github.com/repos/simonw/sqlite-utils/issues/538,1538887361,IC_kwDOCGYnMM5buY7B,9599,2023-05-08T19:01:20Z,2023-05-08T19:01:20Z,OWNER,"Here's the problem: ```python import sqlite3 db = sqlite3.connect("":memory:"") db.execute('create table foo (id integer primary key, name not null)') db.execute('insert into foo (id) values (1)') ``` Produces: ``` IntegrityError: NOT NULL constraint failed: foo.name ``` But this: ```python db.execute('insert or ignore into foo (id) values (1)') ``` Completes without an exception.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1695428235, https://github.com/simonw/sqlite-utils/issues/538#issuecomment-1538801855,https://api.github.com/repos/simonw/sqlite-utils/issues/538,1538801855,IC_kwDOCGYnMM5buEC_,9599,2023-05-08T18:00:17Z,2023-05-08T18:00:17Z,OWNER,"From time in the debugger, after creating the table it ends up doing this: ``` (Pdb) queries_and_params [ ('INSERT OR IGNORE INTO [comments]([id]) VALUES(?);', [1]), ('UPDATE [comments] SET [name] = ? WHERE [id] = ?', ['Cleo', 1]) ] ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1695428235, https://github.com/simonw/sqlite-utils/issues/538#issuecomment-1538793817,https://api.github.com/repos/simonw/sqlite-utils/issues/538,1538793817,IC_kwDOCGYnMM5buCFZ,9599,2023-05-08T17:55:10Z,2023-05-08T17:55:10Z,OWNER,"Confirmed - I added this test and it fails: ```python def test_upsert_all_not_null(fresh_db): # https://github.com/simonw/sqlite-utils/issues/538 fresh_db[""comments""].upsert_all( [{""id"": 1, ""name"": ""Cleo""}], pk=""id"", not_null=[""name""], ) assert list(fresh_db[""comments""].rows) == [{""id"": 1, ""name"": ""Cleo""}] ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1695428235,