github
html_url | issue_url | id | node_id | user | created_at | updated_at | author_association | body | reactions | issue | performed_via_github_app |
---|---|---|---|---|---|---|---|---|---|---|---|
https://github.com/simonw/sqlite-utils/issues/538#issuecomment-1538921774 | https://api.github.com/repos/simonw/sqlite-utils/issues/538 | 1538921774 | IC_kwDOCGYnMM5buhUu | 9599 | 2023-05-08T19:24:41Z | 2023-05-08T19:24:41Z | OWNER | That fix seems to work! | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1695428235 | |
https://github.com/simonw/sqlite-utils/issues/538#issuecomment-1538910894 | https://api.github.com/repos/simonw/sqlite-utils/issues/538 | 1538910894 | IC_kwDOCGYnMM5buequ | 9599 | 2023-05-08T19:16:52Z | 2023-05-08T19:17:00Z | OWNER | How about if I had logic which checked that all not-null columns were provided in the call to `upsert_all()` - and if they were, modified the `INSERT OR IGNORE INTO` to include a placeholder value for those columns that would then be fixed by the later `UPDATE`? Something like this: ```python [ ('INSERT OR IGNORE INTO [comments]([id], name) VALUES(?, ?);', [1, '']), ('UPDATE [comments] SET [name] = ? WHERE [id] = ?', ['Cleo', 1]) ] ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1695428235 | |
https://github.com/simonw/sqlite-utils/issues/538#issuecomment-1538903556 | https://api.github.com/repos/simonw/sqlite-utils/issues/538 | 1538903556 | IC_kwDOCGYnMM5buc4E | 9599 | 2023-05-08T19:11:24Z | 2023-05-08T19:13:23Z | OWNER | I could detect if this happens using `cursor.rowcount` - not sure how I would recover from it though. This would also require some major re-engineering, since currently it all works by generating a list of SQL queries in advance and applying them inside a loop in `.insert_chunk()`: https://github.com/simonw/sqlite-utils/blob/80763edaa2bdaf1113717378b8d62075c4dcbcfb/sqlite_utils/db.py#L2839-L2878 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1695428235 | |
https://github.com/simonw/sqlite-utils/issues/538#issuecomment-1538893329 | https://api.github.com/repos/simonw/sqlite-utils/issues/538 | 1538893329 | IC_kwDOCGYnMM5buaYR | 9599 | 2023-05-08T19:04:47Z | 2023-05-08T19:04:47Z | OWNER | This feels like a fundamental flaw in the way upserts are implemented by `sqlite-utils`. One fix would be to switch to using the `UPSERT` feature in SQLite: https://www.sqlite.org/lang_UPSERT.html But... > UPSERT syntax was added to SQLite with version 3.24.0 (2018-06-04). I still want to support SQLite versions earlier than that. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1695428235 | |
https://github.com/simonw/sqlite-utils/issues/538#issuecomment-1538889482 | https://api.github.com/repos/simonw/sqlite-utils/issues/538 | 1538889482 | IC_kwDOCGYnMM5buZcK | 9599 | 2023-05-08T19:02:38Z | 2023-05-08T19:02:38Z | OWNER | Here's the code at fault: https://github.com/simonw/sqlite-utils/blob/80763edaa2bdaf1113717378b8d62075c4dcbcfb/sqlite_utils/db.py#L2774-L2788 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1695428235 | |
https://github.com/simonw/sqlite-utils/issues/538#issuecomment-1538887361 | https://api.github.com/repos/simonw/sqlite-utils/issues/538 | 1538887361 | IC_kwDOCGYnMM5buY7B | 9599 | 2023-05-08T19:01:20Z | 2023-05-08T19:01:20Z | OWNER | Here's the problem: ```python import sqlite3 db = sqlite3.connect(":memory:") db.execute('create table foo (id integer primary key, name not null)') db.execute('insert into foo (id) values (1)') ``` Produces: ``` IntegrityError: NOT NULL constraint failed: foo.name ``` But this: ```python db.execute('insert or ignore into foo (id) values (1)') ``` Completes without an exception. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1695428235 | |
https://github.com/simonw/sqlite-utils/issues/538#issuecomment-1538801855 | https://api.github.com/repos/simonw/sqlite-utils/issues/538 | 1538801855 | IC_kwDOCGYnMM5buEC_ | 9599 | 2023-05-08T18:00:17Z | 2023-05-08T18:00:17Z | OWNER | From time in the debugger, after creating the table it ends up doing this: ``` (Pdb) queries_and_params [ ('INSERT OR IGNORE INTO [comments]([id]) VALUES(?);', [1]), ('UPDATE [comments] SET [name] = ? WHERE [id] = ?', ['Cleo', 1]) ] ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1695428235 | |
https://github.com/simonw/sqlite-utils/issues/538#issuecomment-1538793817 | https://api.github.com/repos/simonw/sqlite-utils/issues/538 | 1538793817 | IC_kwDOCGYnMM5buCFZ | 9599 | 2023-05-08T17:55:10Z | 2023-05-08T17:55:10Z | OWNER | Confirmed - I added this test and it fails: ```python def test_upsert_all_not_null(fresh_db): # https://github.com/simonw/sqlite-utils/issues/538 fresh_db["comments"].upsert_all( [{"id": 1, "name": "Cleo"}], pk="id", not_null=["name"], ) assert list(fresh_db["comments"].rows) == [{"id": 1, "name": "Cleo"}] ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1695428235 |