{"html_url": "https://github.com/simonw/sqlite-utils/issues/98#issuecomment-928790381", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/98", "id": 928790381, "node_id": "IC_kwDOCGYnMM43XDdt", "user": {"value": 36834097, "label": "patricktrainer"}, "created_at": "2021-09-28T04:38:44Z", "updated_at": "2021-09-28T04:38:44Z", "author_association": "NONE", "body": "Hi @simonw - wondering if you might be able to shed some light here. I've seemed to reproduce this issue. \r\nHere's the stacktrace: \r\n\r\n```\r\n...     db[\"potholes\"].insert(pothole, pk='id', alter=True, replace=True)\r\n... \r\n\r\nTraceback (most recent call last):\r\n  File \"<stdin>\", line 3, in <module>\r\n  File \"/Users/patricktrainer/.pyenv/versions/3.9.0/lib/python3.9/site-packages/sqlite_utils/db.py\", line 2481, in insert\r\n    return self.insert_all(\r\n  File \"/Users/patricktrainer/.pyenv/versions/3.9.0/lib/python3.9/site-packages/sqlite_utils/db.py\", line 2596, in insert_all\r\n    self.insert_chunk(\r\n  File \"/Users/patricktrainer/.pyenv/versions/3.9.0/lib/python3.9/site-packages/sqlite_utils/db.py\", line 2424, in insert_chunk\r\n    row = list(self.rows_where(\"rowid = ?\", [self.last_rowid]))[0]\r\nIndexError: list index out of range\r\n```\r\n\r\nInteresting enough, I found that omitting the `pk` param does not throw the error.\r\n\r\nLet me know how I can help out! \r\n\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 597671518, "label": "Only set .last_rowid and .last_pk for single update/inserts, not for .insert_all()/.upsert_all() with multiple records"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/98#issuecomment-612728047", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/98", "id": 612728047, "node_id": "MDEyOklzc3VlQ29tbWVudDYxMjcyODA0Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-04-13T03:06:10Z", "updated_at": "2020-04-13T03:06:10Z", "author_association": "OWNER", "body": "Implementation plan: `.insert_all()` and `.upsert_all()` should only set `.last_rowid` and `last_pk` if they were called with a single item.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 597671518, "label": "Only set .last_rowid and .last_pk for single update/inserts, not for .insert_all()/.upsert_all() with multiple records"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/98#issuecomment-612708274", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/98", "id": 612708274, "node_id": "MDEyOklzc3VlQ29tbWVudDYxMjcwODI3NA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-04-13T01:25:59Z", "updated_at": "2020-04-13T01:26:11Z", "author_association": "OWNER", "body": "In mucking around with `sqlite3` it looks like `result.lastrowid` is indeed populated for `UPDATE` - in this case with the last inserted rowid in the table. This differs from the documented behaviour I linked to above.\r\n\r\n```\r\nIn [1]: import sqlite3                                                          \r\n\r\nIn [2]: c = sqlite3.connect(\":memory:\")                                         \r\n\r\nIn [3]: c                                                                       \r\nOut[3]: <sqlite3.Connection at 0x103c4d490>\r\n\r\nIn [4]: c.execute('create table foo (bar integer);')                            \r\nOut[4]: <sqlite3.Cursor at 0x103f760a0>\r\n\r\nIn [5]: c.execute('insert into foo (bar) values (1)')                           \r\nOut[5]: <sqlite3.Cursor at 0x103f76650>\r\n\r\nIn [6]: c.execute('select * from foo').fetchall()                               \r\nOut[6]: [(1,)]\r\n\r\nIn [7]: c.execute('insert into foo (bar) values (1)')                           \r\nOut[7]: <sqlite3.Cursor at 0x103f766c0>\r\n\r\nIn [8]: c.execute('select * from foo').fetchall()                               \r\nOut[8]: [(1,), (1,)]\r\n\r\nIn [9]: c.execute('insert into foo (bar) values (1)').lastrowid                 \r\nOut[9]: 3\r\n\r\nIn [10]: c.execute('select * from foo').fetchall()                              \r\nOut[10]: [(1,), (1,), (1,)]\r\n\r\nIn [11]: c.execute('select rowid, bar from foo').fetchall()                     \r\nOut[11]: [(1, 1), (2, 1), (3, 1)]\r\n\r\nIn [12]: c.execute('insert into foo (bar) values (1)').lastrowid                \r\nOut[12]: 4\r\n\r\nIn [13]: c.execute('select rowid, bar from foo').fetchall()                     \r\nOut[13]: [(1, 1), (2, 1), (3, 1), (4, 1)]\r\n\r\nIn [14]: r = c.execute('update foo set bar =2 where rowid = 1')                 \r\n\r\nIn [15]: r.lastrowid                                                            \r\nOut[15]: 4\r\n\r\nIn [16]: c.execute('select rowid, bar from foo').fetchall()                     \r\nOut[16]: [(1, 2), (2, 1), (3, 1), (4, 1)]\r\n\r\nIn [17]: r = c.execute('select rowid, bar from foo')                            \r\n\r\nIn [18]: r.fetchall()                                                           \r\nOut[18]: [(1, 2), (2, 1), (3, 1), (4, 1)]\r\n\r\nIn [19]: r.lastrowid                                                            \r\nOut[19]: 4\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 597671518, "label": "Only set .last_rowid and .last_pk for single update/inserts, not for .insert_all()/.upsert_all() with multiple records"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/98#issuecomment-612707828", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/98", "id": 612707828, "node_id": "MDEyOklzc3VlQ29tbWVudDYxMjcwNzgyOA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-04-13T01:24:05Z", "updated_at": "2020-04-13T01:24:16Z", "author_association": "OWNER", "body": "Why do I even care about `lastrowid` here?\r\n\r\nI'm trying to ensure that after you insert or upsert a row you can use `table.last_pk` to start doing things like building additional foreign key relationships.\r\n\r\nSo maybe it doesn't make sense to make `.last_pk` available _at all_ for cases where you called `.upsert_all()` or `.insert_all()` - it should just be populated for `.upsert()` and `.insert()`.\r\n\r\nThe documentation doesn't say it should work for `.upsert_all()` - it's only documented for the single actions.\r\n\r\nhttps://github.com/simonw/sqlite-utils/blob/6161ebf4de44411b3f33feeacaf4501e803d1116/sqlite_utils/db.py#L1113-L1124", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 597671518, "label": "Only set .last_rowid and .last_pk for single update/inserts, not for .insert_all()/.upsert_all() with multiple records"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/98#issuecomment-612707293", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/98", "id": 612707293, "node_id": "MDEyOklzc3VlQ29tbWVudDYxMjcwNzI5Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-04-13T01:21:22Z", "updated_at": "2020-04-13T01:21:22Z", "author_association": "OWNER", "body": "I have a hunch that the root of the problem here is that accessing `result.lastrowid` during my version of an `.upsert()` doesn't actually make sense:\r\n\r\nhttps://github.com/simonw/sqlite-utils/blob/6161ebf4de44411b3f33feeacaf4501e803d1116/sqlite_utils/db.py#L1102-L1113\r\n\r\nIn the bug I'm seeing (which I still haven't reduced to a reproducible test) the debugger shows me this at that point:\r\n\r\n```\r\n(Pdb) query\r\n'UPDATE [files] SET [createdAt] = ?, [ext] = ?, [updatedAt] = ?, [uri] = ?, [uriType] = ? WHERE [project] = ? AND [name] = ?'\r\n(Pdb) params\r\n['2020-03-04T04:04:40.152000+00:00', 'csv', '2020-03-04T04:04:40.152000+00:00', 'https://storage.googleapis.com/bln_prod/...', 'download', 'UHJvamVjdDo4MTgyMjU2Ny01ZjI0LTQxM2ItYWZmNi05NTlmNGY3MjExMjI=', 'loans_to_documentation.csv']\r\n(Pdb) result.lastrowid\r\n100\r\n```\r\nBut here's the weird thing... there's no row in the table with a rowid of 100!\r\n```\r\n(Pdb) [r['rowid'] for r in self.db.execute_returning_dicts('select rowid, * from files')]\r\n[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27,\r\n28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50,\r\n51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73,\r\n74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99]\r\n```\r\nSo what the heck is going on?\r\n\r\nThe last SQL statement I executed here was an `UPDATE`. The `lastrowid` docs say: https://kite.com/python/docs/sqlite3.Cursor.lastrowid\r\n\r\n> This read-only attribute provides the rowid of the last modified row. It is only set if you issued a INSERT statement using the execute() method. For operations other than INSERT or when executemany() is called, lastrowid is set to None.\r\n\r\nSo where did that `100` come from? It should be `None`!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 597671518, "label": "Only set .last_rowid and .last_pk for single update/inserts, not for .insert_all()/.upsert_all() with multiple records"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/98#issuecomment-612258687", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/98", "id": 612258687, "node_id": "MDEyOklzc3VlQ29tbWVudDYxMjI1ODY4Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-04-10T23:08:48Z", "updated_at": "2020-04-10T23:08:48Z", "author_association": "OWNER", "body": "I need a test that reproduces this.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 597671518, "label": "Only set .last_rowid and .last_pk for single update/inserts, not for .insert_all()/.upsert_all() with multiple records"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/98#issuecomment-612173156", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/98", "id": 612173156, "node_id": "MDEyOklzc3VlQ29tbWVudDYxMjE3MzE1Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-04-10T19:03:32Z", "updated_at": "2020-04-10T23:08:28Z", "author_association": "OWNER", "body": "Investigate this traceback:\r\n```\r\nTraceback (most recent call last):\r\n  File \"fetch_projects.py\", line 60, in <module>\r\n    fetch_projects(db, token)\r\n  File \"fetch_projects.py\", line 41, in fetch_projects\r\n    db[\"projects\"].upsert(project, pk=\"id\")\r\n  File \"/Users/simonw/.local/share/virtualenvs/big-local-datasette-2jT6nJCT/lib/python3.7/site-packages/sqlite_utils/db.py\", line 1139, in upsert\r\n    conversions=conversions,\r\n  File \"/Users/simonw/.local/share/virtualenvs/big-local-datasette-2jT6nJCT/lib/python3.7/site-packages/sqlite_utils/db.py\", line 1168, in upsert_all\r\n    upsert=True,\r\n  File \"/Users/simonw/.local/share/virtualenvs/big-local-datasette-2jT6nJCT/lib/python3.7/site-packages/sqlite_utils/db.py\", line 1107, in insert_all\r\n    row = list(self.rows_where(\"rowid = ?\", [self.last_rowid]))[0]\r\nIndexError: list index out of range\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 597671518, "label": "Only set .last_rowid and .last_pk for single update/inserts, not for .insert_all()/.upsert_all() with multiple records"}, "performed_via_github_app": null}