github
html_url | issue_url | id | node_id | user | created_at | updated_at | author_association | body | reactions | issue | performed_via_github_app |
---|---|---|---|---|---|---|---|---|---|---|---|
https://github.com/simonw/sqlite-utils/issues/73#issuecomment-580745213 | https://api.github.com/repos/simonw/sqlite-utils/issues/73 | 580745213 | MDEyOklzc3VlQ29tbWVudDU4MDc0NTIxMw== | 82988 | 2020-01-31T14:02:38Z | 2020-01-31T14:21:09Z | NONE | So the conundrum continues.. The simple test case above now runs, but if I upsert a large number of new records (successfully) and then try to upsert a fewer number of new records to a different table, I get the same error. If I run the same upserts again (which in the first case means there are no new records to add, because they were already added), the second upsert works correctly. It feels as if the number of items added via an upsert >> the number of items I try to add in an upsert immediately after, I get the error. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
545407916 | |
https://github.com/simonw/sqlite-utils/issues/73#issuecomment-573047321 | https://api.github.com/repos/simonw/sqlite-utils/issues/73 | 573047321 | MDEyOklzc3VlQ29tbWVudDU3MzA0NzMyMQ== | 82988 | 2020-01-10T14:02:56Z | 2020-01-10T14:09:23Z | NONE | Hmmm... just tried with installs from pip and the repo (v2.0.0 and v2.0.1) and I get the error each time (start of second run through the second loop). Could it be sqlite3? I'm on 3.30.1. UPDATE: just tried it on jupyter.org/try and I get the error there, too. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
545407916 | |
https://github.com/simonw/sqlite-utils/issues/73#issuecomment-571138093 | https://api.github.com/repos/simonw/sqlite-utils/issues/73 | 571138093 | MDEyOklzc3VlQ29tbWVudDU3MTEzODA5Mw== | 82988 | 2020-01-06T13:28:31Z | 2020-01-06T13:28:31Z | NONE | I think I actually had several issues in play... The missing key was one, but I think there is also an issue as per below. For example, in the following: ```python def init_testdb(dbname='test.db'): if os.path.exists(dbname): os.remove(dbname) conn = sqlite3.connect(dbname) db = Database(conn) return conn, db conn, db = init_testdb() c = conn.cursor() c.executescript('CREATE TABLE "test1" ("Col1" TEXT, "Col2" TEXT, PRIMARY KEY ("Col1"));') c.executescript('CREATE TABLE "test2" ("Col1" TEXT, "Col2" TEXT, PRIMARY KEY ("Col1"));') print('Test 1...') for i in range(3): db['test1'].upsert_all([{'Col1':'a', 'Col2':'x'},{'Col1':'b', 'Col2':'x'}], pk=('Col1')) db['test2'].upsert_all([{'Col1':'a', 'Col2':'x'},{'Col1':'b', 'Col2':'x'}], pk=('Col1')) print('Test 2...') for i in range(3): db['test1'].upsert_all([{'Col1':'a', 'Col2':'x'},{'Col1':'b', 'Col2':'x'}], pk=('Col1')) db['test2'].upsert_all([{'Col1':'a', 'Col2':'x'},{'Col1':'b', 'Col2':'x'}, {'Col1':'c','Col2':'x'}], pk=('Col1')) print('Done...') --------------------------------------------------------------------------- Test 1... Test 2... IndexError: list index out of range --------------------------------------------------------------------------- IndexError Traceback (most recent call last) <ipython-input-763-444132ca189f> in <module> 22 print('Test 2...') 23 for i in range(3): ---> 24 db['test1'].upsert_all([{'Col1':'a', 'Col2':'x'},{'Col1':'b', 'Col2':'x'}], pk=('Col1')) 25 db['test2'].upsert_all([{'Col1':'a', 'Col2':'x'},{'Col1':'b', 'Col2':'x'}, 26 {'Col1':'c','Col2':'x'}], pk=('Col1')) /usr/local/lib/python3.7/site-packages/sqlite_utils/db.py in upsert_all(self, records, pk, foreign_keys, column_order, not_null, defaults, batch_size, hash_id, alter, extracts) 1157 alter=alter, 1158 extracts=extracts, -> 1… | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
545407916 | |
https://github.com/simonw/sqlite-utils/issues/8#issuecomment-482994231 | https://api.github.com/repos/simonw/sqlite-utils/issues/8 | 482994231 | MDEyOklzc3VlQ29tbWVudDQ4Mjk5NDIzMQ== | 82988 | 2019-04-14T15:04:07Z | 2019-04-14T15:29:33Z | NONE | PLEASE IGNORE THE BELOW... I did a package update and rebuilt the kernel I was working in... may just have been an old version of sqlite_utils, seems to be working now. (Too many containers / too many environments!) Has an issue been reintroduced here with FTS? eg I'm getting an error thrown by spaces in column names here: ``` /usr/local/lib/python3.7/site-packages/sqlite_utils/db.py in insert_all(self, records, pk, foreign_keys, upsert, batch_size, column_order) def enable_fts(self, columns, fts_version="FTS5"): --> 329 "Enables FTS on the specified columns" 330 sql = """ 331 CREATE VIRTUAL TABLE "{table}_fts" USING {fts_version} ( ``` when trying an `insert_all`. Also, if a col has a `.` in it, I seem to get: ``` /usr/local/lib/python3.7/site-packages/sqlite_utils/db.py in insert_all(self, records, pk, foreign_keys, upsert, batch_size, column_order) 327 jsonify_if_needed(record.get(key, None)) for key in all_columns 328 ) --> 329 result = self.db.conn.execute(sql, values) 330 self.db.conn.commit() 331 self.last_id = result.lastrowid OperationalError: near ".": syntax error ``` (Can't post a worked minimal example right now; racing trying to build something against a live timing screen that will stop until next weekend in an hour or two...) PS Hmmm I did a test and they seem to work; I must be messing up s/where else... ``` import sqlite3 from sqlite_utils import Database dbname='testingDB_sqlite_utils.db' #!rm $dbname conn = sqlite3.connect(dbname, timeout=10) #Setup database tables c = conn.cursor() setup=''' CREATE TABLE IF NOT EXISTS "test1" ( "NO" INTEGER, "NAME" TEXT ); CREATE TABLE IF NOT EXISTS "test2" ( "NO" INTEGER, `TIME OF DAY` TEXT ); CREATE TABLE IF NOT EXISTS "test3" ( "NO" INTEGER, `AVG. SPEED (MPH)` FLOAT ); ''' c.executescript(setup) DB = Database(conn) … | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
403922644 | |
https://github.com/simonw/sqlite-utils/issues/18#issuecomment-480621924 | https://api.github.com/repos/simonw/sqlite-utils/issues/18 | 480621924 | MDEyOklzc3VlQ29tbWVudDQ4MDYyMTkyNA== | 82988 | 2019-04-07T19:31:42Z | 2019-04-07T19:31:42Z | NONE | I've just noticed that SQLite lets you IGNORE inserts that collide with a pre-existing key. This can be quite handy if you have a dataset that keeps changing in part, and you don't want to upsert and replace pre-existing PK rows but you do want to ignore collisions to existing PK rows. Do `sqlite_utils` support such (cavalier!) behaviour? | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
413871266 | |
https://github.com/simonw/sqlite-utils/issues/8#issuecomment-464341721 | https://api.github.com/repos/simonw/sqlite-utils/issues/8 | 464341721 | MDEyOklzc3VlQ29tbWVudDQ2NDM0MTcyMQ== | 82988 | 2019-02-16T12:08:41Z | 2019-02-16T12:08:41Z | NONE | We also get an error if a column name contains a `.` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
403922644 |