{"html_url": "https://github.com/simonw/sqlite-utils/issues/99#issuecomment-612727400", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/99", "id": 612727400, "node_id": "MDEyOklzc3VlQ29tbWVudDYxMjcyNzQwMA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-04-13T03:03:09Z", "updated_at": "2020-04-13T03:03:09Z", "author_association": "OWNER", "body": "I think I'm going to leave this as intended behaviour. Or maybe passing multiple dictionaries to `.upsert_all()` with different numbers of keys should raise an error?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 598640234, "label": ".upsert_all() should maybe error if dictionaries passed to it do not have the same keys"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/99#issuecomment-612727814", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/99", "id": 612727814, "node_id": "MDEyOklzc3VlQ29tbWVudDYxMjcyNzgxNA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-04-13T03:05:04Z", "updated_at": "2020-04-13T03:05:04Z", "author_association": "OWNER", "body": "Bit trick from an implementation point of view this, since we want to be able to handle input that is a generator - so we can't scan through the input to validate that every dictionary has the same exact keys without consuming the entire iterator.\r\n\r\nThe alternative would be to raise an error the first time we spot a dictionary with keys that differ... but that's weird because we commit changes in batches, so we may end up only applying half of the changes before exiting with the error.\r\n\r\nOn that basis, I'm going to leave this as-is and mark this as wontfix.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 598640234, "label": ".upsert_all() should maybe error if dictionaries passed to it do not have the same keys"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/98#issuecomment-612173156", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/98", "id": 612173156, "node_id": "MDEyOklzc3VlQ29tbWVudDYxMjE3MzE1Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-04-10T19:03:32Z", "updated_at": "2020-04-10T23:08:28Z", "author_association": "OWNER", "body": "Investigate this traceback:\r\n```\r\nTraceback (most recent call last):\r\n File \"fetch_projects.py\", line 60, in \r\n fetch_projects(db, token)\r\n File \"fetch_projects.py\", line 41, in fetch_projects\r\n db[\"projects\"].upsert(project, pk=\"id\")\r\n File \"/Users/simonw/.local/share/virtualenvs/big-local-datasette-2jT6nJCT/lib/python3.7/site-packages/sqlite_utils/db.py\", line 1139, in upsert\r\n conversions=conversions,\r\n File \"/Users/simonw/.local/share/virtualenvs/big-local-datasette-2jT6nJCT/lib/python3.7/site-packages/sqlite_utils/db.py\", line 1168, in upsert_all\r\n upsert=True,\r\n File \"/Users/simonw/.local/share/virtualenvs/big-local-datasette-2jT6nJCT/lib/python3.7/site-packages/sqlite_utils/db.py\", line 1107, in insert_all\r\n row = list(self.rows_where(\"rowid = ?\", [self.last_rowid]))[0]\r\nIndexError: list index out of range\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 597671518, "label": "Only set .last_rowid and .last_pk for single update/inserts, not for .insert_all()/.upsert_all() with multiple records"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/98#issuecomment-612258687", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/98", "id": 612258687, "node_id": "MDEyOklzc3VlQ29tbWVudDYxMjI1ODY4Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-04-10T23:08:48Z", "updated_at": "2020-04-10T23:08:48Z", "author_association": "OWNER", "body": "I need a test that reproduces this.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 597671518, "label": "Only set .last_rowid and .last_pk for single update/inserts, not for .insert_all()/.upsert_all() with multiple records"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/98#issuecomment-612707293", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/98", "id": 612707293, "node_id": "MDEyOklzc3VlQ29tbWVudDYxMjcwNzI5Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-04-13T01:21:22Z", "updated_at": "2020-04-13T01:21:22Z", "author_association": "OWNER", "body": "I have a hunch that the root of the problem here is that accessing `result.lastrowid` during my version of an `.upsert()` doesn't actually make sense:\r\n\r\nhttps://github.com/simonw/sqlite-utils/blob/6161ebf4de44411b3f33feeacaf4501e803d1116/sqlite_utils/db.py#L1102-L1113\r\n\r\nIn the bug I'm seeing (which I still haven't reduced to a reproducible test) the debugger shows me this at that point:\r\n\r\n```\r\n(Pdb) query\r\n'UPDATE [files] SET [createdAt] = ?, [ext] = ?, [updatedAt] = ?, [uri] = ?, [uriType] = ? WHERE [project] = ? AND [name] = ?'\r\n(Pdb) params\r\n['2020-03-04T04:04:40.152000+00:00', 'csv', '2020-03-04T04:04:40.152000+00:00', 'https://storage.googleapis.com/bln_prod/...', 'download', 'UHJvamVjdDo4MTgyMjU2Ny01ZjI0LTQxM2ItYWZmNi05NTlmNGY3MjExMjI=', 'loans_to_documentation.csv']\r\n(Pdb) result.lastrowid\r\n100\r\n```\r\nBut here's the weird thing... there's no row in the table with a rowid of 100!\r\n```\r\n(Pdb) [r['rowid'] for r in self.db.execute_returning_dicts('select rowid, * from files')]\r\n[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27,\r\n28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50,\r\n51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73,\r\n74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99]\r\n```\r\nSo what the heck is going on?\r\n\r\nThe last SQL statement I executed here was an `UPDATE`. The `lastrowid` docs say: https://kite.com/python/docs/sqlite3.Cursor.lastrowid\r\n\r\n> This read-only attribute provides the rowid of the last modified row. It is only set if you issued a INSERT statement using the execute() method. For operations other than INSERT or when executemany() is called, lastrowid is set to None.\r\n\r\nSo where did that `100` come from? It should be `None`!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 597671518, "label": "Only set .last_rowid and .last_pk for single update/inserts, not for .insert_all()/.upsert_all() with multiple records"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/98#issuecomment-612707828", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/98", "id": 612707828, "node_id": "MDEyOklzc3VlQ29tbWVudDYxMjcwNzgyOA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-04-13T01:24:05Z", "updated_at": "2020-04-13T01:24:16Z", "author_association": "OWNER", "body": "Why do I even care about `lastrowid` here?\r\n\r\nI'm trying to ensure that after you insert or upsert a row you can use `table.last_pk` to start doing things like building additional foreign key relationships.\r\n\r\nSo maybe it doesn't make sense to make `.last_pk` available _at all_ for cases where you called `.upsert_all()` or `.insert_all()` - it should just be populated for `.upsert()` and `.insert()`.\r\n\r\nThe documentation doesn't say it should work for `.upsert_all()` - it's only documented for the single actions.\r\n\r\nhttps://github.com/simonw/sqlite-utils/blob/6161ebf4de44411b3f33feeacaf4501e803d1116/sqlite_utils/db.py#L1113-L1124", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 597671518, "label": "Only set .last_rowid and .last_pk for single update/inserts, not for .insert_all()/.upsert_all() with multiple records"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/98#issuecomment-612708274", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/98", "id": 612708274, "node_id": "MDEyOklzc3VlQ29tbWVudDYxMjcwODI3NA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-04-13T01:25:59Z", "updated_at": "2020-04-13T01:26:11Z", "author_association": "OWNER", "body": "In mucking around with `sqlite3` it looks like `result.lastrowid` is indeed populated for `UPDATE` - in this case with the last inserted rowid in the table. This differs from the documented behaviour I linked to above.\r\n\r\n```\r\nIn [1]: import sqlite3 \r\n\r\nIn [2]: c = sqlite3.connect(\":memory:\") \r\n\r\nIn [3]: c \r\nOut[3]: \r\n\r\nIn [4]: c.execute('create table foo (bar integer);') \r\nOut[4]: \r\n\r\nIn [5]: c.execute('insert into foo (bar) values (1)') \r\nOut[5]: \r\n\r\nIn [6]: c.execute('select * from foo').fetchall() \r\nOut[6]: [(1,)]\r\n\r\nIn [7]: c.execute('insert into foo (bar) values (1)') \r\nOut[7]: \r\n\r\nIn [8]: c.execute('select * from foo').fetchall() \r\nOut[8]: [(1,), (1,)]\r\n\r\nIn [9]: c.execute('insert into foo (bar) values (1)').lastrowid \r\nOut[9]: 3\r\n\r\nIn [10]: c.execute('select * from foo').fetchall() \r\nOut[10]: [(1,), (1,), (1,)]\r\n\r\nIn [11]: c.execute('select rowid, bar from foo').fetchall() \r\nOut[11]: [(1, 1), (2, 1), (3, 1)]\r\n\r\nIn [12]: c.execute('insert into foo (bar) values (1)').lastrowid \r\nOut[12]: 4\r\n\r\nIn [13]: c.execute('select rowid, bar from foo').fetchall() \r\nOut[13]: [(1, 1), (2, 1), (3, 1), (4, 1)]\r\n\r\nIn [14]: r = c.execute('update foo set bar =2 where rowid = 1') \r\n\r\nIn [15]: r.lastrowid \r\nOut[15]: 4\r\n\r\nIn [16]: c.execute('select rowid, bar from foo').fetchall() \r\nOut[16]: [(1, 2), (2, 1), (3, 1), (4, 1)]\r\n\r\nIn [17]: r = c.execute('select rowid, bar from foo') \r\n\r\nIn [18]: r.fetchall() \r\nOut[18]: [(1, 2), (2, 1), (3, 1), (4, 1)]\r\n\r\nIn [19]: r.lastrowid \r\nOut[19]: 4\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 597671518, "label": "Only set .last_rowid and .last_pk for single update/inserts, not for .insert_all()/.upsert_all() with multiple records"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/98#issuecomment-612728047", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/98", "id": 612728047, "node_id": "MDEyOklzc3VlQ29tbWVudDYxMjcyODA0Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-04-13T03:06:10Z", "updated_at": "2020-04-13T03:06:10Z", "author_association": "OWNER", "body": "Implementation plan: `.insert_all()` and `.upsert_all()` should only set `.last_rowid` and `last_pk` if they were called with a single item.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 597671518, "label": "Only set .last_rowid and .last_pk for single update/inserts, not for .insert_all()/.upsert_all() with multiple records"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/97#issuecomment-612732129", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/97", "id": 612732129, "node_id": "MDEyOklzc3VlQ29tbWVudDYxMjczMjEyOQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-04-13T03:25:29Z", "updated_at": "2020-04-13T03:25:29Z", "author_association": "OWNER", "body": "Interesting thought. I've run into this myself a lot - many of my scripts intend to create the database from scratch, so I end up running `!rm /tmp/blah.db` in Jupyter and occasionally getting errors if the file doesn't exist.\r\n\r\nI think adding `recreate=True` could make sense. It could throw an error if you attempt to use it after passing in something other than a path to a file on disk.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 593751293, "label": "Adding a \"recreate\" flag to the `Database` constructor"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/97#issuecomment-612732453", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/97", "id": 612732453, "node_id": "MDEyOklzc3VlQ29tbWVudDYxMjczMjQ1Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-04-13T03:26:46Z", "updated_at": "2020-04-13T03:26:46Z", "author_association": "OWNER", "body": "I wonder if it should delete an recreate the file or if it would be safer to drop every table instead?\r\n\r\nDropping tables gets messy: then you need to drop triggers and views, and you need to run `vacuum` to clean up the space.\r\n\r\nMy worry with deleting and recreating the file is that it could trigger errors in other processes that are currently attached to that database file. But... if you know that's going to be likely, maybe you shouldn't use the `recreate=True` feature?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 593751293, "label": "Adding a \"recreate\" flag to the `Database` constructor"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/97#issuecomment-612738311", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/97", "id": 612738311, "node_id": "MDEyOklzc3VlQ29tbWVudDYxMjczODMxMQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-04-13T03:55:11Z", "updated_at": "2020-04-13T03:55:11Z", "author_association": "OWNER", "body": "Shipped in 2.5 - documentation is here: https://sqlite-utils.readthedocs.io/en/stable/python-api.html#connecting-to-or-creating-a-database", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 593751293, "label": "Adding a \"recreate\" flag to the `Database` constructor"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/96#issuecomment-606394349", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/96", "id": 606394349, "node_id": "MDEyOklzc3VlQ29tbWVudDYwNjM5NDM0OQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-03-31T04:37:16Z", "updated_at": "2020-03-31T04:37:16Z", "author_association": "OWNER", "body": "Test failure was just a Black formatting issue.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 589801352, "label": "Add type conversion for Panda's Timestamp"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/96#issuecomment-606394619", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/96", "id": 606394619, "node_id": "MDEyOklzc3VlQ29tbWVudDYwNjM5NDYxOQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-03-31T04:38:17Z", "updated_at": "2020-03-31T04:40:23Z", "author_association": "OWNER", "body": "I wonder if there are any other Pandas conversions we should be doing? https://pandas.pydata.org/pandas-docs/stable/getting_started/basics.html#dtypes ", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 589801352, "label": "Add type conversion for Panda's Timestamp"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/92#issuecomment-599125455", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/92", "id": 599125455, "node_id": "MDEyOklzc3VlQ29tbWVudDU5OTEyNTQ1NQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-03-14T19:34:35Z", "updated_at": "2020-03-14T19:34:35Z", "author_association": "OWNER", "body": "From https://www.sqlite.org/datatype3.html it looks like `FLOAT` is a supported keyword for creating tables but `REAL` is the correct keyword.\r\n\r\nSo actually `sqlite-utils` gets this wrong, because when we create a table we turn Python `float` values into a `FLOAT` column. Looks like the correct behaviour would be to turn them into a `REAL` column.\r\n\r\nhttps://github.com/simonw/sqlite-utils/blob/43f1c6ab4e3a6b76531fb6f5447adb83d26f3971/sqlite_utils/db.py#L28-L48", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 581339961, "label": ".columns_dict doesn't work for all possible column types"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/92#issuecomment-599125557", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/92", "id": 599125557, "node_id": "MDEyOklzc3VlQ29tbWVudDU5OTEyNTU1Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-03-14T19:35:29Z", "updated_at": "2020-03-14T19:35:29Z", "author_association": "OWNER", "body": "Fixing that would technically constitute a breaking change for library consumers, so it should be a major version release.\r\n\r\nI'm not inclined to release `3.0` just for this one issue, so I'm going to hold back on fixing that and address the smaller issue in this bug as a dot release instead for the moment.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 581339961, "label": ".columns_dict doesn't work for all possible column types"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/92#issuecomment-599126831", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/92", "id": 599126831, "node_id": "MDEyOklzc3VlQ29tbWVudDU5OTEyNjgzMQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-03-14T19:45:28Z", "updated_at": "2020-03-14T19:45:28Z", "author_association": "OWNER", "body": "Turns out there are a TON of valid column definitions that aren't being considered yet - https://www.sqlite.org/datatype3.html#affinity_name_examples - stuff like `VARYING CHARACTER(255)` and `DECIMAL(10,5)`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 581339961, "label": ".columns_dict doesn't work for all possible column types"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/92#issuecomment-599127197", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/92", "id": 599127197, "node_id": "MDEyOklzc3VlQ29tbWVudDU5OTEyNzE5Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-03-14T19:48:06Z", "updated_at": "2020-03-14T19:48:06Z", "author_association": "OWNER", "body": "Actually it looks like I should implement the exact rules described in https://www.sqlite.org/datatype3.html#determination_of_column_affinity\r\n\r\n> The affinity of a column is determined by the declared type of the column, according to the following rules in the order shown:\r\n> \r\n> 1. If the declared type contains the string \"INT\" then it is assigned INTEGER affinity.\r\n> 2. If the declared type of the column contains any of the strings \"CHAR\", \"CLOB\", or \"TEXT\" then that column has TEXT affinity. Notice that the type VARCHAR contains the string \"CHAR\" and is thus assigned TEXT affinity.\r\n> 3. If the declared type for a column contains the string \"BLOB\" or if no type is specified then the column has affinity BLOB.\r\n> 4. If the declared type for a column contains any of the strings \"REAL\", \"FLOA\", or \"DOUB\" then the column has REAL affinity.\r\n> 5. Otherwise, the affinity is NUMERIC.\r\n> \r\n> Note that the order of the rules for determining column affinity is important. A column whose declared type is \"CHARINT\" will match both rules 1 and 2 but the first rule takes precedence and so the column affinity will be INTEGER.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 581339961, "label": ".columns_dict doesn't work for all possible column types"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/92#issuecomment-599127453", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/92", "id": 599127453, "node_id": "MDEyOklzc3VlQ29tbWVudDU5OTEyNzQ1Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-03-14T19:50:08Z", "updated_at": "2020-03-14T19:50:08Z", "author_association": "OWNER", "body": "> If the declared type for a column contains the string \"BLOB\" or if no type is specified then the column has affinity BLOB\r\n\r\nI currently treat those as `str` - it sounds like I should treat them as `bytes`:\r\n\r\nhttps://github.com/simonw/sqlite-utils/blob/43f1c6ab4e3a6b76531fb6f5447adb83d26f3971/sqlite_utils/db.py#L68-L69\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 581339961, "label": ".columns_dict doesn't work for all possible column types"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/92#issuecomment-599128891", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/92", "id": 599128891, "node_id": "MDEyOklzc3VlQ29tbWVudDU5OTEyODg5MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-03-14T20:03:45Z", "updated_at": "2020-03-14T20:03:45Z", "author_association": "OWNER", "body": "I'm going to keep treating them as `str`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 581339961, "label": ".columns_dict doesn't work for all possible column types"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/92#issuecomment-599247833", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/92", "id": 599247833, "node_id": "MDEyOklzc3VlQ29tbWVudDU5OTI0NzgzMw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-03-15T18:37:28Z", "updated_at": "2020-03-15T18:37:43Z", "author_association": "OWNER", "body": "Released in 2.4.2.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 581339961, "label": ".columns_dict doesn't work for all possible column types"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/91#issuecomment-723350956", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/91", "id": 723350956, "node_id": "MDEyOklzc3VlQ29tbWVudDcyMzM1MDk1Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-11-06T23:53:25Z", "updated_at": "2020-11-06T23:53:25Z", "author_association": "OWNER", "body": "This is now possible, for both FTS4 and FTS5 - see #197.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 577302229, "label": "Enable ordering FTS results by rank"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/89#issuecomment-615515867", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/89", "id": 615515867, "node_id": "MDEyOklzc3VlQ29tbWVudDYxNTUxNTg2Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-04-18T00:00:41Z", "updated_at": "2020-04-18T00:00:41Z", "author_association": "OWNER", "body": "Yes pleas, I'd love to see that pull request!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 573578548, "label": "Ability to customize columns used by extracts= feature"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/89#issuecomment-710460242", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/89", "id": 710460242, "node_id": "MDEyOklzc3VlQ29tbWVudDcxMDQ2MDI0Mg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-16T19:17:27Z", "updated_at": "2020-10-16T19:17:50Z", "author_association": "OWNER", "body": "I came up with potential syntax for that here: https://github.com/simonw/sqlite-utils/issues/49#issuecomment-710393550 - based on how `table.extract(...)` works:\r\n```python\r\nfresh_db.table(\"tree\", extracts=[Extract(\r\n columns=(\"CommonName\", \"LatinName\"),\r\n table=\"Species\",\r\n fk_column=\"species_id\",\r\n rename={\"CommonName\": \"name\", \"LatinName\": \"latin\"}\r\n)])\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 573578548, "label": "Ability to customize columns used by extracts= feature"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/88#issuecomment-591768604", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/88", "id": 591768604, "node_id": "MDEyOklzc3VlQ29tbWVudDU5MTc2ODYwNA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-02-27T04:03:03Z", "updated_at": "2020-02-27T04:03:03Z", "author_association": "OWNER", "body": "`drop table resources_fts` drops the FTS table and the other ones that it created (resources_fts_data, resources_fts_idx, resources_fts_docsize, resources_fts_config) - but keeps the triggers.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 571805300, "label": "table.disable_fts() method and \"sqlite-utils disable-fts ...\" command"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/88#issuecomment-591769046", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/88", "id": 591769046, "node_id": "MDEyOklzc3VlQ29tbWVudDU5MTc2OTA0Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-02-27T04:05:15Z", "updated_at": "2020-02-27T04:05:15Z", "author_association": "OWNER", "body": "I can reliably get the list of triggers to delete from `select name from sqlite_master where type = 'trigger' and tbl_name = 'resources';`", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 571805300, "label": "table.disable_fts() method and \"sqlite-utils disable-fts ...\" command"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/88#issuecomment-591769171", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/88", "id": 591769171, "node_id": "MDEyOklzc3VlQ29tbWVudDU5MTc2OTE3MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-02-27T04:05:58Z", "updated_at": "2020-02-27T04:26:31Z", "author_association": "OWNER", "body": "Strange - https://www.sqlite.org/lang_droptrigger.html says \"Note that triggers are automatically dropped when the associated table is dropped\" but that doesn't seem to be true in my experimenting.\r\n\r\nUPDATE: no that makes sense - the triggers are on `resources` which still exists, it was `resources_fts` that was dropped.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 571805300, "label": "table.disable_fts() method and \"sqlite-utils disable-fts ...\" command"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/88#issuecomment-591769373", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/88", "id": 591769373, "node_id": "MDEyOklzc3VlQ29tbWVudDU5MTc2OTM3Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-02-27T04:06:47Z", "updated_at": "2020-02-27T04:06:47Z", "author_association": "OWNER", "body": "Looks like safest option is to loop through those trigger names and run `DROP TRIGGER IF EXISTS foo` on each one.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 571805300, "label": "table.disable_fts() method and \"sqlite-utils disable-fts ...\" command"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/88#issuecomment-591769759", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/88", "id": 591769759, "node_id": "MDEyOklzc3VlQ29tbWVudDU5MTc2OTc1OQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-02-27T04:08:29Z", "updated_at": "2020-02-27T04:08:29Z", "author_association": "OWNER", "body": "I think the method should be called `table.disable_fts()` - the opposite of `table.enable_fts(...)`.\r\n\r\nThere should be a `sqlite-utils disable-fts database.db tablename` command to match it.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 571805300, "label": "table.disable_fts() method and \"sqlite-utils disable-fts ...\" command"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/87#issuecomment-586661250", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/87", "id": 586661250, "node_id": "MDEyOklzc3VlQ29tbWVudDU4NjY2MTI1MA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-02-16T02:19:33Z", "updated_at": "2020-02-16T02:19:33Z", "author_association": "OWNER", "body": "Here's the code: \r\n\r\nhttps://github.com/simonw/sqlite-utils/blob/5e0000609f9be6efafea1b96f610988eb18d6d89/sqlite_utils/utils.py#L18-L24", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 565837965, "label": "Should detect collections.OrderedDict as a regular dictionary"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/87#issuecomment-586661276", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/87", "id": 586661276, "node_id": "MDEyOklzc3VlQ29tbWVudDU4NjY2MTI3Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-02-16T02:20:14Z", "updated_at": "2020-02-16T02:20:14Z", "author_association": "OWNER", "body": "`OrderedDict` is actually a subclass of `dict` - so a smart fix would be for this logic to check and see if the type `t` is a subclass of one of `list`, `tuple` or `dict`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 565837965, "label": "Should detect collections.OrderedDict as a regular dictionary"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/86#issuecomment-586661934", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/86", "id": 586661934, "node_id": "MDEyOklzc3VlQ29tbWVudDU4NjY2MTkzNA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-02-16T02:33:07Z", "updated_at": "2020-02-16T02:33:07Z", "author_association": "OWNER", "body": "Thanks for the example file - looks like it can be trimmed down to just these two lines to replicate the bug:\r\n```csv\r\n\"MTU (CET)\",\"Day-ahead Price [EUR/MWh]\"\r\n\"01.01.2016 00:00 - 01.01.2016 01:00\",\"23.86\"\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 564579430, "label": "Problem with square bracket in CSV column name"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/86#issuecomment-586662404", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/86", "id": 586662404, "node_id": "MDEyOklzc3VlQ29tbWVudDU4NjY2MjQwNA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-02-16T02:43:12Z", "updated_at": "2020-02-16T02:43:12Z", "author_association": "OWNER", "body": "https://stackoverflow.com/a/22694438 looks like the answer:\r\n> When using square brackets, it is not possible to have these characters in the identifier.\r\n> \r\n> When using double quotes, you can escape them in the name by doubling them:\r\n> \r\n> `CREATE TABLE \"hello \"\"world\"\"\"(key INTEGER PRIMARY KEY);`", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 564579430, "label": "Problem with square bracket in CSV column name"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/86#issuecomment-586676640", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/86", "id": 586676640, "node_id": "MDEyOklzc3VlQ29tbWVudDU4NjY3NjY0MA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-02-16T07:16:31Z", "updated_at": "2020-02-16T07:16:31Z", "author_association": "OWNER", "body": "There's something weird about this. I created a test database file like so:\r\n```\r\nsqlite3 /tmp/demo.db < .schema\r\nCREATE TABLE IF NOT EXISTS \"data\" (\r\n \"MTU (CET)\" TEXT,\r\n \"Day-ahead Price [EUR/MWh]\" TEXT\r\n );\r\nsqlite> .headers on\r\nsqlite> select * from data;\r\nMTU (CET)|Day-ahead Price [EUR/MWh]\r\n01.01.2016 00:00 - 01.01.2016 01:00|23.86\r\nsqlite> \r\n```\r\nBUT... if I open the same database in Python, something weird happens:\r\n```\r\nIn [1]: import sqlite3 \r\nIn [2]: conn = sqlite3.connect(\"/tmp/demo.db\") \r\nIn [3]: cursor = conn.cursor() \r\nIn [4]: cursor.execute(\"select * from data\") \r\nOut[4]: \r\nIn [5]: cursor.fetchall() \r\nOut[5]: [('01.01.2016 00:00 - 01.01.2016 01:00', '23.86')]\r\nIn [6]: cursor.description \r\nOut[6]: \r\n(('MTU (CET)', None, None, None, None, None, None),\r\n ('Day-ahead Price', None, None, None, None, None, None))\r\nIn [7]: conn.row_factory = sqlite3.Row \r\nIn [8]: cursor = conn.cursor() \r\nIn [9]: cursor.execute(\"select * from data\") \r\nOut[9]: \r\nIn [10]: row = cursor.fetchall() \r\nIn [12]: row \r\nOut[12]: \r\nIn [15]: row.keys() \r\nOut[15]: ['MTU (CET)', 'Day-ahead Price']\r\n```\r\nNote that in `cursor.description` AND in `row.keys()` above the second column is displayed as `'Day-ahead Price'` - when we would expect it to be displayed as `Day-ahead Price [EUR/MWh]`\r\n\r\nSo.... it looks like there may be a bug in Python's `sqlite3` module where columns with square braces in them have that portion of the name stripped out!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 564579430, "label": "Problem with square bracket in CSV column name"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/86#issuecomment-586676856", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/86", "id": 586676856, "node_id": "MDEyOklzc3VlQ29tbWVudDU4NjY3Njg1Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-02-16T07:20:34Z", "updated_at": "2020-02-16T07:20:34Z", "author_association": "OWNER", "body": "I'm not sure what to do about this one.\r\n\r\nI can't fix it: this bug in Python's `sqlite3` module means that even if I write a database out with column names that include `[]` I won't be able to read them back again.\r\n\r\nSo... I could do one of the following:\r\n\r\n- Throw an error if a column name includes those characters. That's my preferred option I think.\r\n- Automatically replace `[` in column names with `(` and `]` with `)`\r\n- Do the automatic replacement but show a user-visible warning when I do it\r\n- Throw an error, but give the user an option to run with e.g. `--fix-column-names` which applies that automatic fix.\r\n\r\nSince this is likely to be an incredibly rare edge-case I think I'd rather minimize the amount of code that deals with it, so my preferred option is to just throw that error and stop.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 564579430, "label": "Problem with square bracket in CSV column name"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/86#issuecomment-586729798", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/86", "id": 586729798, "node_id": "MDEyOklzc3VlQ29tbWVudDU4NjcyOTc5OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-02-16T17:11:02Z", "updated_at": "2020-02-16T17:11:02Z", "author_association": "OWNER", "body": "I filed a bug in the Python issue tracker here: https://bugs.python.org/issue39652", "reactions": "{\"total_count\": 2, \"+1\": 2, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 564579430, "label": "Problem with square bracket in CSV column name"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/86#issuecomment-591770623", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/86", "id": 591770623, "node_id": "MDEyOklzc3VlQ29tbWVudDU5MTc3MDYyMw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-02-27T04:12:39Z", "updated_at": "2020-02-27T04:12:39Z", "author_association": "OWNER", "body": "I pushed a branch with my experiment in it, but I'm going to fix this by throwing an error on `[` or `]` in a column name instead - I won't implement the changes from that branch.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 564579430, "label": "Problem with square bracket in CSV column name"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/85#issuecomment-584426938", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/85", "id": 584426938, "node_id": "MDEyOklzc3VlQ29tbWVudDU4NDQyNjkzOA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-02-11T00:35:09Z", "updated_at": "2020-02-11T00:35:09Z", "author_association": "OWNER", "body": "Here's why:\r\nhttps://github.com/simonw/sqlite-utils/blob/0c2451e0690c5f4e6463a2f339b0a280e30ed806/sqlite_utils/db.py#L627-L636", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 562911863, "label": "Create index doesn't work for columns containing spaces"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/83#issuecomment-583789015", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/83", "id": 583789015, "node_id": "MDEyOklzc3VlQ29tbWVudDU4Mzc4OTAxNQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-02-08T23:58:35Z", "updated_at": "2020-02-08T23:58:35Z", "author_association": "OWNER", "body": "Shipped as 2.3", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 559374410, "label": "Make db[\"table\"].exists a documented API"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/82#issuecomment-581651409", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/82", "id": 581651409, "node_id": "MDEyOklzc3VlQ29tbWVudDU4MTY1MTQwOQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-02-03T22:32:41Z", "updated_at": "2020-02-03T22:32:41Z", "author_association": "OWNER", "body": "This should work - the data should be chunked automatically. It looks like this is a bug.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 559197745, "label": "Tutorial command no longer works"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/82#issuecomment-581652388", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/82", "id": 581652388, "node_id": "MDEyOklzc3VlQ29tbWVudDU4MTY1MjM4OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-02-03T22:35:44Z", "updated_at": "2020-02-03T22:35:44Z", "author_association": "OWNER", "body": "I can't replicate this problem:\r\n```\r\n/tmp $ sqlite-utils --version\r\nsqlite-utils, version 2.2\r\n/tmp $ curl \"https://data.nasa.gov/resource/y77d-th95.json\" | sqlite-utils insert meteorites.db meteorites - --pk=id\r\n % Total % Received % Xferd Average Speed Time Time Time Current\r\n Dload Upload Total Spent Left Speed\r\n100 240k 0 240k 0 0 185k 0 --:--:-- 0:00:01 --:--:-- 185k\r\n```\r\nCould you run `sqlite-utils --version` and tell me what you get?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 559197745, "label": "Tutorial command no longer works"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/82#issuecomment-591771532", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/82", "id": 591771532, "node_id": "MDEyOklzc3VlQ29tbWVudDU5MTc3MTUzMg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-02-27T04:16:30Z", "updated_at": "2020-02-27T04:16:30Z", "author_association": "OWNER", "body": "Closing as can't reproduce.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 559197745, "label": "Tutorial command no longer works"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/81#issuecomment-581071010", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/81", "id": 581071010, "node_id": "MDEyOklzc3VlQ29tbWVudDU4MTA3MTAxMA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-02-01T21:27:00Z", "updated_at": "2020-02-01T21:27:00Z", "author_association": "OWNER", "body": "Here's the current method:\r\n\r\nhttps://github.com/simonw/sqlite-utils/blob/f7289174e66ae4d91d57de94bbd9d09fabf7aff4/sqlite_utils/db.py#L823-L845\r\n\r\nIf I make it a utility function instead of a class method I could ensure it is directly importable like so:\r\n\r\n```python\r\nfrom sqlite_utils import detect_column_types\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 558600274, "label": "Remove .detect_column_types() from table, make it a documented API"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/81#issuecomment-581071116", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/81", "id": 581071116, "node_id": "MDEyOklzc3VlQ29tbWVudDU4MTA3MTExNg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-02-01T21:28:35Z", "updated_at": "2020-02-01T21:28:53Z", "author_association": "OWNER", "body": "Should I keep `table.detect_column_types()` working so as not to break existing code?\r\n\r\nIf it was part of the documented API then I wouldn't break that without bumping to 3.x. Since it's undocumented I'm going to make it as a breaking change instead (and bump the `geojson-to-sqlite` dependency version).", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 558600274, "label": "Remove .detect_column_types() from table, make it a documented API"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/81#issuecomment-581071235", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/81", "id": 581071235, "node_id": "MDEyOklzc3VlQ29tbWVudDU4MTA3MTIzNQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-02-01T21:30:09Z", "updated_at": "2020-02-01T21:30:09Z", "author_association": "OWNER", "body": "Actually I'll put it in the `utils.py` module.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 558600274, "label": "Remove .detect_column_types() from table, make it a documented API"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/81#issuecomment-581071434", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/81", "id": 581071434, "node_id": "MDEyOklzc3VlQ29tbWVudDU4MTA3MTQzNA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-02-01T21:32:34Z", "updated_at": "2020-02-01T21:32:34Z", "author_association": "OWNER", "body": "While I'm at it I think I'll rename it to `suggest_column_types` - it's not really detecting them since the input is just a list of dictionaries.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 558600274, "label": "Remove .detect_column_types() from table, make it a documented API"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/80#issuecomment-580567505", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/80", "id": 580567505, "node_id": "MDEyOklzc3VlQ29tbWVudDU4MDU2NzUwNQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-01-31T03:39:19Z", "updated_at": "2020-01-31T03:39:19Z", "author_association": "OWNER", "body": "Still needs documentation and tests. Also I'm not certain that this should be an argument you can pass to the `.table()` constructor, need to think that over.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 557892819, "label": "on_create mechanism for after table creation"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/80#issuecomment-580567604", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/80", "id": 580567604, "node_id": "MDEyOklzc3VlQ29tbWVudDU4MDU2NzYwNA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-01-31T03:39:58Z", "updated_at": "2020-01-31T03:39:58Z", "author_association": "OWNER", "body": "Perhaps this should be called `after_create` instead of `on_create`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 557892819, "label": "on_create mechanism for after table creation"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/80#issuecomment-580567886", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/80", "id": 580567886, "node_id": "MDEyOklzc3VlQ29tbWVudDU4MDU2Nzg4Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-01-31T03:41:31Z", "updated_at": "2020-01-31T03:41:31Z", "author_association": "OWNER", "body": "I think it does make sense to be able to pass it to the `.table()` constructor.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 557892819, "label": "on_create mechanism for after table creation"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/80#issuecomment-580569059", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/80", "id": 580569059, "node_id": "MDEyOklzc3VlQ29tbWVudDU4MDU2OTA1OQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-01-31T03:48:41Z", "updated_at": "2020-01-31T03:48:41Z", "author_association": "OWNER", "body": "This may not be the right feature after all, see https://github.com/simonw/geojson-to-sqlite/issues/6#issuecomment-580569002", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 557892819, "label": "on_create mechanism for after table creation"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/80#issuecomment-580584269", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/80", "id": 580584269, "node_id": "MDEyOklzc3VlQ29tbWVudDU4MDU4NDI2OQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-01-31T05:08:04Z", "updated_at": "2020-01-31T05:08:04Z", "author_association": "OWNER", "body": "Ditching this since it won't actually solve my problem.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 557892819, "label": "on_create mechanism for after table creation"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/8#issuecomment-466695500", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/8", "id": 466695500, "node_id": "MDEyOklzc3VlQ29tbWVudDQ2NjY5NTUwMA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-02-23T21:09:03Z", "updated_at": "2019-02-23T21:09:03Z", "author_association": "OWNER", "body": "Fixed in https://github.com/simonw/sqlite-utils/commit/228d595f7d10994f34e948888093c2cd290267c4\r\n\r\n\"screen\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 403922644, "label": "Problems handling column names containing spaces or - "}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/79#issuecomment-1029683977", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/79", "id": 1029683977, "node_id": "IC_kwDOCGYnMM49X7sJ", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-02-04T05:58:15Z", "updated_at": "2022-02-04T05:58:15Z", "author_association": "OWNER", "body": "Documentation: https://sqlite-utils.datasette.io/en/latest/python-api.html#spatialite-helpers", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 557842245, "label": "Helper methods for working with SpatiaLite"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/79#issuecomment-1029703503", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/79", "id": 1029703503, "node_id": "IC_kwDOCGYnMM49YAdP", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-02-04T06:46:32Z", "updated_at": "2022-02-04T06:46:32Z", "author_association": "OWNER", "body": "Shipped in 3.23: https://sqlite-utils.datasette.io/en/stable/changelog.html#v3-23", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 557842245, "label": "Helper methods for working with SpatiaLite"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/77#issuecomment-580515506", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/77", "id": 580515506, "node_id": "MDEyOklzc3VlQ29tbWVudDU4MDUxNTUwNg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-01-30T23:48:41Z", "updated_at": "2020-01-30T23:48:41Z", "author_association": "OWNER", "body": "Potential design: a `conversions={}` option. Used like this:\r\n```python\r\ndb[table].insert(record, conversions={\"geom\": \"GeomFromText(?, 4326)\"})\r\n```\r\nThe `conversions=` key would be supported on `.insert()`, `.insert_all()`, `.upsert()` etc. It could also be passed to the `db.table()` constructor function:\r\n```python\r\ntable = db.table(\r\n \"features\",\r\n pk=\"id\",\r\n conversions={\r\n \"geom\": \"GeomFromText(?, 4326)\"\r\n }\r\n)\r\n# Then used like this:\r\ntable.insert(record)\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 557825032, "label": "Ability to insert data that is transformed by a SQL function"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/77#issuecomment-580527238", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/77", "id": 580527238, "node_id": "MDEyOklzc3VlQ29tbWVudDU4MDUyNzIzOA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-01-31T00:34:02Z", "updated_at": "2020-01-31T00:34:02Z", "author_association": "OWNER", "body": "Documentation: https://sqlite-utils.readthedocs.io/en/stable/python-api.html#python-api-conversions", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 557825032, "label": "Ability to insert data that is transformed by a SQL function"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/76#issuecomment-614354219", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/76", "id": 614354219, "node_id": "MDEyOklzc3VlQ29tbWVudDYxNDM1NDIxOQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-04-16T01:01:34Z", "updated_at": "2020-04-16T01:01:34Z", "author_association": "OWNER", "body": "I think a neat way to do this would be with an optional argument for `.rows_where()`:\r\n\r\n```python\r\nrows = db[\"table\"].rows_where(\"age > 10\", order_by=\"age desc\")\r\n```\r\n\r\nIf you want everything you can use this:\r\n\r\n```python\r\nrows = db[\"table\"].rows_where(order_by=\"age desc\")\r\n```\r\n\r\nIt's a tiny bit weird calling `.rows_where()` without a where clause, but I think it makes sense here - especially since `.rows` is a property that can't take any arguments - though under the hood it actually does this:\r\n\r\nhttps://github.com/simonw/sqlite-utils/blob/ad6ac19470a67867b96cb4c086450b8e4e46bf02/sqlite_utils/db.py#L436-L443", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 549287310, "label": "order_by mechanism"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/76#issuecomment-614400454", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/76", "id": 614400454, "node_id": "MDEyOklzc3VlQ29tbWVudDYxNDQwMDQ1NA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-04-16T03:51:01Z", "updated_at": "2020-04-16T03:51:01Z", "author_association": "OWNER", "body": "Released in 2.6", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 549287310, "label": "order_by mechanism"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/76#issuecomment-614400533", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/76", "id": 614400533, "node_id": "MDEyOklzc3VlQ29tbWVudDYxNDQwMDUzMw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-04-16T03:51:26Z", "updated_at": "2020-04-16T03:51:26Z", "author_association": "OWNER", "body": "Documentation here: https://sqlite-utils.readthedocs.io/en/stable/python-api.html#listing-rows", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 549287310, "label": "order_by mechanism"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/75#issuecomment-580523995", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/75", "id": 580523995, "node_id": "MDEyOklzc3VlQ29tbWVudDU4MDUyMzk5NQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-01-31T00:21:11Z", "updated_at": "2020-01-31T00:21:11Z", "author_association": "OWNER", "body": "This makes sense, thanks!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 546078359, "label": "Explicitly include tests and docs in sdist"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/74#issuecomment-572871797", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/74", "id": 572871797, "node_id": "MDEyOklzc3VlQ29tbWVudDU3Mjg3MTc5Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-01-10T04:47:55Z", "updated_at": "2020-01-10T04:47:55Z", "author_association": "OWNER", "body": "This is odd. I'd love to see more about that result object.\r\n\r\nCould you try running `pytest --pdb` and then `result.exit_code, result.exception` in the PDB prompt, something like this?\r\n\r\n```\r\n$ pytest --pdb\r\n========================================================= test session starts =========================================================\r\nplatform darwin -- Python 3.7.4, pytest-5.2.2, py-1.8.0, pluggy-0.13.0\r\nrootdir: /Users/simonw/Dropbox/Development/sqlite-utils\r\nplugins: cov-2.8.1\r\ncollected 216 items \r\n\r\ntests/test_black.py s [ 0%]\r\ntests/test_cli.py F\r\n>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> traceback >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>\r\n\r\ndb_path = '/private/var/folders/bl/5x847xbj2yb7xmp7f2tz7l280000gn/T/pytest-of-simonw/pytest-3/test_tables0/test.db'\r\n\r\n def test_tables(db_path):\r\n result = CliRunner().invoke(cli.cli, [\"tables1\", db_path])\r\n> assert '[{\"table\": \"Gosh\"},\\n {\"table\": \"Gosh2\"}]' == result.output.strip()\r\nE assert '[{\"table\": \"...e\": \"Gosh2\"}]' == ''\r\nE - [{\"table\": \"Gosh\"},\r\nE - {\"table\": \"Gosh2\"}]\r\n\r\ntests/test_cli.py:28: AssertionError\r\n>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> entering PDB >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>\r\n\r\n>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PDB post_mortem (IO-capturing turned off) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>\r\n> /Users/simonw/Dropbox/Development/sqlite-utils/tests/test_cli.py(28)test_tables()\r\n-> assert '[{\"table\": \"Gosh\"},\\n {\"table\": \"Gosh2\"}]' == result.output.strip()\r\n(Pdb) result.exit_code, result.exception\r\n(1, OperationalError('near \"/\": syntax error'))\r\n```\r\n\r\nThat should show the exception that caused the script to fail to run.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 546073980, "label": "Test failures on openSUSE 15.1: AssertionError: Explicit other_table and other_column"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/73#issuecomment-570930239", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/73", "id": 570930239, "node_id": "MDEyOklzc3VlQ29tbWVudDU3MDkzMDIzOQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-01-05T17:15:18Z", "updated_at": "2020-01-05T17:15:18Z", "author_association": "OWNER", "body": "I think this is because you forgot to include a `pk=` argument. I'll change the code to throw a more useful error in this case.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 545407916, "label": "upsert_all() throws issue when upserting to empty table"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/73#issuecomment-570931650", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/73", "id": 570931650, "node_id": "MDEyOklzc3VlQ29tbWVudDU3MDkzMTY1MA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-01-05T17:34:33Z", "updated_at": "2020-01-05T17:34:33Z", "author_association": "OWNER", "body": "Released as 2.0.1 https://github.com/simonw/sqlite-utils/releases/tag/2.0.1", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 545407916, "label": "upsert_all() throws issue when upserting to empty table"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/73#issuecomment-572870032", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/73", "id": 572870032, "node_id": "MDEyOklzc3VlQ29tbWVudDU3Mjg3MDAzMg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-01-10T04:38:41Z", "updated_at": "2020-01-10T04:38:41Z", "author_association": "OWNER", "body": "Odd.. I'm not able to replicate that error. Here's what I got:\r\n\"Tony_Bug\"\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 545407916, "label": "upsert_all() throws issue when upserting to empty table"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/71#issuecomment-569233996", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/71", "id": 569233996, "node_id": "MDEyOklzc3VlQ29tbWVudDU2OTIzMzk5Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-12-27T09:45:17Z", "updated_at": "2019-12-27T09:45:17Z", "author_association": "OWNER", "body": "It looks like those backports no longer include sqlite3 - Google Searches still find it but when you click through to launchpad you get 404s: https://launchpad.net/~jonathonf/+archive/ubuntu/codelite/+build/10511920\r\n\r\nMaybe Travis have a newer Ubuntu I can use that ships with FTS5 in its SQLite?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 542814756, "label": "Tests are failing due to missing FTS5"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/71#issuecomment-569234096", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/71", "id": 569234096, "node_id": "MDEyOklzc3VlQ29tbWVudDU2OTIzNDA5Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-12-27T09:45:52Z", "updated_at": "2019-12-27T09:45:52Z", "author_association": "OWNER", "body": "I'll try `bionic`: https://docs.travis-ci.com/user/reference/bionic/", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 542814756, "label": "Tests are failing due to missing FTS5"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/71#issuecomment-569234571", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/71", "id": 569234571, "node_id": "MDEyOklzc3VlQ29tbWVudDU2OTIzNDU3MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-12-27T09:48:48Z", "updated_at": "2019-12-27T09:48:48Z", "author_association": "OWNER", "body": "That fixed it: https://travis-ci.com/simonw/sqlite-utils/builds/142443259", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 542814756, "label": "Tests are failing due to missing FTS5"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/70#issuecomment-569130037", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/70", "id": 569130037, "node_id": "MDEyOklzc3VlQ29tbWVudDU2OTEzMDAzNw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-12-26T20:39:04Z", "updated_at": "2019-12-26T20:39:04Z", "author_association": "OWNER", "body": "I hadn't thought about those at all.\r\n\r\nAre you suggesting a utility mechanism in the library for setting it up so that, for a specific foreign key, rows are deleted from other tables if the row they are pointing at is deleted?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 539204432, "label": "Implement ON DELETE and ON UPDATE actions for foreign keys"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/7#issuecomment-457980966", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/7", "id": 457980966, "node_id": "MDEyOklzc3VlQ29tbWVudDQ1Nzk4MDk2Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-01-28T02:29:32Z", "updated_at": "2019-01-28T02:29:32Z", "author_association": "OWNER", "body": "Remember to remove this TODO (and turn the `[]` into `()` on this line) as part of this task:\r\n\r\nhttps://github.com/simonw/sqlite-utils/blob/5309c5c7755818323a0f5353bad0de98ecc866be/sqlite_utils/cli.py#L78-L80", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 403625674, "label": ".insert_all() should accept a generator and process it efficiently"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/7#issuecomment-458011885", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/7", "id": 458011885, "node_id": "MDEyOklzc3VlQ29tbWVudDQ1ODAxMTg4NQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-01-28T06:25:48Z", "updated_at": "2019-01-28T06:25:48Z", "author_association": "OWNER", "body": "Re-opening for the second bit involving the cli tool.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 403625674, "label": ".insert_all() should accept a generator and process it efficiently"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/7#issuecomment-458011906", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/7", "id": 458011906, "node_id": "MDEyOklzc3VlQ29tbWVudDQ1ODAxMTkwNg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-01-28T06:25:55Z", "updated_at": "2019-01-28T06:25:55Z", "author_association": "OWNER", "body": "I tested this with a script called `churn_em_out.py`\r\n```\r\ni = 0\r\nwhile True:\r\n i += 1\r\n print(\r\n '{\"id\": I, \"another\": \"row\", \"number\": J}'.replace(\"I\", str(i)).replace(\r\n \"J\", str(i + 1)\r\n )\r\n )\r\n```\r\n\r\nThen I ran this:\r\n```\r\npython churn_em_out.py | \\\r\n sqlite-utils insert /tmp/getbig.db stats - \\\r\n --nl --batch-size=10000\r\n```\r\nAnd used `watch 'ls -lah /tmp/getbig.db'` to watch the file growing as it had 10,000 lines of junk committed in batches. The memory used by the process never grew about around 50MB.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 403625674, "label": ".insert_all() should accept a generator and process it efficiently"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/69#issuecomment-569130196", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/69", "id": 569130196, "node_id": "MDEyOklzc3VlQ29tbWVudDU2OTEzMDE5Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-12-26T20:40:21Z", "updated_at": "2019-12-26T20:40:21Z", "author_association": "OWNER", "body": "This is a good idea. Datasette has this in the form of the `--load-extension` CLI argument, e.g. for SpatiaLite here: https://datasette.readthedocs.io/en/stable/spatialite.html#installation\r\n\r\nHaving that available for `sqlite-utils` definitely makes sense.", "reactions": "{\"total_count\": 1, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 1}", "issue": {"value": 534507142, "label": "Feature request: enable extensions loading"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/69#issuecomment-710405658", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/69", "id": 710405658, "node_id": "MDEyOklzc3VlQ29tbWVudDcxMDQwNTY1OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-16T18:42:48Z", "updated_at": "2020-10-16T18:42:48Z", "author_association": "OWNER", "body": "Did some work on this for #134 in 7e9aad7e1c09d1cf80d0b4d17d6157212a4b857d\r\n\r\nI still need to add `--load-extension` to other CLI methods, see #137. Closing this issue in favour of that one.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 534507142, "label": "Feature request: enable extensions loading"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/68#issuecomment-695695776", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/68", "id": 695695776, "node_id": "MDEyOklzc3VlQ29tbWVudDY5NTY5NTc3Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-09-20T04:25:47Z", "updated_at": "2020-09-20T04:25:47Z", "author_association": "OWNER", "body": "This is a dupe of #130 ", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 531583658, "label": "Add support for porter stemming in FTS"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/67#issuecomment-559108591", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/67", "id": 559108591, "node_id": "MDEyOklzc3VlQ29tbWVudDU1OTEwODU5MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-11-27T14:24:59Z", "updated_at": "2019-11-27T14:24:59Z", "author_association": "OWNER", "body": "Failed due to black testing dependency: https://travis-ci.com/simonw/sqlite-utils/jobs/260995814", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 529376481, "label": "Run tests against 3.5 too"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/67#issuecomment-569844320", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/67", "id": 569844320, "node_id": "MDEyOklzc3VlQ29tbWVudDU2OTg0NDMyMA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-12-31T01:29:43Z", "updated_at": "2019-12-31T01:29:43Z", "author_association": "OWNER", "body": "I don't really care about 3.5 any more.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 529376481, "label": "Run tests against 3.5 too"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/66#issuecomment-553170650", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/66", "id": 553170650, "node_id": "MDEyOklzc3VlQ29tbWVudDU1MzE3MDY1MA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-11-12T23:49:29Z", "updated_at": "2019-11-12T23:49:29Z", "author_association": "OWNER", "body": "This relates to this bug: https://github.com/dogsheep/github-to-sqlite/pull/8#issuecomment-549233778", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 521868864, "label": "The \".upsert()\" method is misnamed"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/66#issuecomment-553171011", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/66", "id": 553171011, "node_id": "MDEyOklzc3VlQ29tbWVudDU1MzE3MTAxMQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-11-12T23:50:52Z", "updated_at": "2019-11-12T23:50:52Z", "author_association": "OWNER", "body": "Fixing this is going to be a real pain. There's lots of code out there that uses `sqlite-utils` with the expectation that `upsert()` behaves as it currently does.\r\n\r\nMaybe I need to introduce new terms for both of these different patterns and deprecate the existing `.upsert()` and `.upsert_all()` since their behaviour can't be changed?\r\n\r\nOr maybe I fix this and ship `sqlite-utils 2.0` with a breaking change?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 521868864, "label": "The \".upsert()\" method is misnamed"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/66#issuecomment-553171414", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/66", "id": 553171414, "node_id": "MDEyOklzc3VlQ29tbWVudDU1MzE3MTQxNA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-11-12T23:52:35Z", "updated_at": "2019-11-12T23:52:35Z", "author_association": "OWNER", "body": "If I do implement the correct definition of `.upsert()` I think I'll use this pattern, since it works in versions of SQLite prior to 3.24:\r\n```sql\r\nINSERT OR IGNORE INTO book(id) VALUES(1001);\r\nUPDATE book SET name = 'Programming' WHERE id = 1001;\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 521868864, "label": "The \".upsert()\" method is misnamed"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/66#issuecomment-553526685", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/66", "id": 553526685, "node_id": "MDEyOklzc3VlQ29tbWVudDU1MzUyNjY4NQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-11-13T17:58:59Z", "updated_at": "2019-11-13T17:58:59Z", "author_association": "OWNER", "body": "This warrants making a backwards compatible change, which means I'll need to bump the major version number and release 2.0.\r\n\r\nI'm going to rename the existing `upsert()` and `upsert_all()` methods to `replace()` and `replace_all()` - then write new `upsert()` and `upsert_all()` methods that implement the correct behavior.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 521868864, "label": "The \".upsert()\" method is misnamed"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/66#issuecomment-553527384", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/66", "id": 553527384, "node_id": "MDEyOklzc3VlQ29tbWVudDU1MzUyNzM4NA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-11-13T18:00:41Z", "updated_at": "2019-11-13T18:00:41Z", "author_association": "OWNER", "body": "Is `replace()` a good name here? It doesn't really convey the idea that a brand new record will be created if there isn't an existing one to replace.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 521868864, "label": "The \".upsert()\" method is misnamed"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/66#issuecomment-553528386", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/66", "id": 553528386, "node_id": "MDEyOklzc3VlQ29tbWVudDU1MzUyODM4Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-11-13T18:03:10Z", "updated_at": "2019-11-13T18:03:54Z", "author_association": "OWNER", "body": "Maybe `inplace()` (combining \"insert\" and \"replace\")?\r\n\r\nIt could be an alias for `.insert(..., replace=True)`", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 521868864, "label": "The \".upsert()\" method is misnamed"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/66#issuecomment-553528850", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/66", "id": 553528850, "node_id": "MDEyOklzc3VlQ29tbWVudDU1MzUyODg1MA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-11-13T18:04:20Z", "updated_at": "2019-11-13T18:04:20Z", "author_association": "OWNER", "body": "This is going to affect the design of the CLI subcommands as well.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 521868864, "label": "The \".upsert()\" method is misnamed"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/66#issuecomment-553540146", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/66", "id": 553540146, "node_id": "MDEyOklzc3VlQ29tbWVudDU1MzU0MDE0Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-11-13T18:33:30Z", "updated_at": "2019-11-13T18:33:30Z", "author_association": "OWNER", "body": "Maybe instead of inventing a new term I should tell people to use `.insert(..., replace=True)` directly. That matches `ignore=True`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 521868864, "label": "The \".upsert()\" method is misnamed"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/66#issuecomment-553574011", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/66", "id": 553574011, "node_id": "MDEyOklzc3VlQ29tbWVudDU1MzU3NDAxMQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-11-13T19:53:45Z", "updated_at": "2019-11-13T19:53:45Z", "author_association": "OWNER", "body": "First step: add a `replace=True` argument to `insert()` and `insert_all()` that does the same thing as the current `upsert=True`\r\n\r\nhttps://github.com/simonw/sqlite-utils/blob/8dab9fd1ccf571e188eec9ccf606a0c50fccf200/sqlite_utils/db.py#L938-L946", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 521868864, "label": "The \".upsert()\" method is misnamed"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/66#issuecomment-554565198", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/66", "id": 554565198, "node_id": "MDEyOklzc3VlQ29tbWVudDU1NDU2NTE5OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-11-15T23:12:28Z", "updated_at": "2019-11-15T23:12:28Z", "author_association": "OWNER", "body": "Urgh this is going to be quite a bit of work, especially in the CLI module which shares an implementation for `upsert` and `insert` in a way that looks like it will have to be unwrapped.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 521868864, "label": "The \".upsert()\" method is misnamed"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/66#issuecomment-555690319", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/66", "id": 555690319, "node_id": "MDEyOklzc3VlQ29tbWVudDU1NTY5MDMxOQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-11-19T20:10:17Z", "updated_at": "2019-11-19T20:10:17Z", "author_association": "OWNER", "body": "Thinking about this further: I believe every time I've personally used `upsert` in the past (either with the Python library or the CLI tool) I've actually wanted the new behaviour, where \"upsert\" means \"update existing record with these changes, or insert a new record if one does not exist\".\r\n\r\nSo I'm happy with `upsert` doing that, and `insert --replace` being added as an option that does what `upsert` does ta the moment.\r\n\r\nI'll still ship it as version 2.0 since it's technically a breaking change.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 521868864, "label": "The \".upsert()\" method is misnamed"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/66#issuecomment-569131397", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/66", "id": 569131397, "node_id": "MDEyOklzc3VlQ29tbWVudDU2OTEzMTM5Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-12-26T20:49:11Z", "updated_at": "2019-12-26T20:49:11Z", "author_association": "OWNER", "body": "Don't forget to update the documentation. This will be quite an involved task.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 521868864, "label": "The \".upsert()\" method is misnamed"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/66#issuecomment-569226620", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/66", "id": 569226620, "node_id": "MDEyOklzc3VlQ29tbWVudDU2OTIyNjYyMA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-12-27T09:05:29Z", "updated_at": "2019-12-27T09:05:36Z", "author_association": "OWNER", "body": "I'm going to start by ignoring the existing `upsert` entirely and implementing `.insert(..., replace=True)` and `$ sqlite-utils insert --replace`. Including updating the tests.\r\n\r\nThen I'll figure out how to implement the new `.upsert()` / `$ sqlite-utils upsert`.\r\n\r\nThen I'll update the documentation, and ship `sqlite-utils` 2.0.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 521868864, "label": "The \".upsert()\" method is misnamed"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/66#issuecomment-569588216", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/66", "id": 569588216, "node_id": "MDEyOklzc3VlQ29tbWVudDU2OTU4ODIxNg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-12-30T05:31:45Z", "updated_at": "2019-12-30T05:31:45Z", "author_association": "OWNER", "body": "Last step: update changelog and ship 2.0. Then I can close this issue.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 521868864, "label": "The \".upsert()\" method is misnamed"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/66#issuecomment-569844426", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/66", "id": 569844426, "node_id": "MDEyOklzc3VlQ29tbWVudDU2OTg0NDQyNg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-12-31T01:30:20Z", "updated_at": "2019-12-31T01:30:20Z", "author_association": "OWNER", "body": "I shipped 2.0 - release notes here: https://sqlite-utils.readthedocs.io/en/stable/changelog.html#v2\r\n\r\nI also wrote about it on my blog: https://simonwillison.net/2019/Dec/30/sqlite-utils-2/", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 521868864, "label": "The \".upsert()\" method is misnamed"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/63#issuecomment-549430429", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/63", "id": 549430429, "node_id": "MDEyOklzc3VlQ29tbWVudDU0OTQzMDQyOQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-11-04T16:20:35Z", "updated_at": "2019-11-04T16:20:35Z", "author_association": "OWNER", "body": "I don't think we need this. We already have a `dogs.create_index([\"name\"], if_not_exists=True)` option.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 517241040, "label": "ensure_index() method"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/62#issuecomment-549418134", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/62", "id": 549418134, "node_id": "MDEyOklzc3VlQ29tbWVudDU0OTQxODEzNA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-11-04T15:54:18Z", "updated_at": "2019-11-04T15:54:18Z", "author_association": "OWNER", "body": "This is a good idea. The individual row version of it can work like the `.update(pk, ...)` method.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 500783373, "label": "[enhancement] Method to delete a row in python"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/62#issuecomment-549425012", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/62", "id": 549425012, "node_id": "MDEyOklzc3VlQ29tbWVudDU0OTQyNTAxMg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-11-04T16:09:08Z", "updated_at": "2019-11-04T16:09:08Z", "author_association": "OWNER", "body": "Documentation for `table.delete()`: https://github.com/simonw/sqlite-utils/blob/19073d6d972fad9d68dd74c28544cd29083f1c12/docs/python-api.rst#deleting-a-specific-record", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 500783373, "label": "[enhancement] Method to delete a row in python"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/62#issuecomment-549425364", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/62", "id": 549425364, "node_id": "MDEyOklzc3VlQ29tbWVudDU0OTQyNTM2NA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-11-04T16:09:56Z", "updated_at": "2019-11-04T16:09:56Z", "author_association": "OWNER", "body": "Since we have `table.rows_where(where, where_args)` it makes sense to me to also support `table.delete_where(where, where_args)`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 500783373, "label": "[enhancement] Method to delete a row in python"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/62#issuecomment-549429512", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/62", "id": 549429512, "node_id": "MDEyOklzc3VlQ29tbWVudDU0OTQyOTUxMg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-11-04T16:18:48Z", "updated_at": "2019-11-04T16:18:48Z", "author_association": "OWNER", "body": "Documentation for `.delete_where()`: https://github.com/simonw/sqlite-utils/blob/169ea455fc1f1d5e5b6e44cb339ba7ffa9d49c31/docs/python-api.rst#deleting-multiple-records", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 500783373, "label": "[enhancement] Method to delete a row in python"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/62#issuecomment-549435364", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/62", "id": 549435364, "node_id": "MDEyOklzc3VlQ29tbWVudDU0OTQzNTM2NA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-11-04T16:30:34Z", "updated_at": "2019-11-04T16:30:34Z", "author_association": "OWNER", "body": "Released as 1.12.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 500783373, "label": "[enhancement] Method to delete a row in python"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/61#issuecomment-549432592", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/61", "id": 549432592, "node_id": "MDEyOklzc3VlQ29tbWVudDU0OTQzMjU5Mg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-11-04T16:25:01Z", "updated_at": "2019-11-04T16:25:01Z", "author_association": "OWNER", "body": "Yeah I've thought about this a bit and I'm OK leaving it out.\r\n\r\nThe core idea of `sqlite-utils` is that if you can create a list, iterator or generator of Python dictionaries you can efficiently insert those into a SQLite table.\r\n\r\nThe `--csv` function is actually implemented as just a few lines of code which turn that incoming CSV into a generator of dictionaries:\r\n\r\nhttps://github.com/simonw/sqlite-utils/blob/169ea455fc1f1d5e5b6e44cb339ba7ffa9d49c31/sqlite_utils/cli.py#L364-L368\r\n\r\nI could turn this into a reusable helper function but since it wouldn't have anything to do with database inserts (it would just be a helper that turns a CSV into a generator of dictionaries) it doesn't feel like it fits well in the Python library section of this package.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 491219910, "label": "importing CSV to SQLite as library"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/606#issuecomment-1843465748", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/606", "id": 1843465748, "node_id": "IC_kwDOCGYnMM5t4Q4U", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-12-06T18:36:51Z", "updated_at": "2023-12-06T18:36:51Z", "author_association": "OWNER", "body": "I'll add `bytes` too - `float` already works. This makes sense because when you are working with the Python API you use `str` and `float` and `bytes` and `int` to specify column types.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 2029161033, "label": "str and int as aliases for text and integer"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/606#issuecomment-1843579184", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/606", "id": 1843579184, "node_id": "IC_kwDOCGYnMM5t4skw", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-12-06T19:43:55Z", "updated_at": "2023-12-06T19:43:55Z", "author_association": "OWNER", "body": "Updated documentation:\r\n- https://sqlite-utils.datasette.io/en/latest/cli.html#cli-add-column\r\n- https://sqlite-utils.datasette.io/en/latest/cli-reference.html#add-column", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 2029161033, "label": "str and int as aliases for text and integer"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/605#issuecomment-1846554637", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/605", "id": 1846554637, "node_id": "IC_kwDOCGYnMM5uEDAN", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-12-08T05:07:54Z", "updated_at": "2023-12-08T05:07:54Z", "author_association": "OWNER", "body": "Thanks for opening an issue - this should help future Google searchers figure out what's going on here.\r\n\r\nAnother approach here could be to store large integers as `TEXT` in SQLite (or even as `BLOB`).\r\n\r\nBoth storing as `REAL` and storing as `TEXT/BLOB` feel nasty to me, but it looks like SQLite has a hard upper limit of 9223372036854775807 for integers.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 2007893839, "label": "Insert fails with `Error: Python int too large to convert to SQLite INTEGER`; can we use `NUMERIC` here?"}, "performed_via_github_app": null}