{"id": 1149729902, "node_id": "PR_kwDOCGYnMM4zbaJy", "number": 410, "title": "Correct spelling mistakes (found with codespell)", "user": {"value": 3818, "label": "EdwardBetts"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2022-02-24T20:44:18Z", "updated_at": "2022-03-06T08:48:29Z", "closed_at": "2022-03-01T21:05:29Z", "author_association": "CONTRIBUTOR", "pull_request": "simonw/sqlite-utils/pulls/410", "body": null, "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/410/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 919250621, "node_id": "MDU6SXNzdWU5MTkyNTA2MjE=", "number": 269, "title": "bool type not supported", "user": {"value": 4068, "label": "frafra"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2021-06-11T22:00:36Z", "updated_at": "2021-06-15T01:34:10Z", "closed_at": "2021-06-15T01:34:10Z", "author_association": "NONE", "pull_request": null, "body": "Hi! Thank you for sharing this very nice tool :)\r\nIt would be nice to have support for more types, like `bool`: it is not possible to convert to boolean at the moment. My suggestion would be to handle it as `bool(int(value))`, like csvkit does.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/269/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 919314806, "node_id": "MDU6SXNzdWU5MTkzMTQ4MDY=", "number": 270, "title": "Cannot set type JSON", "user": {"value": 4068, "label": "frafra"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2021-06-11T23:53:22Z", "updated_at": "2021-06-16T17:34:49Z", "closed_at": "2021-06-16T15:47:06Z", "author_association": "NONE", "pull_request": null, "body": "It would be great if the column type could be set to JSON. That would not be different from handling a regular string. It would be something like `repr(value)` and it would work with both JSON and CSV inputs, no matter if `value` is a real list or just a string representing a list.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/270/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1250161887, "node_id": "I_kwDOCGYnMM5Kg_Tf", "number": 438, "title": "illegal UTF-16 surrogate", "user": {"value": 4068, "label": "frafra"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2022-05-26T22:49:52Z", "updated_at": "2022-05-27T08:21:53Z", "closed_at": "2022-05-27T08:21:53Z", "author_association": "NONE", "pull_request": null, "body": "I am trying to insert `https://artsdatabanken.no/Fab2018/api/export/csv` into a SQLite database, but I have an error when using `sqlite-utils`:\r\n\r\n```\r\nsqlite-utils insert --csv --delimiter \";\" --encoding=\"utf-16-le\" --pk \"Id\" csv fremmedart test.db\r\n  [------------------------------------]    0%\r\nError: 'utf-16-le' codec can't decode bytes in position 98-99: illegal UTF-16 surrogate\r\n\r\nThe input you provided uses a character encoding other than utf-8.\r\n\r\nYou can fix this by passing the --encoding= option with the encoding of the file.\r\n\r\nIf you do not know the encoding, running 'file filename.csv' may tell you.\r\n\r\nIt's often worth trying: --encoding=latin-1\r\n```\r\n\r\nI tried to convert the file using `iconv -f \"utf-16le\" -t \"utf-8\"`, but I still get a similar error (slightly different position):\r\n\r\n```\r\nsqlite-utils insert --csv --delimiter \";\" --encoding=utf-8 --pk \"Id\" csv_utf8 fremmedart test.db\r\n  [------------------------------------]    0%\r\nError: 'utf-8' codec can't decode byte 0xd9 in position 99: invalid continuation byte\r\n\r\nThe input you provided uses a character encoding other than utf-8.\r\n\r\nYou can fix this by passing the --encoding= option with the encoding of the file.\r\n\r\nIf you do not know the encoding, running 'file filename.csv' may tell you.\r\n\r\nIt's often worth trying: --encoding=latin-1\r\n```\r\n\r\nI have no issues reading such file using this Python code:\r\n```python\r\ncontent = open('csv', encoding='utf-16-le').read())\r\n```\r\n\r\n`in2csv` works too.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/438/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1250495688, "node_id": "I_kwDOCGYnMM5KiQzI", "number": 439, "title": "Misleading progress bar against utf-16-le CSV input", "user": {"value": 4068, "label": "frafra"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 12, "created_at": "2022-05-27T08:34:49Z", "updated_at": "2022-06-15T03:53:43Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "The program crashes without any error.\r\n```\r\nwget \"https://artsdatabanken.no/Fab2018/api/export/csv\"\r\nsqlite-utils create-database test.db\r\nsqlite-utils insert --csv --delimiter \";\" --encoding \"utf-16-le\" test test.db csv \r\n  [------------------------------------]    0%\r\n  [#################-------------------]   49%  00:00:01\r\n```\r\nI would like to highlight various issues:\r\n1. sqlite-utils catches exceptions without printing the stacktrace and/or reraising the exception, so there is no easy way to use `pdb` or similar to debug the program, solution: add a debug option\r\n2. Silent crash: this is related to (1.), and it happens when there is a catch-all mechanism; solution: let the program fail.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/439/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 1250629388, "node_id": "I_kwDOCGYnMM5KixcM", "number": 440, "title": "CSV files with too many values in a row cause errors", "user": {"value": 4068, "label": "frafra"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 20, "created_at": "2022-05-27T10:54:44Z", "updated_at": "2022-06-14T22:23:01Z", "closed_at": "2022-06-14T20:12:46Z", "author_association": "NONE", "pull_request": null, "body": "*Original title: csv.DictReader can have None as key*\r\n\r\nIn some cases, `csv.DictReader` can have `None` as key for unnamed columns, and a list of values as value.\r\n`sqlite_utils.utils.rows_from_file` cannot handle that:\r\n\r\n```python\r\nurl=\"https://artsdatabanken.no/Fab2018/api/export/csv\"\r\ndb = sqlite_utils.Database(\":memory\")\r\n\r\nwith urlopen(url) as fab:\r\n    reader, _ = sqlite_utils.utils.rows_from_file(fab, encoding=\"utf-16le\")   \r\n    db[\"fab2018\"].insert_all(reader, pk=\"Id\")\r\n```\r\n\r\nResult:\r\n```\r\nTraceback (most recent call last):\r\n  File \"<stdin>\", line 3, in <module>\r\n  File \"/home/user/.local/pipx/venvs/sqlite-utils/lib/python3.8/site-packages/sqlite_utils/db.py\", line 2924, in insert_all\r\n    chunk = list(chunk)\r\n  File \"/home/user/.local/pipx/venvs/sqlite-utils/lib/python3.8/site-packages/sqlite_utils/db.py\", line 3454, in fix_square_braces\r\n    if any(\"[\" in key or \"]\" in key for key in record.keys()):\r\n  File \"/home/user/.local/pipx/venvs/sqlite-utils/lib/python3.8/site-packages/sqlite_utils/db.py\", line 3454, in <genexpr>\r\n    if any(\"[\" in key or \"]\" in key for key in record.keys()):\r\nTypeError: argument of type 'NoneType' is not iterable\r\n```\r\n\r\nCode:\r\nhttps://github.com/simonw/sqlite-utils/blob/59be60c471fd7a2c4be7f75e8911163e618ff5ca/sqlite_utils/db.py#L3454\r\n\r\n`sqlite-utils insert` from command line is not affected by this issue.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/440/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 347058326, "node_id": "MDExOlB1bGxSZXF1ZXN0MjA1NzcwOTk2", "number": 1, "title": "Make .indexes compatible with older SQLite versions", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2018-08-02T15:17:05Z", "updated_at": "2018-08-02T15:17:30Z", "closed_at": "2018-08-02T15:17:30Z", "author_association": "OWNER", "pull_request": "simonw/sqlite-utils/pulls/1", "body": "Older SQLite versions return a different set of columns from the PRAGMA we are using.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/1/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 349850687, "node_id": "MDU6SXNzdWUzNDk4NTA2ODc=", "number": 2, "title": "Mechanism for adding foreign keys to an existing table", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2018-08-12T22:50:56Z", "updated_at": "2019-02-24T21:34:41Z", "closed_at": "2019-02-24T21:34:41Z", "author_association": "OWNER", "pull_request": null, "body": "SQLite does not have ALTER TABLE support for adding new foreign keys... but it turns out it's possible to make these changes without having to duplicate the entire table by carefully running `UPDATE sqlite_master SET sql=... WHERE type='table' AND name='X';`\r\n\r\nHere's how Django does it: https://github.com/django/django/blob/d3449faaa915a08c275b35de01e66a7ef6bdb2dc/django/db/backends/sqlite3/schema.py#L103-L125\r\n\r\nAnd here's the official documentation about this: https://sqlite.org/lang_altertable.html#otheralter (scroll to the very bottom of the page)", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/2/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 351845423, "node_id": "MDU6SXNzdWUzNTE4NDU0MjM=", "number": 3, "title": "Experiment with contentless FTS tables", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2018-08-18T19:31:01Z", "updated_at": "2019-07-22T20:58:55Z", "closed_at": "2019-07-22T20:58:55Z", "author_association": "OWNER", "pull_request": null, "body": "Could greatly reduce size of resulting database for large datasets: http://cocoamine.net/blog/2015/09/07/contentless-fts4-for-large-immutable-documents/", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/3/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 403028630, "node_id": "MDExOlB1bGxSZXF1ZXN0MjQ3NTc2OTQy", "number": 4, "title": "Fts5", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2019-01-25T06:54:05Z", "updated_at": "2019-01-25T06:54:33Z", "closed_at": "2019-01-25T06:54:33Z", "author_association": "OWNER", "pull_request": "simonw/sqlite-utils/pulls/4", "body": "", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/4/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 403396009, "node_id": "MDExOlB1bGxSZXF1ZXN0MjQ3ODYxNDE5", "number": 5, "title": "Run Travis tests against Python 3.8-dev", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2019-01-26T02:30:55Z", "updated_at": "2019-01-26T02:37:54Z", "closed_at": "2019-01-26T02:37:54Z", "author_association": "OWNER", "pull_request": "simonw/sqlite-utils/pulls/5", "body": "", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/5/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 403624090, "node_id": "MDU6SXNzdWU0MDM2MjQwOTA=", "number": 6, "title": "\"sqlite-utils insert\" should support newline-delimited JSON", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2019-01-28T02:00:02Z", "updated_at": "2019-01-28T02:17:45Z", "closed_at": "2019-01-28T02:17:45Z", "author_association": "OWNER", "pull_request": null, "body": "We can already export newline delimited JSON. We should learn to import it as well.\r\n\r\nThe neat thing about importing it is that you can import GBs of data without having to read the whole lot into memory in order to decode the wrapping JSON array.\r\n\r\nDatasette can export it now: https://github.com/simonw/datasette/issues/405\r\n\r\nDemo: https://latest.datasette.io/fixtures/facetable.json?_shape=array&_nl=on\r\n\r\nIt should be possible to do this:\r\n\r\n    $ curl \"https://latest.datasette.io/fixtures/facetable.json?_shape=array&_nl=on\" \\\r\n        | sqlite-utils insert data.db facetable - --nl\r\n", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/6/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 403625674, "node_id": "MDU6SXNzdWU0MDM2MjU2NzQ=", "number": 7, "title": ".insert_all() should accept a generator and process it efficiently", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2019-01-28T02:11:58Z", "updated_at": "2019-01-28T06:26:53Z", "closed_at": "2019-01-28T06:26:53Z", "author_association": "OWNER", "pull_request": null, "body": "Right now you have to load every record into memory before passing the list to `.insert_all()` and friends.\r\n\r\nIf you want to process millions of rows, this is inefficient. Python has generators - we should use them!\r\n\r\nThe only catch here is that part of the magic of `sqlite-utils` is that it guesses the column types and creates the table for you. This code will need to be updated to notice if the table needs creating and, if it does, create it using the first X (where x=1,000 but can be customized) records.\r\n\r\nIf a record outside of those first 1,000 has a rogue column, we can crash with an error.\r\n\r\nThis will free us up to make the `--nl` option added in #6 much more efficient.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/7/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 413740684, "node_id": "MDU6SXNzdWU0MTM3NDA2ODQ=", "number": 11, "title": "Detect numpy types when creating tables", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2019-02-23T21:09:35Z", "updated_at": "2019-02-24T04:02:20Z", "closed_at": "2019-02-24T04:02:20Z", "author_association": "OWNER", "pull_request": null, "body": "Inspired by #8", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/11/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 413778585, "node_id": "MDExOlB1bGxSZXF1ZXN0MjU1NjU4MTEy", "number": 12, "title": "Support for numpy types, closes #11", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2019-02-24T03:57:32Z", "updated_at": "2019-02-24T04:02:20Z", "closed_at": "2019-02-24T04:02:20Z", "author_association": "OWNER", "pull_request": "simonw/sqlite-utils/pulls/12", "body": "", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/12/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 413779210, "node_id": "MDU6SXNzdWU0MTM3NzkyMTA=", "number": 13, "title": "Ability to automatically create IDs from content hash of row", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2019-02-24T04:07:08Z", "updated_at": "2019-02-24T04:36:48Z", "closed_at": "2019-02-24T04:36:48Z", "author_association": "OWNER", "pull_request": null, "body": "Sometimes when you are importing data the underlying source provides records without IDs that can be uniquely identified by their contents.\r\n\r\nA utility mechanism for calculating a sha1 hash of the contents and using that as a unique ID would be useful.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/13/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 413842611, "node_id": "MDU6SXNzdWU0MTM4NDI2MTE=", "number": 14, "title": "Utilities for adding indexes", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2019-02-24T16:57:28Z", "updated_at": "2019-02-24T19:11:28Z", "closed_at": "2019-02-24T19:11:28Z", "author_association": "OWNER", "pull_request": null, "body": "Both in the Python API and the CLI tool. For the CLI tool this should work:\r\n\r\n    $ sqlite-utils create-index mydb.db mytable col1 col2\r\n\r\nThis will create a compound index across col1 and col2. The name of the index will be automatically chosen unless you use the `--name=...` option.\r\n\r\nSupport a `--unique` option too.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/14/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 413857257, "node_id": "MDU6SXNzdWU0MTM4NTcyNTc=", "number": 15, "title": "Ability to add columns to tables", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2019-02-24T19:20:51Z", "updated_at": "2019-02-24T20:04:40Z", "closed_at": "2019-02-24T20:04:40Z", "author_association": "OWNER", "pull_request": null, "body": "Makes sense to do this before foreign keys in #2\r\n\r\nPython:\r\n\r\n    db[\"table\"].add_column(\"new_column\", int)\r\n\r\nCLI:\r\n\r\n    $ sqlite-utils add-column table new_column INTEGER\r\n", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/15/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 413867537, "node_id": "MDU6SXNzdWU0MTM4Njc1Mzc=", "number": 16, "title": "add_column() should support REFERENCES {other_table}({other_column})", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2019-02-24T21:00:45Z", "updated_at": "2019-05-29T05:17:59Z", "closed_at": "2019-05-29T04:56:18Z", "author_association": "OWNER", "pull_request": null, "body": "Related to #2 ", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/16/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 413868452, "node_id": "MDU6SXNzdWU0MTM4Njg0NTI=", "number": 17, "title": "Improve and document foreign_keys=... argument to insert/create/etc", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 7, "created_at": "2019-02-24T21:09:11Z", "updated_at": "2019-02-24T23:45:48Z", "closed_at": "2019-02-24T23:45:48Z", "author_association": "OWNER", "pull_request": null, "body": "The `foreign_keys=` argument to `table.insert_all()` and friends can be used to specify foreign key relationships that should be created.\r\n\r\nIt is not yet documented. It also requires you to specify the SQLite type of each column, even though this can be detected by introspecting the referenced table:\r\n\r\n    cols = [c for c in self.db[other_table].columns if c.name == other_column]\r\n    cols[0].type\r\n\r\nRelates to #2 ", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/17/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 413871266, "node_id": "MDU6SXNzdWU0MTM4NzEyNjY=", "number": 18, "title": ".insert/.upsert/.insert_all/.upsert_all should add missing columns", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 4348046, "label": "1.0"}, "comments": 2, "created_at": "2019-02-24T21:36:11Z", "updated_at": "2019-05-25T00:42:11Z", "closed_at": "2019-05-25T00:42:11Z", "author_association": "OWNER", "pull_request": null, "body": "This is a larger change, but it would be incredibly useful: if you attempt to insert or update a document with a field that does not currently exist in the underlying table, sqlite-utils should add the appropriate column for you.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/18/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 432217625, "node_id": "MDU6SXNzdWU0MzIyMTc2MjU=", "number": 19, "title": "Incorrect help text for enable-fts command", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 4348046, "label": "1.0"}, "comments": 0, "created_at": "2019-04-11T19:46:44Z", "updated_at": "2019-05-25T00:44:31Z", "closed_at": "2019-05-25T00:44:31Z", "author_association": "OWNER", "pull_request": null, "body": "I clearly copied-and-pasted this from the `tables` command without updating it:\r\n\r\nhttps://github.com/simonw/sqlite-utils/blob/0b1af42ead3b3902347951180b3364ce1942da6e/sqlite_utils/cli.py#L216-L222", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/19/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 448391492, "node_id": "MDU6SXNzdWU0NDgzOTE0OTI=", "number": 21, "title": "Option to ignore inserts if primary key exists already", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2019-05-25T00:17:12Z", "updated_at": "2019-05-29T05:09:01Z", "closed_at": "2019-05-29T04:18:26Z", "author_association": "OWNER", "pull_request": null, "body": "> I've just noticed that SQLite lets you IGNORE inserts that collide with a pre-existing key. This can be quite handy if you have a dataset that keeps changing in part, and you don't want to upsert and replace pre-existing PK rows but you do want to ignore collisions to existing PK rows.\r\n> \r\n> Do `sqlite_utils` support such (cavalier!) behaviour?\r\n\r\n_Originally posted by @psychemedia in https://github.com/simonw/sqlite-utils/issues/18#issuecomment-480621924_", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/21/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 448395665, "node_id": "MDU6SXNzdWU0NDgzOTU2NjU=", "number": 22, "title": "Release notes for 1.0", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 4348046, "label": "1.0"}, "comments": 2, "created_at": "2019-05-25T00:58:03Z", "updated_at": "2019-05-25T01:18:27Z", "closed_at": "2019-05-25T01:06:52Z", "author_association": "OWNER", "pull_request": null, "body": "https://github.com/simonw/sqlite-utils/compare/0.14...251e473", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/22/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 449565204, "node_id": "MDU6SXNzdWU0NDk1NjUyMDQ=", "number": 23, "title": "Syntactic sugar for creating m2m records", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 10, "created_at": "2019-05-29T02:17:48Z", "updated_at": "2019-08-04T03:54:58Z", "closed_at": "2019-08-04T03:37:34Z", "author_association": "OWNER", "pull_request": null, "body": "Python library only. What would be a syntactically pleasant way of creating a m2m record?", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/23/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 449848803, "node_id": "MDU6SXNzdWU0NDk4NDg4MDM=", "number": 25, "title": "Allow .insert(..., foreign_keys=()) to auto-detect table and primary key", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2019-05-29T14:39:22Z", "updated_at": "2019-06-13T05:32:32Z", "closed_at": "2019-06-13T05:32:32Z", "author_association": "OWNER", "pull_request": null, "body": "The `foreign_keys=` argument currently takes a list of triples:\r\n```python\r\ndb[\"usages\"].insert_all(\r\n    usages_to_insert,\r\n    foreign_keys=(\r\n        (\"line_id\", \"lines\", \"id\"),\r\n        (\"definition_id\", \"definitions\", \"id\"),\r\n    ),\r\n)\r\n```\r\nAs of #16 we have a mechanism for detecting the primary key column (the third item in this triple) - we should use that here too, so foreign keys can be optionally defined as a list of pairs.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/25/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 455486286, "node_id": "MDU6SXNzdWU0NTU0ODYyODY=", "number": 26, "title": "Mechanism for turning nested JSON into foreign keys / many-to-many", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 14, "created_at": "2019-06-13T00:52:06Z", "updated_at": "2022-06-29T23:35:29Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "The GitHub JSON APIs have a really interesting convention with respect to related objects.\r\n\r\nConsider https://api.github.com/repos/simonw/sqlite-utils/issues - here's a truncated subset:\r\n```json\r\n  {\r\n    \"id\": 449818897,\r\n    \"node_id\": \"MDU6SXNzdWU0NDk4MTg4OTc=\",\r\n    \"number\": 24,\r\n    \"title\": \"Additional Column Constraints?\",\r\n    \"user\": {\r\n      \"login\": \"IgnoredAmbience\",\r\n      \"id\": 98555,\r\n      \"node_id\": \"MDQ6VXNlcjk4NTU1\",\r\n      \"avatar_url\": \"https://avatars0.githubusercontent.com/u/98555?v=4\",\r\n      \"gravatar_id\": \"\"\r\n    },\r\n    \"labels\": [\r\n      {\r\n        \"id\": 993377884,\r\n        \"node_id\": \"MDU6TGFiZWw5OTMzNzc4ODQ=\",\r\n        \"url\": \"https://api.github.com/repos/simonw/sqlite-utils/labels/enhancement\",\r\n        \"name\": \"enhancement\",\r\n        \"color\": \"a2eeef\",\r\n        \"default\": true\r\n      }\r\n    ],\r\n    \"state\": \"open\"\r\n  }\r\n```\r\nThe `user` column lists a complete user. The `labels` column has a list of labels.\r\n\r\nSince both user and label have populated `id` field this is actually enough information for us to create records for them AND set up the corresponding foreign key (for user) and m2m relationships (for labels).\r\n\r\nIt would be really neat if `sqlite-utils` had some kind of mechanism for correctly processing these kind of patterns.\r\n\r\nThanks to `jq` there's not much need for extra customization of the shape here - if we support a narrowly defined structure users can use `jq` to reshape arbitrary JSON to match.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/26/reactions\", \"total_count\": 4, \"+1\": 4, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 455496504, "node_id": "MDU6SXNzdWU0NTU0OTY1MDQ=", "number": 27, "title": "sqlite-utils create-table command", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 8, "created_at": "2019-06-13T01:43:30Z", "updated_at": "2020-05-03T15:26:15Z", "closed_at": "2020-05-03T15:26:15Z", "author_association": "OWNER", "pull_request": null, "body": "Spun off from #24 - it would be useful if CLI users could create new tables (with explicit column types, not null rules and defaults) without having to insert an example record.\r\n\r\n- [x] Get it working\r\n- [x] Support `--pk`\r\n- [x] Support `--not-null`\r\n- [x] Support `--default`\r\n- [x] Support `--fk colname othertable othercol`\r\n- [x] Support `--replace` and `--ignore`\r\n- [x] Documentation", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/27/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 455996809, "node_id": "MDU6SXNzdWU0NTU5OTY4MDk=", "number": 28, "title": "Rearrange the docs by area, not CLI vs Python", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2019-06-13T23:33:35Z", "updated_at": "2019-07-15T02:37:20Z", "closed_at": "2019-07-15T02:37:20Z", "author_association": "OWNER", "pull_request": null, "body": "The docs for eg inserting data should live on the same page, rather than being split across the API and CLI pages.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/28/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 458941203, "node_id": "MDU6SXNzdWU0NTg5NDEyMDM=", "number": 29, "title": "Prevent accidental add-foreign-key with invalid column", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2019-06-20T23:57:24Z", "updated_at": "2019-06-20T23:58:26Z", "closed_at": "2019-06-20T23:58:26Z", "author_association": "OWNER", "pull_request": null, "body": "You can corrupt your database by running:\r\n\r\n    $ sqlite-utils add-foreign-key my.db table non_existent_column other_table other_column\r\n", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/29/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 461215118, "node_id": "MDU6SXNzdWU0NjEyMTUxMTg=", "number": 30, "title": "Option to open database in read-only mode", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2019-06-26T22:50:38Z", "updated_at": "2020-05-11T19:17:17Z", "closed_at": "2020-05-11T19:17:17Z", "author_association": "OWNER", "pull_request": null, "body": "Would this make it 100% safe to run reads against a database file that is being written to by another process?", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/30/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 461237618, "node_id": "MDU6SXNzdWU0NjEyMzc2MTg=", "number": 31, "title": "Mechanism for adding multiple foreign key constraints at once", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2019-06-27T00:04:30Z", "updated_at": "2019-06-29T06:27:40Z", "closed_at": "2019-06-29T06:27:40Z", "author_association": "OWNER", "pull_request": null, "body": "Needed by [db-to-sqlite](https://github.com/simonw/db-to-sqlite). It currently works by collecting all of the foreign key relationships it can find and then applying them at the end of the process.\r\n\r\nThe problem is, the `add_foreign_key()` method looks like this:\r\n\r\nhttps://github.com/simonw/sqlite-utils/blob/86bd2bba689e25f09551d611ccfbee1e069e5b66/sqlite_utils/db.py#L498-L516\r\n\r\nThat means it's doing a full `VACUUM` for every single relationship it sets up - and if you have hundreds of foreign key relationships in your database this can take hours.\r\n\r\nI think the right solution is to have a `.add_foreign_keys(list_of_args)` method which does the bulk operation and then a single `VACUUM`. `.add_foreign_key(...)` can then call the bulk action with a single list item.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/31/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 462094937, "node_id": "MDExOlB1bGxSZXF1ZXN0MjkyODc5MjA0", "number": 32, "title": "db.add_foreign_keys() method", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2019-06-28T15:40:33Z", "updated_at": "2019-06-29T06:27:39Z", "closed_at": "2019-06-29T06:27:39Z", "author_association": "OWNER", "pull_request": "simonw/sqlite-utils/pulls/32", "body": "Refs #31. Still TODO:\r\n\r\n- [x] Unit tests\r\n- [x] Documentation", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/32/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 462423839, "node_id": "MDU6SXNzdWU0NjI0MjM4Mzk=", "number": 33, "title": "index_foreign_keys / index-foreign-keys utilities", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2019-06-30T16:42:03Z", "updated_at": "2019-06-30T23:54:11Z", "closed_at": "2019-06-30T23:50:55Z", "author_association": "OWNER", "pull_request": null, "body": "Sometimes it's good to have indices on all columns that are foreign keys, to allow for efficient reverse lookups.\r\n\r\nThis would be a useful utility:\r\n\r\n    $ sqlite-utils index-foreign-keys database.db\r\n", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/33/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 462423972, "node_id": "MDExOlB1bGxSZXF1ZXN0MjkzMTE3MTgz", "number": 34, "title": "sqlite-utils index-foreign-keys / db.index_foreign_keys()", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2019-06-30T16:43:40Z", "updated_at": "2019-06-30T23:50:55Z", "closed_at": "2019-06-30T23:50:55Z", "author_association": "OWNER", "pull_request": "simonw/sqlite-utils/pulls/34", "body": "Refs #33\r\n\r\n- [x] `sqlite-utils index-foreign-keys` command\r\n- [x] `db.index_foreign_keys()` method\r\n- [x] unit tests\r\n- [x] documentation", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/34/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 462430920, "node_id": "MDU6SXNzdWU0NjI0MzA5MjA=", "number": 35, "title": "table.update(...) method", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2019-06-30T18:06:15Z", "updated_at": "2019-07-28T15:43:52Z", "closed_at": "2019-07-28T15:43:52Z", "author_association": "OWNER", "pull_request": null, "body": "Spun off from #23 - this method will allow a user to update a specific row.\r\n\r\nCurrently the only way to do that it is to call `.upsert({full record})` with the primary key field matching an existing record - but this does not support partial updates.\r\n```python\r\ndb[\"events\"].update(3, {\"name\": \"Renamed\"})\r\n```\r\nThis method only works on an existing table, so there's no need for a `pk=\"id\"` specifier - it can detect the primary key by looking at the table.\r\n\r\nIf the primary key is compound the first argument can be a tuple:\r\n```python\r\ndb[\"events_venues\"].update((3, 2), {\"custom_label\": \"Label\"})\r\n```\r\nThe method can be called without the second dictionary argument. Doing this selects the row specified by the primary key (throwing an error if it does not exist) and remembers it so that chained operations can be carried out - see proposal in https://github.com/simonw/sqlite-utils/issues/23#issuecomment-507055345\r\n", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/35/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 462817589, "node_id": "MDU6SXNzdWU0NjI4MTc1ODk=", "number": 36, "title": "Support compound primary keys", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2019-07-01T17:00:07Z", "updated_at": "2019-07-15T04:28:52Z", "closed_at": "2019-07-15T04:28:52Z", "author_association": "OWNER", "pull_request": null, "body": "This should work:\r\n```python\r\ntable = db[\"dog_breeds\"].insert({\r\n    \"dog_id\": 1,\r\n    \"breed_id\": 2\r\n}, pk=(\"dog_id\", \"breed_id\"))\r\n```\r\nNeeded for m2m work in #23", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/36/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 465815372, "node_id": "MDU6SXNzdWU0NjU4MTUzNzI=", "number": 37, "title": "Experiment with type hints", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2019-07-09T14:30:34Z", "updated_at": "2021-08-18T21:48:57Z", "closed_at": "2021-08-18T21:48:57Z", "author_association": "OWNER", "pull_request": null, "body": "Since it's designed to be used in Jupyter or for rapid prototyping in an IDE (and it's still pretty small) `sqlite-utils` feels like a great candidate for me to finally try out Python type hints.\r\n\r\nhttps://veekaybee.github.io/2019/07/08/python-type-hints/ is good.\r\n\r\nIt suggests the mypy docs for getting started: https://mypy.readthedocs.io/en/latest/existing_code.html plus this tutorial: https://pymbook.readthedocs.io/en/latest/typehinting.html", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/37/reactions\", \"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 467862459, "node_id": "MDExOlB1bGxSZXF1ZXN0Mjk3NDEyNDY0", "number": 38, "title": "table.update() method", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2019-07-14T17:03:49Z", "updated_at": "2019-07-28T15:43:51Z", "closed_at": "2019-07-28T15:43:51Z", "author_association": "OWNER", "pull_request": "simonw/sqlite-utils/pulls/38", "body": "Refs #35\r\n\r\nStill to do:\r\n\r\n- [x] Unit tests\r\n- [x] Switch to using `.get()`\r\n- [x] Better exceptions, plus unit tests for what happens if pk does not exist\r\n- [x] Documentation\r\n- [x] Ensure compound primary keys work properly\r\n- [x] `alter=True` support", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/38/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 467864071, "node_id": "MDU6SXNzdWU0Njc4NjQwNzE=", "number": 39, "title": "table.get(...) method", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2019-07-14T17:20:51Z", "updated_at": "2019-07-15T04:28:53Z", "closed_at": "2019-07-15T04:28:53Z", "author_association": "OWNER", "pull_request": null, "body": "Utility method for fetching a record by its primary key.\r\n\r\nAccepts a single value (for primary key / rowid tables) or a list/tuple of values (for compound primary keys, refs #36).\r\n\r\nRaises a `NotFoundError` if the record cannot be found.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/39/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 467928674, "node_id": "MDExOlB1bGxSZXF1ZXN0Mjk3NDU5Nzk3", "number": 40, "title": ".get() method plus support for compound primary keys", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2019-07-15T03:43:13Z", "updated_at": "2019-07-15T04:28:57Z", "closed_at": "2019-07-15T04:28:52Z", "author_association": "OWNER", "pull_request": "simonw/sqlite-utils/pulls/40", "body": "- [x] Tests for the `NotFoundError` exception\r\n- [x] Documentation for `.get()` method\r\n- [x] Support `--pk` multiple times to define CLI compound primary keys\r\n- [x] Documentation for compound primary keys", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/40/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 470131537, "node_id": "MDU6SXNzdWU0NzAxMzE1Mzc=", "number": 41, "title": "sqlite-utils insert --tsv option", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2019-07-19T04:27:21Z", "updated_at": "2019-07-19T04:50:47Z", "closed_at": "2019-07-19T04:50:47Z", "author_association": "OWNER", "pull_request": null, "body": "Right now we only support ingesting CSV, but sometimes interesting data is released as TSV.\r\n\r\nhttps://www.washingtonpost.com/national/2019/07/18/how-download-use-dea-pain-pills-database/ for example.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/41/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 470345929, "node_id": "MDU6SXNzdWU0NzAzNDU5Mjk=", "number": 42, "title": "table.extract(...) method and \"sqlite-utils extract\" command", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 5897911, "label": "2.20"}, "comments": 21, "created_at": "2019-07-19T14:09:36Z", "updated_at": "2020-09-22T23:39:31Z", "closed_at": "2020-09-22T23:37:49Z", "author_association": "OWNER", "pull_request": null, "body": "One of my favourite features of [csvs-to-sqlite](https://github.com/simonw/csvs-to-sqlite) is that it can \"extract\" columns into a separate lookup table - for example:\r\n\r\n    csvs-to-sqlite big_csv_file.csv -c country output.db\r\n\r\nThis will turn the `country` column in the resulting table into a integer foreign key against a new `country` table. You can see an example of what that looks like here: https://san-francisco.datasettes.com/registered-business-locations-3d50679/Business+Corridor was extracted from https://san-francisco.datasettes.com/registered-business-locations-3d50679/Registered_Business_Locations_-_San_Francisco?Business%20Corridor=1\r\n\r\nI'd like to have the same capability in `sqlite-utils` - but with the ability to run it against an existing SQLite table rather than just against a CSV.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/42/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 470691999, "node_id": "MDU6SXNzdWU0NzA2OTE5OTk=", "number": 43, "title": ".add_column() doesn't match indentation of initial creation", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2019-07-20T16:33:10Z", "updated_at": "2019-07-23T13:09:11Z", "closed_at": "2019-07-23T13:09:05Z", "author_association": "OWNER", "pull_request": null, "body": "I spotted a table which was created once and then had columns added to it and the formatted SQL looks like this:\r\n\r\n```sql\r\nCREATE TABLE [records] (\r\n   [type] TEXT,\r\n   [sourceName] TEXT,\r\n   [sourceVersion] TEXT,\r\n   [unit] TEXT,\r\n   [creationDate] TEXT,\r\n   [startDate] TEXT,\r\n   [endDate] TEXT,\r\n   [value] TEXT,\r\n   [metadata_Health Mate App Version] TEXT,\r\n   [metadata_Withings User Identifier] TEXT,\r\n   [metadata_Modified Date] TEXT,\r\n   [metadata_Withings Link] TEXT,\r\n   [metadata_HKWasUserEntered] TEXT\r\n, [device] TEXT, [metadata_HKMetadataKeyHeartRateMotionContext] TEXT, [metadata_HKDeviceManufacturerName] TEXT, [metadata_HKMetadataKeySyncVersion] TEXT, [metadata_HKMetadataKeySyncIdentifier] TEXT, [metadata_HKSwimmingStrokeStyle] TEXT, [metadata_HKVO2MaxTestType] TEXT, [metadata_HKTimeZone] TEXT, [metadata_Average HR] TEXT, [metadata_Recharge] TEXT, [metadata_Lights] TEXT, [metadata_Asleep] TEXT, [metadata_Rating] TEXT, [metadata_Energy Threshold] TEXT, [metadata_Deep Sleep] TEXT, [metadata_Nap] TEXT, [metadata_Edit Slots] TEXT, [metadata_Tags] TEXT, [metadata_Daytime HR] TEXT)\r\n```\r\n\r\nIt would be nice if the columns that were added later matched the indentation of the initial columns.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/43/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 471628483, "node_id": "MDU6SXNzdWU0NzE2Mjg0ODM=", "number": 44, "title": "Utilities for building lookup tables", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2019-07-23T10:59:58Z", "updated_at": "2019-07-23T13:07:01Z", "closed_at": "2019-07-23T13:07:01Z", "author_association": "OWNER", "pull_request": null, "body": "While building https://github.com/dogsheep/healthkit-to-sqlite I found a need for a neat mechanism for easily building lookup tables - tables where each unique value in a column is replaced by a foreign key to a separate table.\r\n\r\ncsvs-to-sqlite currently creates those with its \"extract\" mechanism - but that's written as custom code against Pandas. I'd like to eventually replace Pandas with sqlite-utils there.\r\n\r\nSee also #42 ", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/44/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 471684708, "node_id": "MDExOlB1bGxSZXF1ZXN0MzAwMjg2NTM1", "number": 45, "title": "Implemented table.lookup(...), closes #44", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2019-07-23T13:03:30Z", "updated_at": "2019-07-23T13:07:00Z", "closed_at": "2019-07-23T13:07:00Z", "author_association": "OWNER", "pull_request": "simonw/sqlite-utils/pulls/45", "body": "", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/45/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 471780443, "node_id": "MDU6SXNzdWU0NzE3ODA0NDM=", "number": 46, "title": "extracts= option for insert/update/etc", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2019-07-23T15:55:46Z", "updated_at": "2020-03-01T16:53:40Z", "closed_at": "2019-07-23T17:00:44Z", "author_association": "OWNER", "pull_request": null, "body": "Relates to #42 and #44. I want the ability to extract values out into lookup tables during bulk insert/upsert operations.\r\n\r\n`db.insert_all(rows, extracts=[\"species\"])`\r\n\r\n- creates species table for values in the species column\r\n\r\n`db.insert_all(rows, extracts={\"species\": \"Species\"})`\r\n\r\n- as above but the new table is called `Species`.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/46/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 471797101, "node_id": "MDExOlB1bGxSZXF1ZXN0MzAwMzc3NTk5", "number": 47, "title": "extracts= table parameter", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2019-07-23T16:30:29Z", "updated_at": "2019-07-23T17:00:43Z", "closed_at": "2019-07-23T17:00:43Z", "author_association": "OWNER", "pull_request": "simonw/sqlite-utils/pulls/47", "body": "Still needs docs. Refs #46", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/47/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 471818939, "node_id": "MDU6SXNzdWU0NzE4MTg5Mzk=", "number": 48, "title": "Jupyter notebook demo of the library, launchable on Binder", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2019-07-23T17:05:05Z", "updated_at": "2022-01-26T02:08:46Z", "closed_at": "2022-01-26T02:08:39Z", "author_association": "OWNER", "pull_request": null, "body": "", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/48/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 472115381, "node_id": "MDU6SXNzdWU0NzIxMTUzODE=", "number": 49, "title": "extracts= should support multiple-column extracts", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 10, "created_at": "2019-07-24T07:06:41Z", "updated_at": "2020-10-16T19:18:19Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "Lookup tables can be constructed on compound columns, but the `extracts=` option doesn't currently support that.\r\n\r\nRight now extracts can be defined in two ways:\r\n```python\r\n# Extract these columns into tables with the same name:\r\ndogs = db.table(\"dogs\", extracts=[\"breed\", \"most_recent_trophy\"])\r\n\r\n# Same as above but with custom table names:\r\ndogs = db.table(\"dogs\", extracts={\"breed\": \"Breeds\", \"most_recent_trophy\": \"Trophies\"})\r\n```\r\nNeed some kind of syntax for much more complicated extractions, like when two columns (say \"source\" and \"source_version\") are extracted into a single table.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/49/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 473083260, "node_id": "MDU6SXNzdWU0NzMwODMyNjA=", "number": 50, "title": "\"Too many SQL variables\" on large inserts", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2019-07-25T21:43:31Z", "updated_at": "2022-11-04T14:38:36Z", "closed_at": "2019-07-28T11:59:33Z", "author_association": "OWNER", "pull_request": null, "body": "Reported here: https://github.com/dogsheep/healthkit-to-sqlite/issues/9\r\n\r\nIt looks like there's a default limit of 999 variables - we need to be smart about that, maybe dynamically lower the batch size based on the number of columns.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/50/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 473733752, "node_id": "MDExOlB1bGxSZXF1ZXN0MzAxODI0MDk3", "number": 51, "title": "Fix for too many SQL variables, closes #50", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2019-07-28T11:30:30Z", "updated_at": "2019-07-28T11:59:32Z", "closed_at": "2019-07-28T11:59:32Z", "author_association": "OWNER", "pull_request": "simonw/sqlite-utils/pulls/51", "body": "", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/51/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 476413293, "node_id": "MDU6SXNzdWU0NzY0MTMyOTM=", "number": 52, "title": "Throws error if .insert_all() / .upsert_all() called with empty list", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2019-08-03T04:09:00Z", "updated_at": "2019-11-07T04:32:39Z", "closed_at": "2019-11-07T04:32:39Z", "author_association": "OWNER", "pull_request": null, "body": "See also https://github.com/simonw/db-to-sqlite/issues/18", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/52/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 476436920, "node_id": "MDExOlB1bGxSZXF1ZXN0MzAzOTkwNjgz", "number": 53, "title": "Work in progress: m2m() method for creating many-to-many records", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2019-08-03T10:03:56Z", "updated_at": "2019-08-04T03:38:10Z", "closed_at": "2019-08-04T03:37:33Z", "author_association": "OWNER", "pull_request": "simonw/sqlite-utils/pulls/53", "body": "- [x] `table.insert({\"name\": \"Barry\"}).m2m(\"tags\", lookup={\"tag\": \"Coworker\"})`\r\n- [x] Explicit table name `.m2m(\"humans\", ..., m2m_table=\"relationships\")`\r\n- [x] Automatically use an existing m2m table if a single obvious candidate exists (a table with two foreign keys in the correct directions)\r\n- [x] Require the explicit `m2m_table=` argument if multiple candidates for the m2m table exist\r\n- [x] Documentation\r\n\r\nRefs #23", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/53/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 481887482, "node_id": "MDExOlB1bGxSZXF1ZXN0MzA4MjkyNDQ3", "number": 55, "title": "Ability to introspect and run queries against views", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2019-08-17T13:40:56Z", "updated_at": "2019-08-23T12:19:42Z", "closed_at": "2019-08-23T12:19:42Z", "author_association": "OWNER", "pull_request": "simonw/sqlite-utils/pulls/55", "body": "See #54 ", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/55/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 488338965, "node_id": "MDU6SXNzdWU0ODgzMzg5NjU=", "number": 59, "title": "Ability to introspect triggers", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2019-09-02T23:47:16Z", "updated_at": "2019-09-03T01:52:36Z", "closed_at": "2019-09-03T00:09:42Z", "author_association": "OWNER", "pull_request": null, "body": "Now that we're creating triggers (thanks to @amjith in #57) it would be neat if we could introspect them too.\r\n\r\nI'm thinking:\r\n\r\n`db.triggers` - lists all triggers for the database\r\n`db[\"tablename\"].triggers` - lists triggers for that table\r\n\r\nThe underlying query for this is `select * from sqlite_master where type = 'trigger'`\r\n\r\nI'll return the trigger information in a new namedtuple, similar to how Indexes and ForeignKeys work.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/59/reactions\", \"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 488341021, "node_id": "MDExOlB1bGxSZXF1ZXN0MzEzMzgzMzE3", "number": 60, "title": "db.triggers and table.triggers introspection", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2019-09-03T00:04:32Z", "updated_at": "2019-09-03T00:09:42Z", "closed_at": "2019-09-03T00:09:42Z", "author_association": "OWNER", "pull_request": "simonw/sqlite-utils/pulls/60", "body": "Closes #59", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/60/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 517241040, "node_id": "MDU6SXNzdWU1MTcyNDEwNDA=", "number": 63, "title": "ensure_index() method", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2019-11-04T15:51:22Z", "updated_at": "2019-11-04T16:20:36Z", "closed_at": "2019-11-04T16:20:35Z", "author_association": "OWNER", "pull_request": null, "body": "```python\r\ndb[\"table\"].ensure_index([\"col1\", \"col2\"])\r\n```\r\n\r\nThis will do the following:\r\n- if the specified table or column does not exist, do nothing\r\n- if they exist and already have an index, do nothing\r\n- otherwise, create the index\r\n\r\nI want this for tools like [twitter-to-sqlite search](https://github.com/dogsheep/twitter-to-sqlite/blob/801c0c2daf17d8abce9dcb5d8d610410e7e25dbe/README.md#running-searches) where the `search_runs` table may or not have been created yet but, if it IS created, I want to put an index on the `hash` column.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/63/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 519032008, "node_id": "MDExOlB1bGxSZXF1ZXN0MzM3ODQ3NTcz", "number": 64, "title": "test_insert_upsert_all_empty_list", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2019-11-07T04:24:45Z", "updated_at": "2019-11-07T04:32:38Z", "closed_at": "2019-11-07T04:32:38Z", "author_association": "OWNER", "pull_request": "simonw/sqlite-utils/pulls/64", "body": "", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/64/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 519039316, "node_id": "MDExOlB1bGxSZXF1ZXN0MzM3ODUzMzk0", "number": 65, "title": "Release 1.12.1", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2019-11-07T04:51:29Z", "updated_at": "2019-11-07T04:58:48Z", "closed_at": "2019-11-07T04:58:47Z", "author_association": "OWNER", "pull_request": "simonw/sqlite-utils/pulls/65", "body": "", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/65/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 521868864, "node_id": "MDU6SXNzdWU1MjE4Njg4NjQ=", "number": 66, "title": "The \".upsert()\" method is misnamed", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 15, "created_at": "2019-11-12T23:48:28Z", "updated_at": "2019-12-31T01:30:21Z", "closed_at": "2019-12-31T01:30:20Z", "author_association": "OWNER", "pull_request": null, "body": "This thread here is illuminating: https://stackoverflow.com/questions/3634984/insert-if-not-exists-else-update\r\n\r\nThe term `UPSERT` in SQLite has a specific meaning as-of 3.24.0 (2018-06-04): https://www.sqlite.org/lang_UPSERT.html\r\n\r\nIt means \"behave as an UPDATE or a no-op if the INSERT would violate a uniqueness constraint\". The syntax in 3.24.0+ looks like this (confusingly it does not use the term \"upsert\"):\r\n```sql\r\nINSERT INTO phonebook(name,phonenumber) VALUES('Alice','704-555-1212')\r\n  ON CONFLICT(name) DO UPDATE SET phonenumber=excluded.phonenumber\r\n```\r\nHere's the problem: the `sqlite-utils` `.upsert()` and `.upsert_all()` methods don't do this. They use the following SQL:\r\n\r\n```sql\r\nINSERT OR REPLACE INTO [{table}] ({columns}) VALUES {rows};\r\n```\r\n\r\nIf the record already exists, it will be entirely replaced by a new record - as opposed to updating any specified fields but leaving existing fields as they are (the behaviour of \"upsert\" in SQLite itself).", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/66/reactions\", \"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 529376481, "node_id": "MDExOlB1bGxSZXF1ZXN0MzQ2MjY0OTI2", "number": 67, "title": "Run tests against 3.5 too", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2019-11-27T14:20:35Z", "updated_at": "2019-12-31T01:29:44Z", "closed_at": "2019-12-31T01:29:43Z", "author_association": "OWNER", "pull_request": "simonw/sqlite-utils/pulls/67", "body": "", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/67/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 531583658, "node_id": "MDU6SXNzdWU1MzE1ODM2NTg=", "number": 68, "title": "Add support for porter stemming in FTS", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2019-12-02T22:35:52Z", "updated_at": "2020-09-20T04:25:53Z", "closed_at": "2020-09-20T04:25:47Z", "author_association": "OWNER", "pull_request": null, "body": "FTS5 can have porter stemming enabled.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/68/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 542814756, "node_id": "MDU6SXNzdWU1NDI4MTQ3NTY=", "number": 71, "title": "Tests are failing due to missing FTS5", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2019-12-27T09:41:16Z", "updated_at": "2019-12-27T09:49:37Z", "closed_at": "2019-12-27T09:49:37Z", "author_association": "OWNER", "pull_request": null, "body": "https://travis-ci.com/simonw/sqlite-utils/jobs/268436167\r\n\r\nThis is a recent change: 2 months ago they worked fine.\r\n\r\nI'm not sure what changed here. Maybe something to do with https://launchpad.net/~jonathonf/+archive/ubuntu/backports ?", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/71/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 543738004, "node_id": "MDExOlB1bGxSZXF1ZXN0MzU3OTkyNTg4", "number": 72, "title": "Fixed implementation of upsert", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2019-12-30T05:08:05Z", "updated_at": "2019-12-30T05:29:24Z", "closed_at": "2019-12-30T05:29:24Z", "author_association": "OWNER", "pull_request": "simonw/sqlite-utils/pulls/72", "body": "Refs #66", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/72/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 557825032, "node_id": "MDU6SXNzdWU1NTc4MjUwMzI=", "number": 77, "title": "Ability to insert data that is transformed by a SQL function", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2020-01-30T23:45:55Z", "updated_at": "2022-02-05T00:04:25Z", "closed_at": "2020-01-31T00:24:32Z", "author_association": "OWNER", "pull_request": null, "body": "I want to be able to run the equivalent of this SQL insert:\r\n```python\r\n# Convert to \"Well Known Text\" format\r\nwkt = shape(geojson['geometry']).wkt\r\n# Insert and commit the record\r\nconn.execute(\"INSERT INTO places (id, name, geom) VALUES(null, ?, GeomFromText(?, 4326))\", (\r\n   \"Wales\", wkt\r\n))\r\nconn.commit()\r\n```\r\nFrom the Datasette SpatiaLite docs: https://datasette.readthedocs.io/en/stable/spatialite.html\r\n\r\nTo do this, I need a way of telling `sqlite-utils` that a specific column should be wrapped in `GeomFromText(?, 4326)`.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/77/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 557830332, "node_id": "MDExOlB1bGxSZXF1ZXN0MzY5MzQ4MDg0", "number": 78, "title": "New conversions= feature, refs #77", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2020-01-31T00:02:33Z", "updated_at": "2020-09-22T07:48:29Z", "closed_at": "2020-01-31T00:24:31Z", "author_association": "OWNER", "pull_request": "simonw/sqlite-utils/pulls/78", "body": "", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/78/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 557842245, "node_id": "MDU6SXNzdWU1NTc4NDIyNDU=", "number": 79, "title": "Helper methods for working with SpatiaLite", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 8, "created_at": "2020-01-31T00:39:19Z", "updated_at": "2022-02-05T00:04:25Z", "closed_at": "2022-02-04T05:55:11Z", "author_association": "OWNER", "pull_request": null, "body": "As demonstrated by this piece of documentation, using SpatiaLite with sqlite-utils requires a fair bit of boilerplate:\r\nhttps://github.com/simonw/sqlite-utils/blob/f7289174e66ae4d91d57de94bbd9d09fabf7aff4/docs/python-api.rst#L880-L909", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/79/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 557892819, "node_id": "MDExOlB1bGxSZXF1ZXN0MzY5Mzk0MDQz", "number": 80, "title": "on_create mechanism for after table creation", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 5, "created_at": "2020-01-31T03:38:48Z", "updated_at": "2020-01-31T05:08:04Z", "closed_at": "2020-01-31T05:08:04Z", "author_association": "OWNER", "pull_request": "simonw/sqlite-utils/pulls/80", "body": "I need this for `geojson-to-sqlite`, in particular https://github.com/simonw/geojson-to-sqlite/issues/6", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/80/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 558600274, "node_id": "MDU6SXNzdWU1NTg2MDAyNzQ=", "number": 81, "title": "Remove .detect_column_types() from table, make it a documented API", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2020-02-01T21:25:54Z", "updated_at": "2020-02-01T21:55:35Z", "closed_at": "2020-02-01T21:55:35Z", "author_association": "OWNER", "pull_request": null, "body": "I used it in `geojson-to-sqlite` here: https://github.com/simonw/geojson-to-sqlite/blob/f10e44264712dd59ae7dfa2e6fd5a904b682fb33/geojson_to_sqlite/utils.py#L45-L50\r\n\r\nIt would make more sense for this method to live on the Database rather than the Table - or even to exist as a separate utility method entirely.\r\n\r\nThen it should be documented.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/81/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 559374410, "node_id": "MDU6SXNzdWU1NTkzNzQ0MTA=", "number": 83, "title": "Make db[\"table\"].exists a documented API", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2020-02-03T22:31:44Z", "updated_at": "2020-02-08T23:58:35Z", "closed_at": "2020-02-08T23:56:23Z", "author_association": "OWNER", "pull_request": null, "body": "Right now it's a static thing which might get out-of-sync with the database. It should probably be a live check. Maybe call it `.exists()` instead?", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/83/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 561460274, "node_id": "MDU6SXNzdWU1NjE0NjAyNzQ=", "number": 84, "title": ".upsert() with hash_id throws error", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2020-02-07T07:08:19Z", "updated_at": "2020-02-07T07:17:11Z", "closed_at": "2020-02-07T07:17:11Z", "author_association": "OWNER", "pull_request": null, "body": "```python\r\ndb[table_name].upsert_all(rows, hash_id=\"pk\")\r\n```\r\nThis throws an error: `PrimaryKeyRequired('upsert() requires a pk')`\r\n\r\nThe problem is, if you try this:\r\n\r\n```python\r\ndb[table_name].upsert_all(rows, hash_id=\"pk\", pk=\"pk\")\r\n```\r\nYou get this error: `AssertionError('Use either pk= or hash_id=')`\r\n\r\n`hash_id=` should imply that `pk=` that column.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/84/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 562911863, "node_id": "MDU6SXNzdWU1NjI5MTE4NjM=", "number": 85, "title": "Create index doesn't work for columns containing spaces", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2020-02-11T00:34:46Z", "updated_at": "2020-02-11T05:13:20Z", "closed_at": "2020-02-11T05:13:20Z", "author_association": "OWNER", "pull_request": null, "body": "", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/85/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 565837965, "node_id": "MDU6SXNzdWU1NjU4Mzc5NjU=", "number": 87, "title": "Should detect collections.OrderedDict as a regular dictionary", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2020-02-16T02:06:34Z", "updated_at": "2020-02-16T02:20:59Z", "closed_at": "2020-02-16T02:20:59Z", "author_association": "OWNER", "pull_request": null, "body": "```\r\n  File \"...python3.7/site-packages/sqlite_utils/db.py\", line 292, in create_table\r\n    column_type=COLUMN_TYPE_MAPPING[column_type],\r\nKeyError: <class 'collections.OrderedDict'>\r\n```", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/87/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 571805300, "node_id": "MDU6SXNzdWU1NzE4MDUzMDA=", "number": 88, "title": "table.disable_fts() method and \"sqlite-utils disable-fts ...\" command", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 5, "created_at": "2020-02-27T04:00:50Z", "updated_at": "2020-02-27T04:40:44Z", "closed_at": "2020-02-27T04:40:44Z", "author_association": "OWNER", "pull_request": null, "body": "This would make it easier to iterate on the FTS configuration for a database without having to wipe and recreate the database each time.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/88/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 573578548, "node_id": "MDU6SXNzdWU1NzM1Nzg1NDg=", "number": 89, "title": "Ability to customize columns used by extracts= feature", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2020-03-01T16:54:48Z", "updated_at": "2020-10-16T19:17:50Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "@simonw any thoughts on allow extracts to specify the lookup column name? If I'm understanding the documentation right, `.lookup()` allows you to define the \"value\" column (the documentation uses name), but when you use `extracts` keyword as part of `.insert()`, `.upsert()` etc. the lookup must be done against a column named \"value\". I have an existing lookup table that I've populated with columns \"id\" and \"name\" as opposed to \"id\" and \"value\", and seems I can't use `extracts=`, unless I'm missing something...\r\n\r\nInitial thought on how to do this would be to allow the dictionary value to be a tuple of table name column pair... so:\r\n```\r\ntable = db.table(\"trees\", extracts={\"species_id\": (\"Species\", \"name\"})\r\n```\r\n\r\nI haven't dug too much into the existing code yet, but does this make sense? Worth doing?\r\n\r\n_Originally posted by @chrishas35 in https://github.com/simonw/sqlite-utils/issues/46#issuecomment-592999503_", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/89/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 573740712, "node_id": "MDU6SXNzdWU1NzM3NDA3MTI=", "number": 90, "title": "Cannot .enable_fts() for columns with spaces in their names", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2020-03-02T06:06:03Z", "updated_at": "2020-03-02T06:10:49Z", "closed_at": "2020-03-02T06:10:49Z", "author_association": "OWNER", "pull_request": null, "body": "```\r\nimport sqlite_utils\r\ndb = sqlite_utils.Database(memory=True)                                 \r\ndb[\"test\"].insert({\"space in name\": \"hello\"})                           \r\ndb[\"test\"].enable_fts([\"space in name\"])                                \r\n---------------------------------------------------------------------------\r\nOperationalError                          Traceback (most recent call last)\r\n<ipython-input-8-ce4b87dd1c7a> in <module>\r\n----> 1 db['test'].enable_fts([\"space in name\"])\r\n\r\n/usr/local/lib/python3.7/site-packages/sqlite_utils/db.py in enable_fts(self, columns, fts_version, create_triggers)\r\n    755         )\r\n    756         self.db.conn.executescript(sql)\r\n--> 757         self.populate_fts(columns)\r\n    758 \r\n    759         if create_triggers:\r\n\r\n/usr/local/lib/python3.7/site-packages/sqlite_utils/db.py in populate_fts(self, columns)\r\n    787             table=self.name, columns=\", \".join(columns)\r\n    788         )\r\n--> 789         self.db.conn.executescript(sql)\r\n    790         return self\r\n    791 \r\n\r\nOperationalError: near \"in\": syntax error\r\n```", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/90/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 581339961, "node_id": "MDU6SXNzdWU1ODEzMzk5NjE=", "number": 92, "title": ".columns_dict doesn't work for all possible column types", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 7, "created_at": "2020-03-14T19:30:35Z", "updated_at": "2020-03-15T18:37:43Z", "closed_at": "2020-03-14T20:04:14Z", "author_association": "OWNER", "pull_request": null, "body": "Got this error:\r\n```\r\n  File \".../python3.7/site-packages/sqlite_utils/db.py\", line 462, in <dictcomp>\r\n    for column in self.columns\r\nKeyError: 'REAL'\r\n```\r\n`.columns_dict` uses `REVERSE_COLUMN_TYPE_MAPPING`:\r\nhttps://github.com/simonw/sqlite-utils/blob/43f1c6ab4e3a6b76531fb6f5447adb83d26f3971/sqlite_utils/db.py#L457-L463\r\n`REVERSE_COLUMN_TYPE_MAPPING` defines `FLOAT` not `REAL`A\r\nhttps://github.com/simonw/sqlite-utils/blob/43f1c6ab4e3a6b76531fb6f5447adb83d26f3971/sqlite_utils/db.py#L68-L74", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/92/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 581795570, "node_id": "MDU6SXNzdWU1ODE3OTU1NzA=", "number": 93, "title": "Support more string values for types in .add_column()", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2020-03-15T19:32:49Z", "updated_at": "2020-09-24T20:36:46Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "https://sqlite-utils.readthedocs.io/en/2.4.2/python-api.html#adding-columns says:\r\n> SQLite types you can specify are \"TEXT\", \"INTEGER\", \"FLOAT\" or \"BLOB\".\r\n\r\nAs discovered in #92 this isn't the right list of values. I should expand this to match https://www.sqlite.org/datatype3.html", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/93/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 586477757, "node_id": "MDU6SXNzdWU1ODY0Nzc3NTc=", "number": 94, "title": "If column data is a mixture of integers and nulls, detected type should be INTEGER", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2020-03-23T19:51:46Z", "updated_at": "2020-03-23T19:57:10Z", "closed_at": "2020-03-23T19:57:10Z", "author_association": "OWNER", "pull_request": null, "body": "It looks like detected type for that case is TEXT at the moment.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/94/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 586486367, "node_id": "MDU6SXNzdWU1ODY0ODYzNjc=", "number": 95, "title": "Columns with only null values are no longer created in the database", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2020-03-23T20:07:42Z", "updated_at": "2020-03-23T20:31:15Z", "closed_at": "2020-03-23T20:31:15Z", "author_association": "OWNER", "pull_request": null, "body": "Bug introduced in #94, and released in `2.4.3`.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/95/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 597671518, "node_id": "MDU6SXNzdWU1OTc2NzE1MTg=", "number": 98, "title": "Only set .last_rowid and .last_pk for single update/inserts, not for .insert_all()/.upsert_all() with multiple records", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 7, "created_at": "2020-04-10T03:19:40Z", "updated_at": "2021-09-28T04:38:44Z", "closed_at": "2020-04-13T03:29:15Z", "author_association": "OWNER", "pull_request": null, "body": "", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/98/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 598640234, "node_id": "MDU6SXNzdWU1OTg2NDAyMzQ=", "number": 99, "title": ".upsert_all() should maybe error if dictionaries passed to it do not have the same keys", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2020-04-13T03:02:25Z", "updated_at": "2020-04-13T03:05:20Z", "closed_at": "2020-04-13T03:05:04Z", "author_association": "OWNER", "pull_request": null, "body": "While investigating #98 I stumbled across this:\r\n```\r\n    def test_upsert_compound_primary_key(fresh_db):\r\n        table = fresh_db[\"table\"]\r\n        table.upsert_all(\r\n            [\r\n                {\"species\": \"dog\", \"id\": 1, \"name\": \"Cleo\", \"age\": 4},\r\n                {\"species\": \"cat\", \"id\": 1, \"name\": \"Catbag\"},\r\n            ],\r\n            pk=(\"species\", \"id\"),\r\n        )\r\n        table.upsert_all(\r\n            [\r\n                {\"species\": \"dog\", \"id\": 1, \"age\": 5},\r\n                {\"species\": \"dog\", \"id\": 2, \"name\": \"New Dog\", \"age\": 1},\r\n            ],\r\n            pk=(\"species\", \"id\"),\r\n        )\r\n>       assert [\r\n            {\"species\": \"dog\", \"id\": 1, \"name\": \"Cleo\", \"age\": 5},\r\n            {\"species\": \"cat\", \"id\": 1, \"name\": \"Catbag\", \"age\": None},\r\n            {\"species\": \"dog\", \"id\": 2, \"name\": \"New Dog\", \"age\": 1},\r\n        ] == list(table.rows)\r\nE       AssertionError: assert [{'age': 5, '...cies': 'dog'}] == [{'age': 5, '...cies': 'dog'}]\r\nE         At index 0 diff: {'species': 'dog', 'id': 1, 'name': 'Cleo', 'age': 5} != {'species': 'dog', 'id': 1, 'name': None, 'age': 5}\r\nE         Full diff:\r\nE         - [{'age': 5, 'id': 1, 'name': 'Cleo', 'species': 'dog'},\r\nE         ?                              ^^^ --\r\nE         + [{'age': 5, 'id': 1, 'name': None, 'species': 'dog'},\r\nE         ?                              ^^^\r\nE         {'age': None, 'id': 1, 'name': 'Catbag', 'species': 'cat'},\r\nE         {'age': 1, 'id': 2, 'name': 'New Dog', 'species': 'dog'}]\r\n```\r\nIf you run `.upsert_all()` with multiple dictionaries it doesn't quite have the effect you might expect.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/99/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 601358649, "node_id": "MDU6SXNzdWU2MDEzNTg2NDk=", "number": 100, "title": "Mechanism for forcing column-type, over-riding auto-detection", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2020-04-16T19:12:52Z", "updated_at": "2020-04-17T23:53:32Z", "closed_at": "2020-04-17T23:53:32Z", "author_association": "OWNER", "pull_request": null, "body": "As seen in https://github.com/dogsheep/github-to-sqlite/issues/27#issuecomment-614843406 - there's a problem where you insert a record with a `None` value for a column and that column is created as `TEXT` - but actually you intended it to be an `INT` (as later examples will demonstrate).\r\n\r\nSome kind of mechanism for over-riding the detected types of columns would be useful here.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/100/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 601392318, "node_id": "MDU6SXNzdWU2MDEzOTIzMTg=", "number": 101, "title": "README should include an example of CLI data insertion", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2020-04-16T19:45:37Z", "updated_at": "2020-04-17T23:59:49Z", "closed_at": "2020-04-17T23:59:49Z", "author_association": "OWNER", "pull_request": null, "body": "Maybe using `curl` from the GitHub API.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/101/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 602569315, "node_id": "MDU6SXNzdWU2MDI1NjkzMTU=", "number": 102, "title": "Can't store an array or dictionary containing a bytes value", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2020-04-18T22:49:21Z", "updated_at": "2020-05-01T20:45:45Z", "closed_at": "2020-05-01T20:45:45Z", "author_association": "OWNER", "pull_request": null, "body": "```\r\nIn [1]: import sqlite_utils                                                     \r\n\r\nIn [2]: db = sqlite_utils.Database(memory=True)                                 \r\n\r\nIn [3]: db[\"t\"].insert({\"id\": 1, \"data\": {\"foo\": b\"bytes\"}})                    \r\n---------------------------------------------------------------------------\r\nTypeError                                 Traceback (most recent call last)\r\n<ipython-input-3-a8ab1f72c72c> in <module>\r\n----> 1 db[\"t\"].insert({\"id\": 1, \"data\": {\"foo\": b\"bytes\"}})\r\n\r\n~/Dropbox/Development/sqlite-utils/sqlite_utils/db.py in insert(self, record, pk, foreign_keys, column_order, not_null, defaults, hash_id, alter, ignore, replace, extracts, conversions, columns)\r\n    950             extracts=extracts,\r\n    951             conversions=conversions,\r\n--> 952             columns=columns,\r\n    953         )\r\n    954 \r\n\r\n~/Dropbox/Development/sqlite-utils/sqlite_utils/db.py in insert_all(self, records, pk, foreign_keys, column_order, not_null, defaults, batch_size, hash_id, alter, ignore, replace, extracts, conversions, columns, upsert)\r\n   1052                 for key in all_columns:\r\n   1053                     value = jsonify_if_needed(\r\n-> 1054                         record.get(key, None if key != hash_id else _hash(record))\r\n   1055                     )\r\n   1056                     if key in extracts:\r\n\r\n~/Dropbox/Development/sqlite-utils/sqlite_utils/db.py in jsonify_if_needed(value)\r\n   1318 def jsonify_if_needed(value):\r\n   1319     if isinstance(value, (dict, list, tuple)):\r\n-> 1320         return json.dumps(value)\r\n   1321     elif isinstance(value, (datetime.time, datetime.date, datetime.datetime)):\r\n   1322         return value.isoformat()\r\n\r\n/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/__init__.py in dumps(obj, skipkeys, ensure_ascii, check_circular, allow_nan, cls, indent, separators, default, sort_keys, **kw)\r\n    229         cls is None and indent is None and separators is None and\r\n    230         default is None and not sort_keys and not kw):\r\n--> 231         return _default_encoder.encode(obj)\r\n    232     if cls is None:\r\n    233         cls = JSONEncoder\r\n\r\n/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/encoder.py in encode(self, o)\r\n    197         # exceptions aren't as detailed.  The list call should be roughly\r\n    198         # equivalent to the PySequence_Fast that ''.join() would do.\r\n--> 199         chunks = self.iterencode(o, _one_shot=True)\r\n    200         if not isinstance(chunks, (list, tuple)):\r\n    201             chunks = list(chunks)\r\n\r\n/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/encoder.py in iterencode(self, o, _one_shot)\r\n    255                 self.key_separator, self.item_separator, self.sort_keys,\r\n    256                 self.skipkeys, _one_shot)\r\n--> 257         return _iterencode(o, 0)\r\n    258 \r\n    259 def _make_iterencode(markers, _default, _encoder, _indent, _floatstr,\r\n\r\n/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/encoder.py in default(self, o)\r\n    177 \r\n    178         \"\"\"\r\n--> 179         raise TypeError(f'Object of type {o.__class__.__name__} '\r\n    180                         f'is not JSON serializable')\r\n    181 \r\n\r\nTypeError: Object of type bytes is not JSON serializable\r\n```", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/102/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 610853393, "node_id": "MDU6SXNzdWU2MTA4NTMzOTM=", "number": 104, "title": "--schema option to \"sqlite-utils tables\"", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2020-05-01T16:55:49Z", "updated_at": "2020-05-01T17:12:37Z", "closed_at": "2020-05-01T17:12:37Z", "author_association": "OWNER", "pull_request": null, "body": "Adds output showing the table schema.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/104/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 610853576, "node_id": "MDU6SXNzdWU2MTA4NTM1NzY=", "number": 105, "title": "\"sqlite-utils views\" command", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2020-05-01T16:56:11Z", "updated_at": "2020-05-01T20:40:07Z", "closed_at": "2020-05-01T20:38:36Z", "author_association": "OWNER", "pull_request": null, "body": "Similar to `sqlite-utils tables`. See also #104.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/105/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 611216862, "node_id": "MDU6SXNzdWU2MTEyMTY4NjI=", "number": 106, "title": "create_view(..., ignore=True, replace=True) parameters", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2020-05-02T15:45:21Z", "updated_at": "2020-05-02T16:04:51Z", "closed_at": "2020-05-02T16:02:10Z", "author_association": "OWNER", "pull_request": null, "body": "Two new parameters which specify what should happen if the view already exists. I want this for https://github.com/dogsheep/github-to-sqlite/issues/37\r\n\r\nHere's the current `create_view()` implementation:\r\n\r\nhttps://github.com/simonw/sqlite-utils/blob/b4d953d3ccef28bb81cea40ca165a647b59971fa/sqlite_utils/db.py#L325-L332\r\n\r\n`ignore=True` will not do anything if the view exists already.\r\n\r\n`replace=True` will drop and redefine the view - but only if its SQL definition differs, otherwise it will be left alone.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/106/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 611222968, "node_id": "MDU6SXNzdWU2MTEyMjI5Njg=", "number": 107, "title": "sqlite-utils create-view CLI command", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2020-05-02T16:15:13Z", "updated_at": "2020-05-03T15:36:58Z", "closed_at": "2020-05-03T15:36:37Z", "author_association": "OWNER", "pull_request": null, "body": "Can go with #27 - `sqlite-utils create-table`.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/107/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 611326701, "node_id": "MDU6SXNzdWU2MTEzMjY3MDE=", "number": 108, "title": "Documentation unit tests for CLI commands", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2020-05-03T03:58:42Z", "updated_at": "2020-05-03T04:13:57Z", "closed_at": "2020-05-03T04:13:57Z", "author_association": "OWNER", "pull_request": null, "body": "Have a test that ensures all CLI commands are documented.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/108/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 612658444, "node_id": "MDU6SXNzdWU2MTI2NTg0NDQ=", "number": 109, "title": "table.create_index(..., ignore=True)", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2020-05-05T14:44:21Z", "updated_at": "2020-05-05T14:46:53Z", "closed_at": "2020-05-05T14:46:53Z", "author_association": "OWNER", "pull_request": null, "body": "Option to silently do nothing if the index already exists.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/109/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 615477131, "node_id": "MDU6SXNzdWU2MTU0NzcxMzE=", "number": 111, "title": "sqlite-utils drop-table and drop-view commands", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2020-05-10T21:10:42Z", "updated_at": "2020-05-11T01:58:36Z", "closed_at": "2020-05-11T00:44:26Z", "author_association": "OWNER", "pull_request": null, "body": "Would be useful to be able to drop views and tables from the CLI.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/111/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 616271236, "node_id": "MDU6SXNzdWU2MTYyNzEyMzY=", "number": 112, "title": "add_foreign_key(...., ignore=True)", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 5896742, "label": "2.19"}, "comments": 4, "created_at": "2020-05-12T00:24:00Z", "updated_at": "2020-09-20T22:17:34Z", "closed_at": "2020-09-20T22:17:34Z", "author_association": "OWNER", "pull_request": null, "body": "When using this library I often find myself wanting to \"add this foreign key, but only if it doesn't exist yet\". The `ignore=True` parameter is increasingly being used for this else where in the library (e.g. in `create_view()`).", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/112/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 621286870, "node_id": "MDU6SXNzdWU2MjEyODY4NzA=", "number": 113, "title": "Syntactic sugar for ATTACH DATABASE", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2020-05-19T21:10:00Z", "updated_at": "2021-02-19T05:09:12Z", "closed_at": "2021-02-19T04:56:36Z", "author_association": "OWNER", "pull_request": null, "body": "https://www.sqlite.org/lang_attach.html\r\n\r\nMaybe something like this:\r\n```python\r\ndb.attach(\"other_db\", \"other_db.db\")\r\n```", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/113/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 621989740, "node_id": "MDU6SXNzdWU2MjE5ODk3NDA=", "number": 114, "title": "table.transform() method for advanced alter table", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 5897911, "label": "2.20"}, "comments": 26, "created_at": "2020-05-20T18:20:46Z", "updated_at": "2020-09-22T07:51:37Z", "closed_at": "2020-09-22T04:20:02Z", "author_association": "OWNER", "pull_request": null, "body": "SQLite's `ALTER TABLE` can only do the following:\r\n\r\n* Rename a table\r\n* Rename a column\r\n* Add a column\r\n\r\nNotably, it cannot drop columns - so tricks like \"add a float version of this text column, populate it, then drop the old one and rename\" won't work.\r\n\r\nThe docs here https://www.sqlite.org/lang_altertable.html#making_other_kinds_of_table_schema_changes describe a way of implementing full alters safely within a transaction, but it's fiddly.\r\n\r\n1. Create new table\r\n2. Copy data\r\n3. Drop old table\r\n4. Rename new into old \r\n\r\nIt would be great if `sqlite-utils` provided an abstraction to help make these kinds of changes safely.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/114/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 637889964, "node_id": "MDU6SXNzdWU2Mzc4ODk5NjQ=", "number": 115, "title": "Ability to execute insert/update statements with the CLI", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2020-06-12T17:01:17Z", "updated_at": "2020-06-12T17:51:11Z", "closed_at": "2020-06-12T17:41:10Z", "author_association": "OWNER", "pull_request": null, "body": "```\r\n$ sqlite-utils github.db \"update stars set starred_at = ''\"\r\nTraceback (most recent call last):\r\n  File \"/Users/simon/.local/bin/sqlite-utils\", line 8, in <module>\r\n    sys.exit(cli())\r\n  File \"/Users/simon/.local/pipx/venvs/sqlite-utils/lib/python3.8/site-packages/click/core.py\", line 829, in __call__\r\n    return self.main(*args, **kwargs)\r\n  File \"/Users/simon/.local/pipx/venvs/sqlite-utils/lib/python3.8/site-packages/click/core.py\", line 782, in main\r\n    rv = self.invoke(ctx)\r\n  File \"/Users/simon/.local/pipx/venvs/sqlite-utils/lib/python3.8/site-packages/click/core.py\", line 1259, in invoke\r\n    return _process_result(sub_ctx.command.invoke(sub_ctx))\r\n  File \"/Users/simon/.local/pipx/venvs/sqlite-utils/lib/python3.8/site-packages/click/core.py\", line 1066, in invoke\r\n    return ctx.invoke(self.callback, **ctx.params)\r\n  File \"/Users/simon/.local/pipx/venvs/sqlite-utils/lib/python3.8/site-packages/click/core.py\", line 610, in invoke\r\n    return callback(*args, **kwargs)\r\n  File \"/Users/simon/.local/pipx/venvs/sqlite-utils/lib/python3.8/site-packages/sqlite_utils/cli.py\", line 673, in query\r\n    headers = [c[0] for c in cursor.description]\r\nTypeError: 'NoneType' object is not iterable\r\n```", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/115/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 644122661, "node_id": "MDU6SXNzdWU2NDQxMjI2NjE=", "number": 116, "title": "Documentation for table.pks introspection property", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2020-06-23T20:27:24Z", "updated_at": "2020-06-23T21:21:33Z", "closed_at": "2020-06-23T21:03:14Z", "author_association": "OWNER", "pull_request": null, "body": "https://github.com/simonw/sqlite-utils/blob/4d9a3204361d956440307a57bd18c829a15861db/sqlite_utils/db.py#L535-L540", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/116/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 644161221, "node_id": "MDU6SXNzdWU2NDQxNjEyMjE=", "number": 117, "title": "Support for compound (composite) foreign keys", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2020-06-23T21:33:42Z", "updated_at": "2020-06-23T21:40:31Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "It turns out SQLite supports composite foreign keys: https://www.sqlite.org/foreignkeys.html#fk_composite\r\n\r\nTheir example looks like this:\r\n```sql\r\nCREATE TABLE album(\r\n  albumartist TEXT,\r\n  albumname TEXT,\r\n  albumcover BINARY,\r\n  PRIMARY KEY(albumartist, albumname)\r\n);\r\n\r\nCREATE TABLE song(\r\n  songid     INTEGER,\r\n  songartist TEXT,\r\n  songalbum TEXT,\r\n  songname   TEXT,\r\n  FOREIGN KEY(songartist, songalbum) REFERENCES album(albumartist, albumname)\r\n);\r\n```\r\n\r\nHere's what that looks like in sqlite-utils:\r\n\r\n```\r\nIn [1]: import sqlite_utils                                                                                                                \r\n\r\nIn [2]: import sqlite3                                                                                                                     \r\n\r\nIn [3]: conn = sqlite3.connect(\":memory:\")                                                                                                 \r\n\r\nIn [4]: conn                                                                                                                               \r\nOut[4]: <sqlite3.Connection at 0x1087186c0>\r\n\r\nIn [5]: conn.executescript(\"\"\" \r\n   ...: CREATE TABLE album( \r\n   ...:   albumartist TEXT, \r\n   ...:   albumname TEXT, \r\n   ...:   albumcover BINARY, \r\n   ...:   PRIMARY KEY(albumartist, albumname) \r\n   ...: ); \r\n   ...:  \r\n   ...: CREATE TABLE song( \r\n   ...:   songid     INTEGER, \r\n   ...:   songartist TEXT, \r\n   ...:   songalbum TEXT, \r\n   ...:   songname   TEXT, \r\n   ...:   FOREIGN KEY(songartist, songalbum) REFERENCES album(albumartist, albumname) \r\n   ...: ); \r\n   ...: \"\"\")                                                                                                                               \r\nOut[5]: <sqlite3.Cursor at 0x1088def10>\r\n\r\nIn [6]: db = sqlite_utils.Database(conn)                                                                                                   \r\n\r\nIn [7]: db.tables                                                                                                                          \r\nOut[7]: \r\n[<Table album (albumartist, albumname, albumcover)>,\r\n <Table song (songid, songartist, songalbum, songname)>]\r\n\r\nIn [8]: db.tables[0].foreign_keys                                                                                                          \r\nOut[8]: []\r\n\r\nIn [9]: db.tables[1].foreign_keys                                                                                                          \r\nOut[9]: \r\n[ForeignKey(table='song', column='songartist', other_table='album', other_column='albumartist'),\r\n ForeignKey(table='song', column='songalbum', other_table='album', other_column='albumname')]\r\n```\r\nThe table appears to have two separate foreign keys, when actually it has a single compound composite foreign key.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/117/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 652700770, "node_id": "MDU6SXNzdWU2NTI3MDA3NzA=", "number": 119, "title": "Ability to remove a foreign key", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2020-07-07T22:31:37Z", "updated_at": "2020-09-24T20:36:59Z", "closed_at": "2020-09-24T20:36:59Z", "author_association": "OWNER", "pull_request": null, "body": "Useful if you add one but make a mistake and need to undo it without recreating the database from scratch.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/119/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}