issues
604 rows where repo = 140912432 sorted by reactions
This data as json, CSV (advanced)
id | node_id | number | title | user | state | locked | assignee | milestone | comments | created_at | updated_at | closed_at | author_association | pull_request | body | repo | type | active_lock_reason | performed_via_github_app | reactions ▼ | draft | state_reason |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
432217625 | MDU6SXNzdWU0MzIyMTc2MjU= | 19 | Incorrect help text for enable-fts command | simonw 9599 | closed | 0 | 1.0 4348046 | 0 | 2019-04-11T19:46:44Z | 2019-05-25T00:44:31Z | 2019-05-25T00:44:31Z | OWNER | I clearly copied-and-pasted this from the |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/19/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
730693696 | MDExOlB1bGxSZXF1ZXN0NTEwOTU2MTM0 | 190 | Progress bar for sqlite-utils insert command | simonw 9599 | closed | 0 | 0 | 2020-10-27T18:08:53Z | 2020-10-27T18:16:03Z | 2020-10-27T18:16:03Z | OWNER | simonw/sqlite-utils/pulls/190 | Refs #173 |
sqlite-utils 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/190/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
731740458 | MDU6SXNzdWU3MzE3NDA0NTg= | 191 | Idea: @db.register_function(deterministic=True) | simonw 9599 | closed | 0 | 2 | 2020-10-28T19:45:18Z | 2020-10-28T21:31:06Z | 2020-10-28T21:31:06Z | OWNER | Python 3.8 added a
https://sqlite-utils.readthedocs.io/en/stable/python-api.html#registering-custom-sql-functions ```python db = Database(memory=True) @db.register_function(deterministic=True) def reverse_string(s): return "".join(reversed(list(s))) ``` |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/191/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
735532751 | MDU6SXNzdWU3MzU1MzI3NTE= | 192 | sqlite-utils search command | simonw 9599 | closed | 0 | 3.0 6079500 | 9 | 2020-11-03T18:07:59Z | 2020-11-08T17:07:01Z | 2020-11-08T17:07:01Z | OWNER | A command that knows how to run a search against a FTS enabled table and return results ranked by relevance. |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/192/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
735648209 | MDU6SXNzdWU3MzU2NDgyMDk= | 193 | --tsv output format option | simonw 9599 | closed | 0 | 3.0 6079500 | 0 | 2020-11-03T21:31:18Z | 2020-11-07T00:09:52Z | 2020-11-07T00:09:52Z | OWNER | We already support |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/193/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
735650864 | MDU6SXNzdWU3MzU2NTA4NjQ= | 194 | 3.0 release with some minor breaking changes | simonw 9599 | closed | 0 | 3.0 6079500 | 3 | 2020-11-03T21:36:31Z | 2020-11-08T17:19:35Z | 2020-11-08T17:19:34Z | OWNER | While working on search (#192) I've spotted a few small changes I would like to make that would break backwards compatibility in minor ways, hence requiring a 3.x release.
Also I'd like to free up the I'd like |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/194/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
735663855 | MDExOlB1bGxSZXF1ZXN0NTE1MDE0ODgz | 195 | table.search() improvements plus sqlite-utils search command | simonw 9599 | closed | 0 | 3 | 2020-11-03T22:02:08Z | 2020-11-06T18:30:49Z | 2020-11-06T18:30:42Z | OWNER | simonw/sqlite-utils/pulls/195 | Refs #192. Still needs tests. |
sqlite-utils 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/195/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
736520310 | MDU6SXNzdWU3MzY1MjAzMTA= | 196 | Introspect if table is FTS4 or FTS5 | simonw 9599 | closed | 0 | 19 | 2020-11-05T00:45:50Z | 2020-11-05T03:54:07Z | 2020-11-05T03:54:07Z | OWNER |
Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/192#issuecomment-722054264 |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/196/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
737153927 | MDU6SXNzdWU3MzcxNTM5Mjc= | 197 | Rethink how table.search() method works | simonw 9599 | closed | 0 | 3.0 6079500 | 5 | 2020-11-05T18:04:34Z | 2020-11-08T17:07:37Z | 2020-11-08T17:07:37Z | OWNER | I need to improve this method to help build The challenge is deciding how it should handle sorting by relevance - especially since that is easy in FTS5 but not at all easy in FTS4.
Originally posted by @simonw in https://github.com/simonw/sqlite-utils/pull/195#issuecomment-722542895 |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/197/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
737476423 | MDU6SXNzdWU3Mzc0NzY0MjM= | 198 | Support order by relevance against FTS4 | simonw 9599 | closed | 0 | 6 | 2020-11-06T05:36:31Z | 2020-11-06T18:30:44Z | 2020-11-06T18:30:44Z | OWNER | For #192 and #197 I've decided I want to be able to order by relevance in FTS4 as well as FTS5. This means I need to port over my work on bm25() from https://github.com/simonw/sqlite-fts4 (since I don't want to add a full dependency). |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/198/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
737855731 | MDU6SXNzdWU3Mzc4NTU3MzE= | 199 | @db.register_function(..., replace=False) to avoid double-registering custom functions | simonw 9599 | closed | 0 | 1 | 2020-11-06T15:39:21Z | 2020-11-06T18:30:44Z | 2020-11-06T18:30:44Z | OWNER | I'd like a mechanism to optionally avoid registering a custom function if it has already been registered. SQLite doesn't seem to offer a way to introspect registered custom functions so I'll need to track what has already been registered in
Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/198#issuecomment-723145383 |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/199/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
349850687 | MDU6SXNzdWUzNDk4NTA2ODc= | 2 | Mechanism for adding foreign keys to an existing table | simonw 9599 | closed | 0 | 1 | 2018-08-12T22:50:56Z | 2019-02-24T21:34:41Z | 2019-02-24T21:34:41Z | OWNER | SQLite does not have ALTER TABLE support for adding new foreign keys... but it turns out it's possible to make these changes without having to duplicate the entire table by carefully running Here's how Django does it: https://github.com/django/django/blob/d3449faaa915a08c275b35de01e66a7ef6bdb2dc/django/db/backends/sqlite3/schema.py#L103-L125 And here's the official documentation about this: https://sqlite.org/lang_altertable.html#otheralter (scroll to the very bottom of the page) |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/2/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
432727685 | MDU6SXNzdWU0MzI3Mjc2ODU= | 20 | JSON column values get extraneously quoted | mhalle 649467 | closed | 0 | 1.0 4348046 | 1 | 2019-04-12T20:15:30Z | 2019-05-25T00:57:19Z | 2019-05-25T00:57:19Z | NONE | If the input to ``` echo '[{"key": ["one", "two", "three"]}]' | sqlite-utils insert t.db t -sqlite-utils t.db 'select * from t'[{"key": "[\"one\", \"two\", \"three\"]"}] sqlite3 t.db 'select * from t'["one", "two", "three"] ``` This might require an imperfect solution, since sqlite3 doesn't have a JSON type. Perhaps fields that start with |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/20/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
738115165 | MDU6SXNzdWU3MzgxMTUxNjU= | 200 | sqlite-utils rows -c option | simonw 9599 | closed | 0 | 3.0 6079500 | 1 | 2020-11-07T00:22:12Z | 2020-11-07T00:28:48Z | 2020-11-07T00:28:47Z | OWNER | To let you specify the exact columns you want. Based on the |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/200/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
738128913 | MDU6SXNzdWU3MzgxMjg5MTM= | 201 | .search(columns=) and sqlite-utils search -c ... bug | simonw 9599 | closed | 0 | 3.0 6079500 | 1 | 2020-11-07T01:27:26Z | 2020-11-08T16:54:15Z | 2020-11-08T16:54:15Z | OWNER | Both This should be fixed before the 3.0 non-alpha release. |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/201/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
738514367 | MDU6SXNzdWU3Mzg1MTQzNjc= | 202 | sqlite-utils insert -f colname - for configuring full-text search | simonw 9599 | closed | 0 | 2 | 2020-11-08T17:30:09Z | 2021-01-03T05:00:36Z | 2021-01-03T05:00:27Z | OWNER | A mechanism for specifying columns that should be configured for full-text search as part of the initial data import:
|
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/202/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
743384829 | MDExOlB1bGxSZXF1ZXN0NTIxMjg3OTk0 | 203 | changes to allow for compound foreign keys | drkane 1049910 | open | 0 | 7 | 2020-11-16T00:30:10Z | 2023-01-25T18:47:18Z | FIRST_TIME_CONTRIBUTOR | simonw/sqlite-utils/pulls/203 | Add support for compound foreign keys, as per issue #117 Not sure if this is the right approach. In particular I'm unsure about:
The PR also contains a minor related change that columns and tables are always quoted in foreign key definitions. |
sqlite-utils 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/203/reactions", "total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 1, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||||
752888228 | MDExOlB1bGxSZXF1ZXN0NTI5MDkwNTYw | 204 | use jsonify_if_need for sql updates | mfa 78035 | closed | 0 | 1 | 2020-11-29T10:49:00Z | 2020-12-08T17:49:42Z | 2020-12-08T17:49:42Z | CONTRIBUTOR | simonw/sqlite-utils/pulls/204 | sqlite-utils 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/204/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||||
760960559 | MDU6SXNzdWU3NjA5NjA1NTk= | 205 | sqlite3.OperationalError: near "(": syntax error | kaihendry 765871 | closed | 0 | 2 | 2020-12-10T06:44:40Z | 2020-12-10T19:18:22Z | 2020-12-10T07:24:23Z | NONE | The sqlite version is 3.22.0 2018-01-22 18:45:57 0c55d179733b46d8d0ba4d88e01a25e10677046ee3da1d5b1581e86726f2alt1 sqlite-utils, version 3.0 It fails here: https://github.com/kaihendry/aws-partners-datasette/runs/1528432635?check_suite_focus=true I'm not sure where the problem is, since it works fine locally on Archlinux system running 3.34.0 2020-12-01 16:14:00 a26b6597e3ae272231b96f9982c3bcc17ddec2f2b6eb4df06a224b91089fed5b https://github.com/kaihendry/aws-partners-datasette/blob/main/create-summary-view.sh Maybe I need to bump up from ubuntu-latest to ? |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/205/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
761915790 | MDU6SXNzdWU3NjE5MTU3OTA= | 206 | sqlite-utils should suggest --csv if JSON parsing fails | simonw 9599 | closed | 0 | 4 | 2020-12-11T05:17:56Z | 2021-10-30T15:52:17Z | 2021-01-03T18:42:22Z | OWNER |
|
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/206/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
763283616 | MDU6SXNzdWU3NjMyODM2MTY= | 207 | sqlite-utils analyze-tables command | simonw 9599 | closed | 0 | 4 | 2020-12-12T04:33:12Z | 2020-12-13T07:25:23Z | 2020-12-13T07:20:13Z | OWNER | A command which analyzes a table (potentially taking quite a while if the table is large) and outputs information for each column - things like:
The command can output this information to the terminal, but it should also provide an option for writing the information to a database table so it can be explored later. |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/207/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
763320133 | MDExOlB1bGxSZXF1ZXN0NTM3NzkxNjc1 | 208 | sqlite-utils analyze-tables command and table.analyze_column() method | simonw 9599 | closed | 0 | 6 | 2020-12-12T05:27:49Z | 2020-12-13T07:20:16Z | 2020-12-13T07:20:12Z | OWNER | simonw/sqlite-utils/pulls/208 | Refs #207
|
sqlite-utils 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/208/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
766156875 | MDU6SXNzdWU3NjYxNTY4NzU= | 209 | Test failure with sqlite 3.34 in test_cli.py::test_optimize | meatcar 191622 | closed | 0 | 1 | 2020-12-14T08:58:18Z | 2021-01-01T23:52:46Z | 2021-01-01T23:52:46Z | CONTRIBUTOR | pytest output:
Came across this while packaging ``` docker run --rm -it alpine:edge /bin/sh apk update && apk add git sqlite python3 gcc python3-dev musl-dev && python3 -m ensurepipgit clone https://github.com/simonw/sqlite-utils.gitcd sqlite-utils/pip3 install -e .[test]pytest``` This definitely works on sqlite v3.33. |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/209/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
448391492 | MDU6SXNzdWU0NDgzOTE0OTI= | 21 | Option to ignore inserts if primary key exists already | simonw 9599 | closed | 0 | 3 | 2019-05-25T00:17:12Z | 2019-05-29T05:09:01Z | 2019-05-29T04:18:26Z | OWNER |
Originally posted by @psychemedia in https://github.com/simonw/sqlite-utils/issues/18#issuecomment-480621924 |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/21/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
767685961 | MDU6SXNzdWU3Njc2ODU5NjE= | 210 | Support of RData files | PeterBailey 23739126 | closed | 0 | 1 | 2020-12-15T15:04:14Z | 2021-01-02T00:02:40Z | 2021-01-02T00:02:40Z | NONE | Hi Simon, Would be great if you could ingest RData files! I could do this in a few lines of code but I am too lazy - sorry! Peter |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/210/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
777386465 | MDU6SXNzdWU3NzczODY0NjU= | 211 | table.triggers_dict introspection property | simonw 9599 | closed | 0 | 0 | 2021-01-02T02:04:00Z | 2021-01-02T02:10:10Z | 2021-01-02T02:10:10Z | OWNER |
|
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/211/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
777392020 | MDU6SXNzdWU3NzczOTIwMjA= | 212 | Mechanism for maintaining cache of table counts using triggers | simonw 9599 | closed | 0 | 1 | 2021-01-02T02:58:53Z | 2021-01-02T21:40:27Z | 2021-01-02T21:40:27Z | OWNER | Counting all of the rows in a large table is expensive - this is one of the main causes of performance problems in Datasette when running against large databases. Carefully constructed SQL triggers could be used to maintain accurate cached counts for a table, by incrementing and decrementing a counter every time a row is inserted or deleted.
|
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/212/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
777529979 | MDU6SXNzdWU3Nzc1Mjk5Nzk= | 213 | db.enable_counts() method | simonw 9599 | closed | 0 | 2 | 2021-01-02T21:44:55Z | 2021-01-02T22:04:02Z | 2021-01-02T22:04:02Z | OWNER | Following #212 it would be useful if there was a utility method for enabling counts for ALL tables in a database:
Open question: should this setup triggers for virtual tables such as FTS tables? Could that break things? |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/213/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
777530107 | MDU6SXNzdWU3Nzc1MzAxMDc= | 214 | sqlite-utils enable-counts command | simonw 9599 | closed | 0 | 0 | 2021-01-02T21:45:48Z | 2021-01-03T04:26:44Z | 2021-01-03T04:26:44Z | OWNER | The CLI version of #212 and #213.
|
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/214/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
777535402 | MDU6SXNzdWU3Nzc1MzU0MDI= | 215 | Use _counts to speed up counts | simonw 9599 | closed | 0 | 9 | 2021-01-02T22:30:17Z | 2021-01-03T20:19:40Z | 2021-01-03T20:19:40Z | OWNER | Utility mechanism for taking advantage of the new These can trigger automatically if the |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/215/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
777540352 | MDU6SXNzdWU3Nzc1NDAzNTI= | 216 | database.triggers_dict introspection property | simonw 9599 | closed | 0 | 1 | 2021-01-02T23:13:00Z | 2021-01-03T04:27:14Z | 2021-01-03T04:25:36Z | OWNER | Following #211 |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/216/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
777543336 | MDU6SXNzdWU3Nzc1NDMzMzY= | 217 | Rename .escape() to .quote() | simonw 9599 | closed | 0 | 2 | 2021-01-02T23:40:52Z | 2021-01-03T04:27:38Z | 2021-01-03T04:15:23Z | OWNER |
This method has never been documented so I'm going to rename it without a major version bump. |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/217/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
777560474 | MDU6SXNzdWU3Nzc1NjA0NzQ= | 218 | "sqlite-utils triggers" command | simonw 9599 | closed | 0 | 1 | 2021-01-03T02:34:50Z | 2021-01-03T03:49:51Z | 2021-01-03T03:03:35Z | OWNER | A command to list the triggers in the database.
Can optionally take one or more tables:
|
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/218/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
777707544 | MDU6SXNzdWU3Nzc3MDc1NDQ= | 219 | reset_counts() method and command | simonw 9599 | closed | 0 | 4 | 2021-01-03T20:08:28Z | 2021-01-03T20:59:37Z | 2021-01-03T20:59:37Z | OWNER |
Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/215#issuecomment-753545757 |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/219/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
448395665 | MDU6SXNzdWU0NDgzOTU2NjU= | 22 | Release notes for 1.0 | simonw 9599 | closed | 0 | 1.0 4348046 | 2 | 2019-05-25T00:58:03Z | 2019-05-25T01:18:27Z | 2019-05-25T01:06:52Z | OWNER | sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/22/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
783778672 | MDU6SXNzdWU3ODM3Nzg2NzI= | 220 | Better error message for *_fts methods against views | mhalle 649467 | closed | 0 | 3 | 2021-01-11T23:24:00Z | 2021-02-22T20:44:51Z | 2021-02-14T22:34:26Z | NONE | enable_fts and its related methods only work on tables, not views. Could those methods and possibly others move up to the Queryable superclass? |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/220/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
783910901 | MDU6SXNzdWU3ODM5MTA5MDE= | 221 | .add_missing_columns() does not take case insensitivity into account | simonw 9599 | closed | 0 | 0 | 2021-01-12T05:01:00Z | 2021-01-12T23:17:33Z | 2021-01-12T23:17:33Z | OWNER | SQLite columns are case insensitive - but the |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/221/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
787900412 | MDU6SXNzdWU3ODc5MDA0MTI= | 222 | .m2m() should accept alter=True parameter | simonw 9599 | closed | 0 | 0 | 2021-01-18T04:15:43Z | 2021-01-18T04:26:10Z | 2021-01-18T04:26:10Z | OWNER | sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/222/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||||
788527932 | MDU6SXNzdWU3ODg1Mjc5MzI= | 223 | --delimiter option for CSV import | simonw 9599 | closed | 0 | 2 | 2021-01-18T20:25:03Z | 2021-02-06T01:39:47Z | 2021-02-06T01:34:54Z | OWNER |
Would be useful to be able to do this:
|
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/223/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
792297010 | MDExOlB1bGxSZXF1ZXN0NTYwMjA0MzA2 | 224 | Add fts offset docs. | polyrand 37962604 | closed | 0 | 2 | 2021-01-22T20:50:58Z | 2021-02-14T19:31:06Z | 2021-02-14T19:31:06Z | NONE | simonw/sqlite-utils/pulls/224 | The limit can be passed as a string to the query builder to have an offset. I have tested it using the shorthand |
sqlite-utils 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/224/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
797159961 | MDExOlB1bGxSZXF1ZXN0NTY0MjE1MDEx | 225 | fix for problem in Table.insert_all on search for columns per chunk of rows | nieuwenhoven 261237 | closed | 0 | 2 | 2021-01-29T20:16:07Z | 2021-02-14T21:04:13Z | 2021-02-14T21:04:13Z | NONE | simonw/sqlite-utils/pulls/225 | Hi, I ran into a problem when trying to create a database from my Apple Healthkit data using healthkit-to-sqlite. The program crashed because of an invalid insert statement that was generated for table The actual problem turned out to be in sqlite-utils. I'm using a Windows machine and had to make a few adjustments to the tests in order to be able to run them because they had a posix dependency. Thanks, kind regards, Frans ``` this is a (condensed) chunk of data from my Apple healthkit export that caused the problem.the 3 last items in the chunk have additional keys: metadata_HKMetadataKeySyncVersion and metadata_HKMetadataKeySyncIdentifierchunk = [{'sourceName': 'AppleÂ\xa0Watch van Frans', 'sourceVersion': '7.0.1', 'device': '<<HKDevice: 0x281cf6c70>, name:Apple Watch, manufacturer:Apple Inc., model:Watch, hardware:Watch3,4, software:7.0.1>', 'unit': 'km', 'creationDate': '2020-10-10 12:29:09 +0100', 'startDate': '2020-10-10 12:29:06 +0100', 'endDate': '2020-10-10 12:29:07 +0100', 'value': '0.00518016'}, {'sourceName': 'AppleÂ\xa0Watch van Frans', 'sourceVersion': '7.0.1', 'device': '<<HKDevice: 0x281cf6c70>, name:Apple Watch, manufacturer:Apple Inc., model:Watch, hardware:Watch3,4, software:7.0.1>', 'unit': 'km', 'creationDate': '2020-10-10 12:29:10 +0100', 'startDate': '2020-10-10 12:29:07 +0100', 'endDate': '2020-10-10 12:29:08 +0100', 'value': '0.00544049'}, {'sourceName': 'AppleÂ\xa0Watch van Frans', 'sourceVersion': '6.2.6', 'device': '<<HKDevice: 0x281cf83e0>, name:Apple Watch, manufacturer:Apple Inc., model:Watch, hardware:Watch3,4, software:6.2.6>', 'unit': 'km', 'creationDate': '2020-10-14 05:54:12 +0100', 'startDate': '2020-07-15 16:40:50 +0100', 'endDate': '2020-07-15 16:42:49 +0100', 'value': '0.952092', 'metadata_HKMetadataKeySyncVersion': '1', 'metadata_HKMetadataKeySyncIdentifier': '3:674DBCDB-3FE8-40D1-9FC1-E54A2B413805:616520450.99823:616520569.99360:119'}, {'sourceName': 'AppleÂ\xa0Watch van Frans', 'sourceVersion': '6.2.6', 'device': '<<HKDevice: 0x281cf83e0>, name:Apple Watch, manufacturer:Apple Inc., model:Watch, hardware:Watch3,4, software:6.2.6>', 'unit': 'km', 'creationDate': '2020-10-14 05:54:12 +0100', 'startDate': '2020-07-15 16:42:49 +0100', 'endDate': '2020-07-15 16:44:51 +0100', 'value': '0.848983', 'metadata_HKMetadataKeySyncVersion': '1', 'metadata_HKMetadataKeySyncIdentifier': '3:674DBCDB-3FE8-40D1-9FC1-E54A2B413805:616520569.99360:616520691.98826:119'}, {'sourceName': 'AppleÂ\xa0Watch van Frans', 'sourceVersion': '6.2.6', 'device': '<<HKDevice: 0x281cf83e0>, name:Apple Watch, manufacturer:Apple Inc., model:Watch, hardware:Watch3,4, software:6.2.6>', 'unit': 'km', 'creationDate': '2020-10-14 05:54:12 +0100', 'startDate': '2020-07-15 16:44:51 +0100', 'endDate': '2020-07-15 16:46:50 +0100', 'value': '0.834403', 'metadata_HKMetadataKeySyncVersion': '1', 'metadata_HKMetadataKeySyncIdentifier': '3:674DBCDB-3FE8-40D1-9FC1-E54A2B413805:616520691.98826:616520810.98305:119'}] def all_columns_old(): all_columns = [col for col in chunk[0]] all_columns += [column for record in chunk for column in record if column not in all_columns] return all_columns def all_columns_new(): all_columns = [col for col in chunk[0]] for record in chunk: all_columns += [column for column in record if column not in all_columns] return all_columns if name == 'main': from pprint import pprint
``` |
sqlite-utils 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/225/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
802583450 | MDU6SXNzdWU4MDI1ODM0NTA= | 226 | 3.4 release is broken - includes a rogue line | simonw 9599 | closed | 0 | 0 | 2021-02-06T02:08:01Z | 2021-02-06T02:10:26Z | 2021-02-06T02:10:26Z | OWNER | I started seeing weird errors, caused by this line: https://github.com/simonw/sqlite-utils/blob/f8010ca78fed8c5fca6cde19658ec09fdd468420/sqlite_utils/cli.py#L1-L3 That was added by accident in 1b666f9315d4ea6bb332b2e75e48480c26100199 I'm surprised the tests didn't catch this! |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/226/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
807174161 | MDU6SXNzdWU4MDcxNzQxNjE= | 227 | Error reading csv files with large column data | camallen 295329 | closed | 0 | 4 | 2021-02-12T11:51:47Z | 2021-02-16T11:48:03Z | 2021-02-14T21:17:19Z | NONE | Feel free to close this issue - I mostly added it for reference for future folks that run into this :) I have a CSV file with one column that has very long strings. When i try to import this file via the Traceback (most recent call last):
File "/usr/local/bin/sqlite-utils", line 10, in <module>
sys.exit(cli())
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 829, in call
return self.main(args, kwargs)
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, ctx.params)
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 610, in invoke
return callback(args, kwargs)
File "/usr/local/lib/python3.7/site-packages/sqlite_utils/cli.py", line 774, in insert
default=default,
File "/usr/local/lib/python3.7/site-packages/sqlite_utils/cli.py", line 705, in insert_upsert_implementation
docs, pk=pk, batch_size=batch_size, alter=alter, extra_kwargs
File "/usr/local/lib/python3.7/site-packages/sqlite_utils/db.py", line 1852, in insert_all
first_record = next(records)
File "/usr/local/lib/python3.7/site-packages/sqlite_utils/cli.py", line 703, in <genexpr>
docs = (decode_base64_values(doc) for doc in docs)
File "/usr/local/lib/python3.7/site-packages/sqlite_utils/cli.py", line 681, in <genexpr>
docs = (dict(zip(headers, row)) for row in reader)
_csv.Error: field larger than field limit (131072)
sqlite-utils --versionsqlite-utils, version 3.4.1 datasette --versiondatasette, version 0.54 ``` It appears this is a known issue reading in csv files in python and doesn't look to be modifiable through system / env vars (i may be very wrong on this). Noting that using sqlite3 Finally, I'm loving https://datasette.io/ thank you very much for an amazing tool and data ecosytem 🙇♀️ |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/227/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
807437089 | MDU6SXNzdWU4MDc0MzcwODk= | 228 | --no-headers option for CSV and TSV | simonw 9599 | closed | 0 | 10 | 2021-02-12T17:56:51Z | 2021-12-26T07:01:31Z | 2021-02-14T22:25:17Z | OWNER | https://bl.iro.bl.uk/work/ns/3037474a-761c-456d-a00c-9ef3c6773f4c has a fascinating CSV file that doesn't have a header row - it starts like this:
It would be useful if |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/228/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
807817197 | MDU6SXNzdWU4MDc4MTcxOTc= | 229 | Hitting `_csv.Error: field larger than field limit (131072)` | frosencrantz 631242 | closed | 0 | 3 | 2021-02-13T19:52:44Z | 2021-02-14T21:33:33Z | 2021-02-14T21:33:33Z | NONE | I have a csv file where one of the fields is so large it is throwing an exception with this error and stops loading:
The stack trace occurs here: https://github.com/simonw/sqlite-utils/blob/3.1/sqlite_utils/cli.py#L633 There is a way to handle this that helps: https://stackoverflow.com/questions/15063936/csv-error-field-larger-than-field-limit-131072 One issue I had with this problem was sqlite-utils only provides limited context as to where the problem line is. There is the progress bar, but that is by percent rather than by line number. It would have been helpful if it could have provided a line number. Also, it would have been useful if it had allowed the loading to continue with later lines. |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/229/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
449565204 | MDU6SXNzdWU0NDk1NjUyMDQ= | 23 | Syntactic sugar for creating m2m records | simonw 9599 | closed | 0 | 10 | 2019-05-29T02:17:48Z | 2019-08-04T03:54:58Z | 2019-08-04T03:37:34Z | OWNER | Python library only. What would be a syntactically pleasant way of creating a m2m record? |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/23/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
808008305 | MDU6SXNzdWU4MDgwMDgzMDU= | 230 | --sniff option for sniffing delimiters | simonw 9599 | closed | 0 | 8 | 2021-02-14T17:43:54Z | 2021-02-14T21:15:33Z | 2021-02-14T19:24:32Z | OWNER |
Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/228#issuecomment-778812050 |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/230/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
808028757 | MDU6SXNzdWU4MDgwMjg3NTc= | 231 | limit=X, offset=Y parameters for more Python methods | simonw 9599 | closed | 0 | 2 | 2021-02-14T19:31:23Z | 2021-02-14T20:03:08Z | 2021-02-14T20:03:08Z | OWNER |
Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/224#issuecomment-778828495 |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/231/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
808036774 | MDU6SXNzdWU4MDgwMzY3NzQ= | 232 | Run tests against Windows in GitHub Actions | simonw 9599 | closed | 0 | 0 | 2021-02-14T20:09:45Z | 2021-02-14T20:39:55Z | 2021-02-14T20:39:55Z | OWNER |
Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/225#issuecomment-778834504 |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/232/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
808037010 | MDExOlB1bGxSZXF1ZXN0NTczMTQ3MTY4 | 233 | Run tests against Ubuntu, macOS and Windows | simonw 9599 | closed | 0 | 0 | 2021-02-14T20:11:02Z | 2021-02-14T20:39:54Z | 2021-02-14T20:39:54Z | OWNER | simonw/sqlite-utils/pulls/233 | Refs #232 |
sqlite-utils 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/233/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
808046597 | MDU6SXNzdWU4MDgwNDY1OTc= | 234 | .insert_all() fails if subsequent chunks contain additional columns | simonw 9599 | closed | 0 | 1 | 2021-02-14T21:01:51Z | 2021-02-14T21:03:40Z | 2021-02-14T21:03:40Z | OWNER | Reported by @nieuwenhoven in #225 along with a proposed fix. |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/234/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
810618495 | MDU6SXNzdWU4MTA2MTg0OTU= | 235 | Extract columns cannot create foreign key relation: sqlite3.OperationalError: table sqlite_master may not be modified | kristomi 6913891 | closed | 0 | 18 | 2021-02-17T23:33:23Z | 2023-06-26T01:47:01Z | 2023-06-25T23:25:53Z | NONE | Thanks for what seems like a truly great suite of libraries. I wanted to try out Datasette, but never got more than half way through your YouTube video with the SF tree dataset. Whenever I try to extract a column, I get a I have tried googling the problem, but all I've found is that this might be a problem with the sqlite3 database running in defensive mode, but I definitely can't know for sure. Does the problem seem familiar to you? |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/235/reactions", "total_count": 3, "+1": 3, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
811680502 | MDU6SXNzdWU4MTE2ODA1MDI= | 236 | --attach command line option for attaching extra databases | simonw 9599 | closed | 0 | 1 | 2021-02-19T04:38:30Z | 2021-02-19T05:10:41Z | 2021-02-19T05:08:43Z | OWNER | This will enable cross-database joins, as seen in https://github.com/simonw/datasette/issues/283 Also refs #113 |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/236/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
815554385 | MDU6SXNzdWU4MTU1NTQzODU= | 237 | db["my_table"].drop(ignore=True) parameter, plus sqlite-utils drop-table --ignore and drop-view --ignore | mhalle 649467 | closed | 0 | 3 | 2021-02-24T14:55:06Z | 2021-02-25T17:11:41Z | 2021-02-25T17:11:41Z | NONE | When I'm generating a derived table in python, I often drop the table and create it from scratch. However, the first time I generate the table, it doesn't exist, so the drop raises an exception. That means more boilerplate. I was going to submit a pull request that adds an "if_exists" option to the However, for a utility like sqlite_utils, perhaps the "IF EXISTS" SQL semantics is what you want most of the time, and thus should be the default. What do you think? |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/237/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
816523763 | MDU6SXNzdWU4MTY1MjM3NjM= | 238 | .add_foreign_key() corrupts database if column contains a space | simonw 9599 | closed | 0 | 1 | 2021-02-25T15:07:20Z | 2021-02-25T16:54:02Z | 2021-02-25T16:54:02Z | OWNER | I ran this:
And got this: ``` ~/jupyter-venv/lib/python3.9/site-packages/sqlite_utils/db.py in add_foreign_keys(self, foreign_keys) 616 # Have to VACUUM outside the transaction to ensure .foreign_keys property 617 # can see the newly created foreign key. --> 618 self.vacuum() 619 620 def index_foreign_keys(self): ~/jupyter-venv/lib/python3.9/site-packages/sqlite_utils/db.py in vacuum(self) 629 630 def vacuum(self): --> 631 self.execute("VACUUM;") 632 633 ~/jupyter-venv/lib/python3.9/site-packages/sqlite_utils/db.py in execute(self, sql, parameters) 234 return self.conn.execute(sql, parameters) 235 else: --> 236 return self.conn.execute(sql) 237 238 def executescript(self, sql): DatabaseError: database disk image is malformed ``` |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/238/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
816526538 | MDU6SXNzdWU4MTY1MjY1Mzg= | 239 | sqlite-utils extract could handle nested objects | simonw 9599 | open | 0 | 16 | 2021-02-25T15:10:28Z | 2022-09-03T23:46:02Z | OWNER | Imagine a table (imported from a nested JSON file) where one of the columns contains values that look like this:
The |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/239/reactions", "total_count": 6, "+1": 5, "-1": 0, "laugh": 0, "hooray": 1, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
449818897 | MDU6SXNzdWU0NDk4MTg4OTc= | 24 | Additional Column Constraints? | IgnoredAmbience 98555 | closed | 0 | 6 | 2019-05-29T13:47:03Z | 2019-06-13T06:47:17Z | 2019-06-13T06:30:26Z | NONE | I'm looking to import data from XML with a pre-defined schema that maps fairly closely to a relational database. In particular, it has explicit annotations for when fields are required, optional, or when a default value should be inferred. Would there be value in adding the ability to define |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/24/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
816560819 | MDU6SXNzdWU4MTY1NjA4MTk= | 240 | table.pks_and_rows_where() method returning primary keys along with the rows | simonw 9599 | closed | 0 | 7 | 2021-02-25T15:49:28Z | 2021-02-25T16:39:23Z | 2021-02-25T16:28:23Z | OWNER | Original title: Easier way to update a row returned from .rows Here's a surprisingly hard problem I ran into while trying to implement #239 - given a row returned by The problem is that the Instead, currently, you need to introspect the table and, if A utility mechanism to make this easier would be very welcome. |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/240/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
816601354 | MDExOlB1bGxSZXF1ZXN0NTgwMjM1NDI3 | 241 | Extract expand - work in progress | simonw 9599 | open | 0 | 0 | 2021-02-25T16:36:38Z | 2021-02-25T16:36:38Z | OWNER | simonw/sqlite-utils/pulls/241 | Refs #239. Still needs documentation and CLI implementation. |
sqlite-utils 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/241/reactions", "total_count": 3, "+1": 3, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1 | ||||||
817989436 | MDU6SXNzdWU4MTc5ODk0MzY= | 242 | Async support | eyeseast 25778 | open | 0 | 13 | 2021-02-27T18:29:38Z | 2021-10-28T14:37:56Z | CONTRIBUTOR | Following our conversation last week, want to note this here before I forget. I've had a couple situations where I'd like to do a bunch of updates in an async event loop, but I run into SQLite's issues with concurrent writes. This feels like something sqlite-utils could help with. PeeWee ORM has a SQLite write queue that might be a good model. It's using threads or gevent, but I think that approach would translate well enough to asyncio. Happy to help with this, too. |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/242/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
818684978 | MDU6SXNzdWU4MTg2ODQ5Nzg= | 243 | How can i use this utils to deal with fts on column meta of tables ? | svjack 27874014 | open | 0 | 0 | 2021-03-01T09:45:05Z | 2021-03-01T09:45:05Z | NONE | Thank you to release this bravo project. When i use this project on multi table db, I want to implement convenient search on column name from different tables. I want to develop a meta table to save the meta data of different columns of different tables and search on this meta table to get rows from the data table (which the meta table describes) does this project provide some simple function on it ? You can think a have a knowledge graph about the table in the db, and i save this knowledge graph into the db with fts enabled. |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/243/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
820468864 | MDExOlB1bGxSZXF1ZXN0NTgzNDA3OTg5 | 244 | Typo in upsert example | j-e-d 387669 | closed | 0 | 1 | 2021-03-02T23:14:14Z | 2021-05-19T02:58:21Z | 2021-05-19T02:58:21Z | CONTRIBUTOR | simonw/sqlite-utils/pulls/244 | Remove extra |
sqlite-utils 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/244/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
830803173 | MDExOlB1bGxSZXF1ZXN0NTkyMjg5MzI0 | 245 | Correct some typos | dbready 1076745 | closed | 0 | 1 | 2021-03-13T04:26:56Z | 2021-05-19T02:58:04Z | 2021-05-19T02:58:04Z | CONTRIBUTOR | simonw/sqlite-utils/pulls/245 | Noticed a typo in the docs and followed that up with a spellcheck. Had to bite my tongue at some of the British spellings. |
sqlite-utils 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/245/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
831751367 | MDU6SXNzdWU4MzE3NTEzNjc= | 246 | Escaping FTS search strings | DeNeutoy 16001974 | closed | 0 | 4 | 2021-03-15T12:15:09Z | 2021-08-18T18:57:13Z | 2021-08-18T18:43:12Z | CONTRIBUTOR | Thanks for the excellent library, it's very nice to use! I've been building some in memory search functionality for a data annotation tool i'm making, and I got tripped up a little bit with escaping the full text search queries. First I tried using http://search-24ways.herokuapp.com/24ways-f8f455f/articles?_search=acces%2A I got around this by aggressively escaping quotes inside the query string like this: ```python quoted = q.replace('"', '""') quoted = f'"{quoted}"' print(quoted) results = db["data"].search(quoted, columns=["id"]) return [x["id"] for x in results] ``` This works in the sense it doesn't crash, but it also removes access to the search query syntax. Given the well specified definition, it might be possible for sqlite-utils to provide a |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/246/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
832687563 | MDExOlB1bGxSZXF1ZXN0NTkzODA1ODA0 | 247 | FTS quote functionality from datasette | DeNeutoy 16001974 | closed | 0 | 2 | 2021-03-16T11:17:34Z | 2021-08-18T18:43:12Z | 2021-08-18T18:43:12Z | CONTRIBUTOR | simonw/sqlite-utils/pulls/247 | Addresses #246 - this is a bit of a kludge because it doesn't actually validate the FTS string, just makes sure that it will not crash when executed, but I figured that building a query parser is a bit out of the scope of sqlite-utils and if you actually want to use the query language, you probably need to parse that yourself. |
sqlite-utils 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/247/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
836829560 | MDU6SXNzdWU4MzY4Mjk1NjA= | 248 | support for Apache Arrow / parquet files I/O | mhalle 649467 | open | 0 | 1 | 2021-03-20T14:59:30Z | 2021-10-28T23:46:48Z | NONE | I just started looking at Apache Arrow using pyarrow for import and export of tabular datasets, and it looks quite compelling. It might be worth looking at for sqlite-utils and/or datasette. As a test, I took a random jsonl data dump of a dataset I have with floats, strings, and ints and converted it to arrow's parquet format using the naive The only hangup is the automatic type inference of the naive reader. It's great for general laziness and for parsing JSON columns (it correctly interpreted a table of mine with a JSON array). However, I did get an exception for a string column where most entries looked integer-like but had a couple values that weren't -- the reader tried to coerce all of them for some reason, even though the JSON type is string. Since the writer optionally takes a schema, it shouldn't be too hard to grab the sqlite header types. With some additional hinting, you might get datetime columns and JSON, which are native Arrow types. Somewhat tangentially, someone even wrote an sqlite vfs extension for Parquet: https://cldellow.com/2018/06/22/sqlite-parquet-vtable.html |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/248/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
836963850 | MDU6SXNzdWU4MzY5NjM4NTA= | 249 | Full text search possibly broken? | prabhur 36287 | closed | 0 | 2 | 2021-03-21T02:03:44Z | 2021-03-21T02:43:32Z | 2021-03-21T02:43:32Z | NONE | I'm not quite sure if this is an issue with sqlite-utils or datasette. Background I was previously using sqlite-utils version < 3.6. I have a bunch of csv files that have some data scraped from a website.
Data is loaded to the table via
Note that the data contains text in my language Tamil. Loading happens just fine. datasette serves the db file just fine. It recognizes FTS and shows the "search" box. However, none of the queries work. Whatever text I supply, it always returns 0 rows. I literally copy paste words from the row listing on the screen and paste it on the search box. Interestingly, only thing I can remember is switching to sqlite-utils 3.6. I had to do this because the prior version had an issue with column size. I have attached one of the csv files that can be loaded to the table. Substitute "${filename}" with that file for the sqlite-utils insert command. posts_20200417-20201231.csv.zip Interestingly, the FTS based search from datasette worked just fine before this version upgrade. That is, the queries returned results. I will try to downgrade just to see if the theory is correct. I appreciate any help here. Thanks. |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/249/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
449848803 | MDU6SXNzdWU0NDk4NDg4MDM= | 25 | Allow .insert(..., foreign_keys=()) to auto-detect table and primary key | simonw 9599 | closed | 0 | 4 | 2019-05-29T14:39:22Z | 2019-06-13T05:32:32Z | 2019-06-13T05:32:32Z | OWNER | The |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/25/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
838148087 | MDU6SXNzdWU4MzgxNDgwODc= | 250 | Handle byte order marks (BOMs) in CSV files | simonw 9599 | closed | 0 | 3 | 2021-03-22T22:13:18Z | 2021-05-29T05:34:21Z | 2021-05-29T05:34:21Z | OWNER | I often find |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/250/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
841377702 | MDU6SXNzdWU4NDEzNzc3MDI= | 251 | "sqlite-utils convert" command to replace the separate "sqlite-transform" tool | simonw 9599 | closed | 0 | 15 | 2021-03-25T22:36:36Z | 2021-08-02T22:39:46Z | 2021-08-02T04:47:40Z | OWNER | See https://github.com/simonw/sqlite-transform/issues/11 - I built a separate |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/251/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
842062949 | MDU6SXNzdWU4NDIwNjI5NDk= | 252 | Support json-line files | rathboma 279769 | closed | 0 | 1 | 2021-03-26T15:19:39Z | 2021-03-26T15:21:38Z | 2021-03-26T15:21:38Z | NONE | It's common for many processes to create flat files where each line is a JSON object. So the file isn't a json array. Many tools (like jq) support this natively, it'd be great for sqlite-utils to also! |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/252/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
847423559 | MDU6SXNzdWU4NDc0MjM1NTk= | 253 | fixtures.db example error in sql-utils blog post | mroswell 192568 | closed | 0 | 2 | 2021-03-31T22:07:36Z | 2021-05-19T03:31:48Z | 2021-05-19T03:31:47Z | NONE | En route to trying to understand column order transform documentation, I tried the instructions here: https://simonwillison.net/2020/Sep/23/sqlite-advanced-alter-table/ I get a malformed database schema syntax error. ``` $ wget https://latest.datasette.io/fixtures.db --2021-03-31 18:00:23-- https://latest.datasette.io/fixtures.db Resolving latest.datasette.io (latest.datasette.io)... 2607:f8b0:4004:801::2013, 142.250.73.211 Connecting to latest.datasette.io (latest.datasette.io)|2607:f8b0:4004:801::2013|:443... connected. HTTP request sent, awaiting response... 200 OK Length: unspecified [application/octet-stream] Saving to: ‘fixtures.db’ fixtures.db [ <=> ] 260.00K --.-KB/s in 0.1s 2021-03-31 18:00:23 (2.41 MB/s) - ‘fixtures.db’ saved [266240] $ sqlite3 fixtures.db '.schema facetable' Error: malformed database schema (generated_columns) - near "AS": syntax error $ sqlite3 fixtures.db SQLite version 3.28.0 2019-04-15 14:49:49 Enter ".help" for usage hints. sqlite> .schema Error: malformed database schema (generated_columns) - near "AS": syntax error ``` |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/253/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
857280617 | MDExOlB1bGxSZXF1ZXN0NjE0NzI3MDM2 | 254 | Fix incorrect create-table cli description | robjwells 1935268 | closed | 0 | 1 | 2021-04-13T20:03:15Z | 2021-05-19T04:43:46Z | 2021-05-19T02:57:26Z | CONTRIBUTOR | simonw/sqlite-utils/pulls/254 | The description for |
sqlite-utils 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/254/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
858501079 | MDU6SXNzdWU4NTg1MDEwNzk= | 255 | transform --help should tell you the available types | simonw 9599 | closed | 0 | 0 | 2021-04-15T05:24:48Z | 2021-05-29T03:55:52Z | 2021-05-29T03:55:52Z | OWNER | ``` Usage: sqlite-utils transform [OPTIONS] PATH TABLE Transform a table beyond the capabilities of ALTER TABLE Options: --type <TEXT TEXT>... Change column type to X ``` This should specify that the possible types are 'INTEGER', 'TEXT', 'FLOAT', 'BLOB'. |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/255/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
861622839 | MDU6SXNzdWU4NjE2MjI4Mzk= | 256 | inserting with --nl errors with: sqlite3.OperationalError: table <table> has no column named <column> | rathboma 279769 | closed | 0 | 2 | 2021-04-19T18:01:03Z | 2021-05-19T03:26:54Z | 2021-05-19T03:26:54Z | NONE | I have a Inserting from the cli with This seems similar to #18 and #139, but maybe it's unique to |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/256/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
868188068 | MDU6SXNzdWU4NjgxODgwNjg= | 257 | Insert from JSON containing strings with non-ascii characters are escaped as unicode for lists, tuples, dicts. | dylan-wu 6586811 | closed | 0 | 0 | 2021-04-26T20:46:25Z | 2021-05-19T02:57:05Z | 2021-05-19T02:57:05Z | CONTRIBUTOR | JSON Test File (test.json):
Command to import:
Resulting table view from datasette: Original, db.py line 2225:
Fix, db.py line 2225:
|
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/257/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
868191959 | MDExOlB1bGxSZXF1ZXN0NjIzNzU1NzIz | 258 | Fixing insert from JSON containing strings with non-ascii characters … | dylan-wu 6586811 | closed | 0 | 1 | 2021-04-26T20:50:00Z | 2021-05-19T02:47:44Z | 2021-05-19T02:47:44Z | CONTRIBUTOR | simonw/sqlite-utils/pulls/258 | …are escaped aps unicode for lists, tuples, dicts Fix of #257 |
sqlite-utils 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/258/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
894948100 | MDU6SXNzdWU4OTQ5NDgxMDA= | 259 | Suggest the --alter option if a new column cannot be added | simonw 9599 | closed | 0 | 1 | 2021-05-19T03:17:38Z | 2021-05-19T03:27:33Z | 2021-05-19T03:26:26Z | OWNER | Refs #256. |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/259/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
455486286 | MDU6SXNzdWU0NTU0ODYyODY= | 26 | Mechanism for turning nested JSON into foreign keys / many-to-many | simonw 9599 | open | 0 | 14 | 2019-06-13T00:52:06Z | 2022-06-29T23:35:29Z | OWNER | The GitHub JSON APIs have a really interesting convention with respect to related objects. Consider https://api.github.com/repos/simonw/sqlite-utils/issues - here's a truncated subset:
Since both user and label have populated It would be really neat if Thanks to |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/26/reactions", "total_count": 4, "+1": 4, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
906330187 | MDU6SXNzdWU5MDYzMzAxODc= | 260 | Support creating descending order indexes | simonw 9599 | closed | 0 | 12 | 2021-05-29T03:42:59Z | 2021-05-29T05:01:39Z | 2021-05-29T05:01:39Z | OWNER | SQLite lets you create indexes in reverse order, which can have a surprisingly big impact on performance, see https://github.com/simonw/covid-19-datasette/issues/27 I tried doing this using
|
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/260/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
906345899 | MDU6SXNzdWU5MDYzNDU4OTk= | 261 | `table.xindexes` using `PRAGMA index_xinfo(table)` | simonw 9599 | closed | 0 | 5 | 2021-05-29T04:23:48Z | 2021-06-03T03:54:14Z | 2021-06-03T03:51:32Z | OWNER |
|
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/261/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
906355849 | MDExOlB1bGxSZXF1ZXN0NjU3MzczNzI2 | 262 | Ability to add descending order indexes | simonw 9599 | closed | 0 | 0 | 2021-05-29T04:51:04Z | 2021-05-29T05:01:42Z | 2021-05-29T05:01:39Z | OWNER | simonw/sqlite-utils/pulls/262 | Refs #260 |
sqlite-utils 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/262/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
906356331 | MDU6SXNzdWU5MDYzNTYzMzE= | 263 | `sqlite-utils indexes` command | simonw 9599 | closed | 0 | 6 | 2021-05-29T04:52:34Z | 2021-06-03T04:34:38Z | 2021-06-03T04:34:38Z | OWNER | While working on #260 I realized there's no command to show indexes in a database, even though there is one for showing tables and one for triggers. I should implement #261 first. |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/263/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
907642546 | MDU6SXNzdWU5MDc2NDI1NDY= | 264 | Supporting additional output formats, like GeoJSON | eyeseast 25778 | closed | 0 | 3 | 2021-05-31T18:03:32Z | 2021-06-03T05:12:21Z | 2021-06-03T05:12:21Z | CONTRIBUTOR | I have a project going where it would be useful to do some spatial processing in SQLite (instead of large files) and then output GeoJSON. So my workflow would be something like this:
I'm wondering if this is worth adding to SQLite-utils itself (GeoJSON, at least), or if it's better to make a counterpart to the ecosystem of |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/264/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
907795562 | MDU6SXNzdWU5MDc3OTU1NjI= | 265 | Using enable_fts before search term | prabhur 36287 | open | 0 | 1 | 2021-06-01T01:43:34Z | 2023-04-01T17:27:18Z | NONE | Many thanks for the sqlite-utils suite of utilities. Has made my life much much easier. I used this to create a table and enable FTS. All works fine. The datasette utility detects FTS and shows a text box. Searching for a term using that interface works well. However, when I start to use features by following https://www.sqlite.org/fts5.html section "3. Full-text Query Syntax" I seem to run into issues that I suspect is due to As an example, if i search for the term Similarly, when I try to restrict the search to a single column in FTS using a spec like
Any ideas why? How can I get the benefits of both escaping as well as utilizing different facets of providing / controlling search terms? Thanks. |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/265/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
913135723 | MDU6SXNzdWU5MTMxMzU3MjM= | 266 | Add some types, enforce with mypy | simonw 9599 | closed | 0 | 3 | 2021-06-07T06:05:56Z | 2021-08-18T22:25:38Z | 2021-08-18T22:25:38Z | OWNER | A good starting point would be adding type information to the members of these named tuples and the introspection methods that return them: |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/266/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
915421499 | MDU6SXNzdWU5MTU0MjE0OTk= | 267 | row.update() or row.pk | Gravitar64 12721157 | open | 0 | 4 | 2021-06-08T19:56:00Z | 2021-06-22T17:27:27Z | NONE | Hi, fantastic framework for working with Sqlite3 databases!!! I tried to update spezific rows in a table and used for row in db[tablename]:
newValue = row["counter"] * row["prize"] This updates the value in the printet row, but not in the database. So I switched to db[tablename].update(id, {"Filedname": newValue}) This works fine. But row.update would be nicer, because no need for the id (its that row), no need for the tablename and the db (all defined in the for row ... loop). Thx |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/267/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
919181559 | MDU6SXNzdWU5MTkxODE1NTk= | 268 | db.schema property and sqlite-utils schema command | simonw 9599 | closed | 0 | 4 | 2021-06-11T20:25:47Z | 2021-06-11T20:51:56Z | 2021-06-11T20:51:56Z | OWNER |
Can do this using https://latest.datasette.io/fixtures?sql=select+sql+from+sqlite_master+where+sql+is+not+null |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/268/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
919250621 | MDU6SXNzdWU5MTkyNTA2MjE= | 269 | bool type not supported | frafra 4068 | closed | 0 | 3 | 2021-06-11T22:00:36Z | 2021-06-15T01:34:10Z | 2021-06-15T01:34:10Z | NONE | Hi! Thank you for sharing this very nice tool :)
It would be nice to have support for more types, like |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/269/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
455496504 | MDU6SXNzdWU0NTU0OTY1MDQ= | 27 | sqlite-utils create-table command | simonw 9599 | closed | 0 | 8 | 2019-06-13T01:43:30Z | 2020-05-03T15:26:15Z | 2020-05-03T15:26:15Z | OWNER | Spun off from #24 - it would be useful if CLI users could create new tables (with explicit column types, not null rules and defaults) without having to insert an example record.
|
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/27/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
919314806 | MDU6SXNzdWU5MTkzMTQ4MDY= | 270 | Cannot set type JSON | frafra 4068 | closed | 0 | 4 | 2021-06-11T23:53:22Z | 2021-06-16T17:34:49Z | 2021-06-16T15:47:06Z | NONE | It would be great if the column type could be set to JSON. That would not be different from handling a regular string. It would be something like |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/270/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
919702451 | MDU6SXNzdWU5MTk3MDI0NTE= | 271 | table.upsert_all() fails if input has a single column that should be a primary key | simonw 9599 | closed | 0 | 1 | 2021-06-13T02:50:27Z | 2021-06-13T02:57:29Z | 2021-06-13T02:57:29Z | OWNER | This works: ```pycon <Table foo (name)> ``` But this fails: ``` >>> db['foo3'].upsert_all([{"name": "hello"}], pk="name") Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/Users/simon/.local/share/virtualenvs/datasette.io-TK86ygSO/lib/python3.9/site-packages/sqlite_utils/db.py", line 1837, in upsert_all return self.insert_all( File "/Users/simon/.local/share/virtualenvs/datasette.io-TK86ygSO/lib/python3.9/site-packages/sqlite_utils/db.py", line 1778, in insert_all self.insert_chunk( File "/Users/simon/.local/share/virtualenvs/datasette.io-TK86ygSO/lib/python3.9/site-packages/sqlite_utils/db.py", line 1588, in insert_chunk result = self.db.execute(query, params) File "/Users/simon/.local/share/virtualenvs/datasette.io-TK86ygSO/lib/python3.9/site-packages/sqlite_utils/db.py", line 213, in execute return self.conn.execute(sql, parameters) sqlite3.OperationalError: near "WHERE": syntax error ``` With the debugger: ``` >>> import pdb; pdb.pm() > /Users/simon/.local/share/virtualenvs/datasette.io-TK86ygSO/lib/python3.9/site-packages/sqlite_utils/db.py(213)execute() -> return self.conn.execute(sql, parameters) (Pdb) print(sql, parameters) UPDATE [foo3] SET WHERE [name] = ? ['hello'] ``` |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/271/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
921878733 | MDU6SXNzdWU5MjE4Nzg3MzM= | 272 | Idea: import CSV to memory, run SQL, export in a single command | simonw 9599 | closed | 0 | 22 | 2021-06-15T23:02:48Z | 2021-06-19T23:36:48Z | 2021-06-18T15:05:03Z | OWNER | I quite often load a CSV file into a SQLite DB, then do stuff with it (like export results back out again as a new CSV) without any intention of keeping the CSV file around afterwards. What if
|
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/272/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
922099793 | MDExOlB1bGxSZXF1ZXN0NjcxMDE0NzUx | 273 | sqlite-utils memory command for directly querying CSV/JSON data | simonw 9599 | closed | 0 | 8 | 2021-06-16T05:04:58Z | 2021-06-18T15:01:17Z | 2021-06-18T15:00:52Z | OWNER | simonw/sqlite-utils/pulls/273 | Refs #272. Initial implementation only does CSV data, still needs:
|
sqlite-utils 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/273/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
922832113 | MDU6SXNzdWU5MjI4MzIxMTM= | 274 | sqlite-utils dump my.db command | simonw 9599 | closed | 0 | 0 | 2021-06-16T16:30:14Z | 2021-06-16T23:51:54Z | 2021-06-16T23:51:54Z | OWNER | Inspired by the
|
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/274/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
922955697 | MDU6SXNzdWU5MjI5NTU2OTc= | 275 | Enable code coverage | simonw 9599 | closed | 0 | 1 | 2021-06-16T18:33:49Z | 2021-06-17T00:12:12Z | 2021-06-17T00:12:12Z | OWNER | https://app.codecov.io/gh/simonw/sqlite-utils Same mechanism as Datasette. Need to copy across the token from that page and add an equivalent of this workflow: https://github.com/simonw/datasette/blob/main/.github/workflows/test-coverage.yml |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/275/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
923602693 | MDU6SXNzdWU5MjM2MDI2OTM= | 276 | support small help flag -h | mcint 601708 | closed | 0 | 0 | 2021-06-17T07:59:31Z | 2021-06-18T14:56:59Z | 2021-06-18T14:56:59Z | CONTRIBUTOR | sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/276/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||||
923612361 | MDExOlB1bGxSZXF1ZXN0NjcyMzU5NjA5 | 277 | add -h support closes #276 | mcint 601708 | closed | 0 | 2 | 2021-06-17T08:08:26Z | 2021-06-18T14:56:59Z | 2021-06-18T14:56:59Z | CONTRIBUTOR | simonw/sqlite-utils/pulls/277 | This appears to be the canonical solution. |
sqlite-utils 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/277/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
923697888 | MDU6SXNzdWU5MjM2OTc4ODg= | 278 | Support db as first parameter before subcommand, or as environment variable | mcint 601708 | closed | 0 | 3 | 2021-06-17T09:26:29Z | 2021-06-20T22:39:57Z | 2021-06-18T15:43:19Z | CONTRIBUTOR | sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/278/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||||
924990677 | MDU6SXNzdWU5MjQ5OTA2Nzc= | 279 | sqlite-utils memory should handle TSV and JSON in addition to CSV | simonw 9599 | closed | 0 | 7 | 2021-06-18T15:02:54Z | 2021-06-19T03:11:59Z | 2021-06-19T03:11:59Z | OWNER |
Follow-on from #272 |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/279/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed |
Advanced export
JSON shape: default, array, newline-delimited, object
CREATE TABLE [issues] ( [id] INTEGER PRIMARY KEY, [node_id] TEXT, [number] INTEGER, [title] TEXT, [user] INTEGER REFERENCES [users]([id]), [state] TEXT, [locked] INTEGER, [assignee] INTEGER REFERENCES [users]([id]), [milestone] INTEGER REFERENCES [milestones]([id]), [comments] INTEGER, [created_at] TEXT, [updated_at] TEXT, [closed_at] TEXT, [author_association] TEXT, [pull_request] TEXT, [body] TEXT, [repo] INTEGER REFERENCES [repos]([id]), [type] TEXT , [active_lock_reason] TEXT, [performed_via_github_app] TEXT, [reactions] TEXT, [draft] INTEGER, [state_reason] TEXT); CREATE INDEX [idx_issues_repo] ON [issues] ([repo]); CREATE INDEX [idx_issues_milestone] ON [issues] ([milestone]); CREATE INDEX [idx_issues_assignee] ON [issues] ([assignee]); CREATE INDEX [idx_issues_user] ON [issues] ([user]);