issues
152 rows where comments = 1 and repo = 140912432 sorted by author_association
This data as json, CSV (advanced)
Suggested facets: milestone, author_association, created_at (date), updated_at (date), closed_at (date)
id | node_id | number | title | user | state | locked | assignee | milestone | comments | created_at | updated_at | closed_at | author_association ▼ | pull_request | body | repo | type | active_lock_reason | performed_via_github_app | reactions | draft | state_reason |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
488293926 | MDU6SXNzdWU0ODgyOTM5MjY= | 58 | Support enabling FTS on views | amjith 49260 | closed | 0 | 1 | 2019-09-02T18:56:36Z | 2020-10-16T18:39:36Z | 2020-10-16T18:39:31Z | CONTRIBUTOR | Right now enable_fts() is only implemented for Table(). Technically sqlite supports enabling fts on views. But it requires deeper thought since views don't have It is possible to provide an alternative rowid using the Ref: https://sqlite.org/fts5.html#fts5_table_creation_and_initialization
This will further complicate |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/58/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
546078359 | MDExOlB1bGxSZXF1ZXN0MzU5ODIyNzcz | 75 | Explicitly include tests and docs in sdist | jayvdb 15092 | closed | 0 | 1 | 2020-01-07T04:53:20Z | 2020-01-31T00:21:27Z | 2020-01-31T00:21:27Z | CONTRIBUTOR | simonw/sqlite-utils/pulls/75 | Also exclude 'tests' from runtime installation. |
sqlite-utils 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/75/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
652816158 | MDExOlB1bGxSZXF1ZXN0NDQ1ODMzOTA4 | 120 | Fix query command's support for DML | tsibley 79913 | closed | 0 | 1 | 2020-07-08T01:36:34Z | 2020-07-08T05:14:04Z | 2020-07-08T05:14:04Z | CONTRIBUTOR | simonw/sqlite-utils/pulls/120 | See commit messages for details. I ran into this while investigating another feature/issue. |
sqlite-utils 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/120/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
697030843 | MDExOlB1bGxSZXF1ZXN0NDgzMDI3NTg3 | 156 | Typos in tests | simonwiles 96218 | closed | 0 | 1 | 2020-09-09T18:00:58Z | 2020-09-09T18:24:50Z | 2020-09-09T18:21:23Z | CONTRIBUTOR | simonw/sqlite-utils/pulls/156 | One of these is my fault, and the other is one I just happened to come across. They're harmless, but might as well be fixed. |
sqlite-utils 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/156/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
697203800 | MDExOlB1bGxSZXF1ZXN0NDgzMTc1NTA5 | 158 | Fix accidental mega long line in docs | tomviner 167319 | closed | 0 | 1 | 2020-09-09T22:31:23Z | 2020-09-16T06:21:43Z | 2020-09-16T06:21:43Z | CONTRIBUTOR | simonw/sqlite-utils/pulls/158 | sqlite-utils 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/158/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||||
709043182 | MDExOlB1bGxSZXF1ZXN0NDkzMTYyNzY3 | 178 | Update README.md | shakeel 19921 | closed | 0 | 1 | 2020-09-25T15:52:11Z | 2020-10-01T14:18:30Z | 2020-09-30T20:29:28Z | CONTRIBUTOR | simonw/sqlite-utils/pulls/178 | The |
sqlite-utils 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/178/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
752888228 | MDExOlB1bGxSZXF1ZXN0NTI5MDkwNTYw | 204 | use jsonify_if_need for sql updates | mfa 78035 | closed | 0 | 1 | 2020-11-29T10:49:00Z | 2020-12-08T17:49:42Z | 2020-12-08T17:49:42Z | CONTRIBUTOR | simonw/sqlite-utils/pulls/204 | sqlite-utils 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/204/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||||
766156875 | MDU6SXNzdWU3NjYxNTY4NzU= | 209 | Test failure with sqlite 3.34 in test_cli.py::test_optimize | meatcar 191622 | closed | 0 | 1 | 2020-12-14T08:58:18Z | 2021-01-01T23:52:46Z | 2021-01-01T23:52:46Z | CONTRIBUTOR | pytest output:
Came across this while packaging ``` docker run --rm -it alpine:edge /bin/sh apk update && apk add git sqlite python3 gcc python3-dev musl-dev && python3 -m ensurepipgit clone https://github.com/simonw/sqlite-utils.gitcd sqlite-utils/pip3 install -e .[test]pytest``` This definitely works on sqlite v3.33. |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/209/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
820468864 | MDExOlB1bGxSZXF1ZXN0NTgzNDA3OTg5 | 244 | Typo in upsert example | j-e-d 387669 | closed | 0 | 1 | 2021-03-02T23:14:14Z | 2021-05-19T02:58:21Z | 2021-05-19T02:58:21Z | CONTRIBUTOR | simonw/sqlite-utils/pulls/244 | Remove extra |
sqlite-utils 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/244/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
830803173 | MDExOlB1bGxSZXF1ZXN0NTkyMjg5MzI0 | 245 | Correct some typos | dbready 1076745 | closed | 0 | 1 | 2021-03-13T04:26:56Z | 2021-05-19T02:58:04Z | 2021-05-19T02:58:04Z | CONTRIBUTOR | simonw/sqlite-utils/pulls/245 | Noticed a typo in the docs and followed that up with a spellcheck. Had to bite my tongue at some of the British spellings. |
sqlite-utils 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/245/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
857280617 | MDExOlB1bGxSZXF1ZXN0NjE0NzI3MDM2 | 254 | Fix incorrect create-table cli description | robjwells 1935268 | closed | 0 | 1 | 2021-04-13T20:03:15Z | 2021-05-19T04:43:46Z | 2021-05-19T02:57:26Z | CONTRIBUTOR | simonw/sqlite-utils/pulls/254 | The description for |
sqlite-utils 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/254/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
868191959 | MDExOlB1bGxSZXF1ZXN0NjIzNzU1NzIz | 258 | Fixing insert from JSON containing strings with non-ascii characters … | dylan-wu 6586811 | closed | 0 | 1 | 2021-04-26T20:50:00Z | 2021-05-19T02:47:44Z | 2021-05-19T02:47:44Z | CONTRIBUTOR | simonw/sqlite-utils/pulls/258 | …are escaped aps unicode for lists, tuples, dicts Fix of #257 |
sqlite-utils 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/258/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
979612115 | MDExOlB1bGxSZXF1ZXN0NzE5OTk4MjI1 | 322 | Add dict type to be mapped as TEXT in sqllite | minaeid90 2496189 | closed | 0 | 1 | 2021-08-25T20:54:26Z | 2021-11-15T00:27:40Z | 2021-11-15T00:27:40Z | CONTRIBUTOR | simonw/sqlite-utils/pulls/322 | the library deal with Postgres type jsonb as dictionary, add dict type as a TEXT for mapping to sqlite |
sqlite-utils 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/322/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
988013247 | MDExOlB1bGxSZXF1ZXN0NzI3MDEyOTk2 | 324 | Use python-dateutil package instead of dateutils | meatcar 191622 | closed | 0 | 1 | 2021-09-03T18:31:19Z | 2021-11-14T23:25:40Z | 2021-11-14T23:25:40Z | CONTRIBUTOR | simonw/sqlite-utils/pulls/324 | While working on updating In 5ec6686153e29ae10d4921a1ad4c841f192f20e2, a new dependency was added on I believe this is unintentional, and instead My reasoning is:
- Seems like the trailing "s" seems to be the source of confusion 😅 I've swapped the dependencies out, hope this helps. |
sqlite-utils 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/324/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
1044267332 | I_kwDOCGYnMM4-PkFE | 336 | sqlite-util tranform --column-order mangles columns of type "timestamp" | fgregg 536941 | closed | 0 | 1 | 2021-11-04T01:15:38Z | 2023-05-08T21:13:38Z | 2023-05-08T21:13:38Z | CONTRIBUTOR | Reproducible code below: ```bash
|
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/336/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1149729902 | PR_kwDOCGYnMM4zbaJy | 410 | Correct spelling mistakes (found with codespell) | EdwardBetts 3818 | closed | 0 | 1 | 2022-02-24T20:44:18Z | 2022-03-06T08:48:29Z | 2022-03-01T21:05:29Z | CONTRIBUTOR | simonw/sqlite-utils/pulls/410 | sqlite-utils 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/410/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||||
1178484369 | PR_kwDOCGYnMM405rPe | 419 | Ignore common generated files | eyeseast 25778 | closed | 0 | 1 | 2022-03-23T18:06:22Z | 2022-03-24T21:01:44Z | 2022-03-24T21:01:44Z | CONTRIBUTOR | simonw/sqlite-utils/pulls/419 | Closes #418 This adds four files to
Those are all generated in the course of development and testing. |
sqlite-utils 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/419/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
1244294227 | PR_kwDOCGYnMM44P4GG | 437 | docs to dogs | yurivish 114388 | closed | 0 | 1 | 2022-05-22T15:50:33Z | 2022-05-30T21:32:41Z | 2022-05-30T21:32:41Z | CONTRIBUTOR | simonw/sqlite-utils/pulls/437 | Fixes a typo. |
sqlite-utils 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/437/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
1299760627 | PR_kwDOCGYnMM47JUun | 452 | Add duplicate table feature | davidleejy 1690072 | closed | 0 | 1 | 2022-07-09T20:24:31Z | 2022-07-15T21:21:37Z | 2022-07-15T21:21:36Z | CONTRIBUTOR | simonw/sqlite-utils/pulls/452 | This PR addresses a feature request raised in issue #449. Specifically this PR adds a functionality that lets users duplicate a table via:
Test added in file Happy to make changes to meet maintainers' feedback, if any. |
sqlite-utils 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/452/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
1334416486 | PR_kwDOCGYnMM488n6D | 463 | Use Read the Docs action v1 | humitos 244656 | closed | 0 | 1 | 2022-08-10T10:31:47Z | 2022-08-18T08:30:14Z | 2022-08-17T23:11:16Z | CONTRIBUTOR | simonw/sqlite-utils/pulls/463 | Read the Docs repository was renamed from :books: Documentation preview :books:: https://sqlite-utils--463.org.readthedocs.build/en/463/ |
sqlite-utils 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/463/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
1361355564 | I_kwDOCGYnMM5RJKMs | 482 | balanced table default column_order | chapmanjacobd 7908073 | closed | 0 | 1 | 2022-09-05T03:00:18Z | 2022-10-10T17:43:02Z | 2022-09-06T20:17:27Z | CONTRIBUTOR | Is there any performance or size difference with column order in SQLITE ? similar to this https://www.cybertec-postgresql.com/en/column-order-in-postgresql-does-matter/ It might be interesting to have an option to create with an optimized column order. I'm assuming this would look something like INTEGER columns, REAL columns, BLOB columns, TEXT columns, NULL columns. NULL columns at the end because they are more likely to be TEXT and it is impossible to know if they will become INTEGER (Of course, any schema evolution would reduce optimization but maybe column order could also be re-evaluated when schema changes) edit: this is easy to accomplish with the existing
|
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/482/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1366423176 | I_kwDOCGYnMM5RcfaI | 485 | Progressbar not shown when inserting/upserting jsonlines file | MischaU8 99098079 | closed | 0 | 1 | 2022-09-08T14:13:18Z | 2022-09-15T20:39:52Z | 2022-09-15T20:37:52Z | CONTRIBUTOR | When inserting or upserting a jsonlines file, no progressbar is shown. Expected behavior is that, just like with .csv/.tsv files, also for a jsonlines file (--nl), unless --silent is provided, a progressbar is shown.
Currently |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/485/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1393212964 | I_kwDOCGYnMM5TCr4k | 497 | column_names | chapmanjacobd 7908073 | closed | 0 | 1 | 2022-10-01T03:34:21Z | 2022-10-25T21:09:28Z | 2022-10-25T21:09:28Z | CONTRIBUTOR | It would be nice to have a Or if you could get one or all of the following syntax to work for both Database and Table that might be even better: Style 1
- Style 2
- maybe the table ones actually work but I'm too lazy to check. I just know that I have to do:
Edit: This is possible with |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/497/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1430325103 | I_kwDOCGYnMM5VQQdv | 507 | conn.execute: UnicodeEncodeError: 'utf-8' codec can't encode character | chapmanjacobd 7908073 | closed | 0 | 1 | 2022-10-31T18:49:51Z | 2022-11-01T00:40:17Z | 2022-11-01T00:40:16Z | CONTRIBUTOR | I'm not really sure what caused this and it happened in the middle of my program (after running for 35775 seconds).
This might be relevant: https://stackoverflow.com/questions/31898353/python-cant-encode-with-surrogateescape I'm going to try re-running with
``` |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/507/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1505568103 | PR_kwDOCGYnMM5F609a | 519 | Fixes breaking DEFAULT values | rhoboro 13819005 | closed | 0 | 1 | 2022-12-21T01:27:52Z | 2023-05-08T21:13:37Z | 2023-05-08T21:13:37Z | CONTRIBUTOR | simonw/sqlite-utils/pulls/519 | Fixes #509, Fixes #336 Thanks for the great library!
I fixed a bug that In #509 case, fixed here. ```shell $ sqlite3 test.db << EOF CREATE TABLE mytable ( col1 TEXT DEFAULT 'foo', col2 TEXT DEFAULT (STRFTIME('%Y-%m-%d %H:%M:%f', 'NOW')) ) EOF $ sqlite3 test.db "SELECT sql FROM sqlite_master WHERE name = 'mytable';" CREATE TABLE mytable ( col1 TEXT DEFAULT 'foo', col2 TEXT DEFAULT (STRFTIME('%Y-%m-%d %H:%M:%f', 'NOW')) ) $ sqlite3 test.db "INSERT INTO mytable DEFAULT VALUES; SELECT * FROM mytable;" foo|2022-12-21 01:15:39.669 $ sqlite-utils transform test.db mytable --rename col1 renamedcol1 $ sqlite3 test.db "SELECT sql FROM sqlite_master WHERE name = 'mytable';" CREATE TABLE "mytable" ( [renamedcol1] TEXT DEFAULT 'foo', [col2] TEXT DEFAULT (STRFTIME('%Y-%m-%d %H:%M:%f', 'NOW')) # ← Non-String Value ) $ sqlite3 test.db "INSERT INTO mytable DEFAULT VALUES; SELECT * FROM mytable;" foo|2022-12-21 01:15:39.669 foo|2022-12-21 01:15:56.432 ``` And #336 case also fixed. Special values are described here.
```shell $ echo 'create table bar (baz text, created_at timestamp default CURRENT_TIMESTAMP)' | sqlite3 foo.db $ sqlite3 foo.db SQLite version 3.39.5 2022-10-14 20:58:05 Enter ".help" for usage hints. sqlite> .schema bar CREATE TABLE bar (baz text, created_at timestamp default CURRENT_TIMESTAMP); sqlite> .exit $ sqlite-utils transform foo.db bar --column-order baz $ sqlite3 foo.db SQLite version 3.39.5 2022-10-14 20:58:05 Enter ".help" for usage hints. sqlite> .schema bar CREATE TABLE IF NOT EXISTS "bar" ( [baz] TEXT, [created_at] FLOAT DEFAULT CURRENT_TIMESTAMP ); sqlite> .exit $ sqlite-utils transform foo.db bar --column-order baz $ sqlite3 foo.db SQLite version 3.39.5 2022-10-14 20:58:05 Enter ".help" for usage hints. sqlite> .schema bar CREATE TABLE IF NOT EXISTS "bar" ( [baz] TEXT, [created_at] FLOAT DEFAULT CURRENT_TIMESTAMP # ← Non-String Value ); ``` :books: Documentation preview :books:: https://sqlite-utils--519.org.readthedocs.build/en/519/ |
sqlite-utils 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/519/reactions", "total_count": 3, "+1": 3, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
1560651350 | I_kwDOCGYnMM5dBaZW | 523 | Feature request: trim all leading and trailing white space for all columns for all tables in a database | fgregg 536941 | open | 0 | 1 | 2023-01-28T02:40:10Z | 2023-01-28T02:41:14Z | CONTRIBUTOR | It's pretty common that i need to trim leading or trailing white space from lots of columns in a database a part of an initial ETL. I use the following recipe a lot, and it would be great to include this functionality into sqlite-utils
then something like:
|
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/523/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
1578793661 | PR_kwDOCGYnMM5Jqn1u | 528 | Enable `Table.convert()` on falsey values | mcarpenter 167893 | closed | 0 | 1 | 2023-02-10T00:04:09Z | 2023-05-08T21:08:23Z | 2023-05-08T21:08:23Z | CONTRIBUTOR | simonw/sqlite-utils/pulls/528 | Fixes #527 :books: Documentation preview :books:: https://sqlite-utils--528.org.readthedocs.build/en/528/ |
sqlite-utils 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/528/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
1581090327 | I_kwDOCGYnMM5ePYYX | 529 | Microsoft line endings | chapmanjacobd 7908073 | closed | 0 | 1 | 2023-02-12T02:20:48Z | 2023-06-14T23:12:12Z | 2023-06-14T23:11:47Z | CONTRIBUTOR | sqlite-utils prints It has tripped me up a few times when piping the output of sqlite-utils to other programs:
|
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/529/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1659525418 | PR_kwDOCGYnMM5N35VZ | 536 | Add paths for homebrew on Apple silicon | eyeseast 25778 | closed | 0 | 1 | 2023-04-08T13:34:21Z | 2023-04-13T01:44:43Z | 2023-04-13T01:44:43Z | CONTRIBUTOR | simonw/sqlite-utils/pulls/536 | Does what it says and nothing else. This is the same set of paths as Datasette uses. :books: Documentation preview :books:: https://sqlite-utils--536.org.readthedocs.build/en/536/ |
sqlite-utils 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/536/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
1740026046 | I_kwDOCGYnMM5ntrC- | 556 | Support storing incrementally piped values | mcint 601708 | open | 0 | 1 | 2023-06-04T00:45:23Z | 2023-06-04T01:21:15Z | CONTRIBUTOR | I'm trying to use sqlite-utils to data generated incrementally. There are a few aspects of this that I don't currently know how to handle. I would like an option to apply writes incrementally, line-by-line as they are received. I would like an option to echo incremental progress. And, it would be nice to have In particular, I'm using CoreLocationCLI -w -j to generate, newline-delimited JSON. One variant of the command
It looks like I can get what I want with:
|
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/556/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
1919296686 | PR_kwDOCGYnMM5bifPC | 596 | Fixes mapping for time fields related to mysql, closes #522 | nezhar 4420927 | closed | 0 | 1 | 2023-09-29T13:41:48Z | 2023-11-04T00:49:50Z | 2023-11-04T00:49:50Z | CONTRIBUTOR | simonw/sqlite-utils/pulls/596 | Adds :books: Documentation preview :books:: https://sqlite-utils--596.org.readthedocs.build/en/596/ |
sqlite-utils 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/596/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
1976986318 | I_kwDOCGYnMM511mrO | 599 | Cannot find spatialite on arm64 linux | MikeCoats 37802088 | closed | 0 | 1 | 2023-11-03T22:05:51Z | 2023-11-04T01:06:31Z | 2023-11-04T00:33:28Z | CONTRIBUTOR | Initially, I found an issue in
I did some digging and realised the issue originates in this project. Even with the ``` $ apt list --installed | grep spatial […] libsqlite3-mod-spatialite/stable,now 5.0.1-3 arm64 [installed] $ ls -l /usr/lib//spatial* lrwxrwxrwx 1 root root 23 Dec 1 2022 /usr/lib/aarch64-linux-gnu/mod_spatialite.so -> mod_spatialite.so.7.1.0 lrwxrwxrwx 1 root root 23 Dec 1 2022 /usr/lib/aarch64-linux-gnu/mod_spatialite.so.7 -> mod_spatialite.so.7.1.0 -rw-r--r-- 1 root root 7348584 Dec 1 2022 /usr/lib/aarch64-linux-gnu/mod_spatialite.so.7.1.0 ```
I tracked the issue down to the |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/599/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
411066700 | MDU6SXNzdWU0MTEwNjY3MDA= | 10 | Error in upsert if column named 'order' | psychemedia 82988 | closed | 0 | 1 | 2019-02-16T12:05:18Z | 2019-02-24T16:55:38Z | 2019-02-24T16:55:37Z | NONE | The following works fine: ``` connX = sqlite3.connect('DELME.db', timeout=10) dfX=pd.DataFrame({'col1':range(3),'col2':range(3)}) DBX = Database(connX) DBX['test'].upsert_all(dfX.to_dict(orient='records')) ``` But if a column is named dfX=pd.DataFrame({'order':range(3),'col2':range(3)}) DBX = Database(connX) DBX['test'].upsert_all(dfX.to_dict(orient='records')) ``` it throws an error: ```OperationalError Traceback (most recent call last) <ipython-input-130-7dba33cd806c> in <module> 3 dfX=pd.DataFrame({'order':range(3),'col2':range(3)}) 4 DBX = Database(connX) ----> 5 DBX['test'].upsert_all(dfX.to_dict(orient='records')) /usr/local/lib/python3.7/site-packages/sqlite_utils/db.py in upsert_all(self, records, pk, foreign_keys, column_order) 347 foreign_keys=foreign_keys, 348 upsert=True, --> 349 column_order=column_order, 350 ) 351 /usr/local/lib/python3.7/site-packages/sqlite_utils/db.py in insert_all(self, records, pk, foreign_keys, upsert, batch_size, column_order) 327 jsonify_if_needed(record.get(key, None)) for key in all_columns 328 ) --> 329 result = self.db.conn.execute(sql, values) 330 self.db.conn.commit() 331 self.last_id = result.lastrowid OperationalError: near "order": syntax error ``` |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/10/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
432727685 | MDU6SXNzdWU0MzI3Mjc2ODU= | 20 | JSON column values get extraneously quoted | mhalle 649467 | closed | 0 | 1.0 4348046 | 1 | 2019-04-12T20:15:30Z | 2019-05-25T00:57:19Z | 2019-05-25T00:57:19Z | NONE | If the input to ``` echo '[{"key": ["one", "two", "three"]}]' | sqlite-utils insert t.db t -sqlite-utils t.db 'select * from t'[{"key": "[\"one\", \"two\", \"three\"]"}] sqlite3 t.db 'select * from t'["one", "two", "three"] ``` This might require an imperfect solution, since sqlite3 doesn't have a JSON type. Perhaps fields that start with |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/20/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
577302229 | MDU6SXNzdWU1NzczMDIyMjk= | 91 | Enable ordering FTS results by rank | gfrmin 416374 | closed | 0 | 3.0 6079500 | 1 | 2020-03-07T08:43:51Z | 2020-11-06T23:53:26Z | 2020-11-06T23:53:25Z | NONE | According to https://www.sqlite.org/fts5.html (not sure about FTS4) results can be sorted by relevance. At the moment results are returned by default by |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/91/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
723708310 | MDU6SXNzdWU3MjM3MDgzMTA= | 188 | About loading spatialite | aborruso 30607 | closed | 0 | 1 | 2020-10-17T08:47:02Z | 2022-02-05T00:04:26Z | 2020-10-17T08:52:58Z | NONE | Hi @simonw , If I run
I have If I run
I have
How to load properly spatialite extension in sqlite-utils? Thank you very muc |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/188/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
767685961 | MDU6SXNzdWU3Njc2ODU5NjE= | 210 | Support of RData files | PeterBailey 23739126 | closed | 0 | 1 | 2020-12-15T15:04:14Z | 2021-01-02T00:02:40Z | 2021-01-02T00:02:40Z | NONE | Hi Simon, Would be great if you could ingest RData files! I could do this in a few lines of code but I am too lazy - sorry! Peter |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/210/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
836829560 | MDU6SXNzdWU4MzY4Mjk1NjA= | 248 | support for Apache Arrow / parquet files I/O | mhalle 649467 | open | 0 | 1 | 2021-03-20T14:59:30Z | 2021-10-28T23:46:48Z | NONE | I just started looking at Apache Arrow using pyarrow for import and export of tabular datasets, and it looks quite compelling. It might be worth looking at for sqlite-utils and/or datasette. As a test, I took a random jsonl data dump of a dataset I have with floats, strings, and ints and converted it to arrow's parquet format using the naive The only hangup is the automatic type inference of the naive reader. It's great for general laziness and for parsing JSON columns (it correctly interpreted a table of mine with a JSON array). However, I did get an exception for a string column where most entries looked integer-like but had a couple values that weren't -- the reader tried to coerce all of them for some reason, even though the JSON type is string. Since the writer optionally takes a schema, it shouldn't be too hard to grab the sqlite header types. With some additional hinting, you might get datetime columns and JSON, which are native Arrow types. Somewhat tangentially, someone even wrote an sqlite vfs extension for Parquet: https://cldellow.com/2018/06/22/sqlite-parquet-vtable.html |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/248/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
842062949 | MDU6SXNzdWU4NDIwNjI5NDk= | 252 | Support json-line files | rathboma 279769 | closed | 0 | 1 | 2021-03-26T15:19:39Z | 2021-03-26T15:21:38Z | 2021-03-26T15:21:38Z | NONE | It's common for many processes to create flat files where each line is a JSON object. So the file isn't a json array. Many tools (like jq) support this natively, it'd be great for sqlite-utils to also! |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/252/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
907795562 | MDU6SXNzdWU5MDc3OTU1NjI= | 265 | Using enable_fts before search term | prabhur 36287 | open | 0 | 1 | 2021-06-01T01:43:34Z | 2023-04-01T17:27:18Z | NONE | Many thanks for the sqlite-utils suite of utilities. Has made my life much much easier. I used this to create a table and enable FTS. All works fine. The datasette utility detects FTS and shows a text box. Searching for a term using that interface works well. However, when I start to use features by following https://www.sqlite.org/fts5.html section "3. Full-text Query Syntax" I seem to run into issues that I suspect is due to As an example, if i search for the term Similarly, when I try to restrict the search to a single column in FTS using a spec like
Any ideas why? How can I get the benefits of both escaping as well as utilizing different facets of providing / controlling search terms? Thanks. |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/265/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
934123448 | MDU6SXNzdWU5MzQxMjM0NDg= | 295 | Insert with --tsv and --no-headers give error about --nl arguments | davidscotson 7288187 | closed | 0 | 1 | 2021-06-30T21:01:01Z | 2021-08-18T20:19:04Z | 2021-08-18T20:18:57Z | NONE | Not quite sure if this is a bug, or just an assumption I made but I thought Instead it says:
As if it has interpreted the --no-headers as --nl. The --help does specifically say CSV:
And this heading in the documentation also only refers to CSV, but the text does mention TSV in passing, and I'd generally expect them to behave the same in most cases. https://sqlite-utils.datasette.io/en/stable/cli.html#csv-files-without-a-header-row |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/295/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
956832836 | MDU6SXNzdWU5NTY4MzI4MzY= | 300 | Returning underlying cause for User Defined Functions | wsargent 71236 | closed | 0 | 1 | 2021-07-30T15:08:21Z | 2021-08-02T21:53:50Z | 2021-08-02T21:53:50Z | NONE | The sqlite3 client takes user defined functions and replaces the text with "user-defined function raised exception`" so it's not apparent what's gone wrong:
As mentioned in https://code.djangoproject.com/ticket/29500 and https://stackoverflow.com/questions/45824209/how-to-get-an-error-kind-from-sqlite-create-function/45834923#45834923 the workaround for this is to enable callback tracebacks:
It would be nice if https://sqlite-utils.datasette.io/en/stable/python-api.html#registering-custom-sql-functions either included a reference to |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/300/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1028056713 | I_kwDOCGYnMM49RuaJ | 332 | `sqlite-utils memory --flatten` option to flatten nested JSON | rdtq 22523840 | closed | 0 | 1 | 2021-10-16T14:04:42Z | 2021-11-14T23:05:05Z | 2021-11-14T23:05:05Z | NONE | currently --flatten option works only for |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/332/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1046271107 | PR_kwDOCGYnMM4uK5z2 | 337 | Default values for `--attach` and `--param` options | urbas 771193 | closed | 0 | 1 | 2021-11-05T21:57:53Z | 2021-11-05T22:33:03Z | 2021-11-05T22:33:02Z | NONE | simonw/sqlite-utils/pulls/337 | It seems that This change makes the code forward-compatible with See this build failure for more info: https://hydra.nixos.org/build/156926608 |
sqlite-utils 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/337/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
1079422215 | I_kwDOCGYnMM5AVq0H | 357 | pytest-runner is not required | pgajdos 4067843 | closed | 0 | 1 | 2021-12-14T07:51:24Z | 2021-12-16T20:43:19Z | 2021-12-16T20:43:13Z | NONE | Deprecated pytest-runner is not necessary for running the testsuite. |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/357/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1091819089 | I_kwDOCGYnMM5BE9ZR | 360 | MemoryError | nzaar9 559453 | closed | 0 | 1 | 2022-01-01T13:39:17Z | 2022-03-21T04:22:46Z | 2022-03-21T04:22:46Z | NONE | HI, when dealing with large json file (~170GB) i got the following error
|
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/360/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1199158210 | I_kwDOCGYnMM5HebPC | 423 | .extract() doesn't set foreign key when extracted columns contain NULL value | jlieth 37447552 | closed | 0 | 1 | 2022-04-10T20:05:30Z | 2022-08-27T14:45:04Z | 2022-08-27T14:45:04Z | NONE | I've run into an issue with I'm working with a database with music listening information. Currently it has one large table A simplified demonstration with just In [2]: db = sqlite_utils.Database(memory=True) In [3]: db["listens"].insert_all([ ...: {"id": 1, "track_title": "foo", "album_title": "bar"}, ...: {"id": 2, "track_title": "baz", "album_title": None} ...: ], pk="id") Out[3]: <Table listens (id, track_title, album_title)> ``` The track in the first row has an album, the second track doesn't. Now I extract album information into a separate column: ```ipython In [4]: db["listens"].extract(columns=["album_title"], table="albums", fk_column="album_id") Out[4]: <Table listens (id, track_title, album_id)> In [5]: list(db["albums"].rows) Out[5]: [{'id': 1, 'album_title': 'bar'}, {'id': 2, 'album_title': None}] In [6]: list(db["listens"].rows) Out[6]: [{'id': 1, 'track_title': 'foo', 'album_id': 1}, {'id': 2, 'track_title': 'baz', 'album_id': None}] ``` This behaves as expected -- the Now I want to extract the track information as well. Album information belongs to the track so I want to extract both columns to a new table. ```ipython In [7]: db["listens"].extract(columns=["track_title", "album_id"], table="tracks", fk_column="track_id") Out[7]: <Table listens (id, track_id)> In [8]: list(db["tracks"].rows) Out[8]: [{'id': 1, 'track_title': 'foo', 'album_id': 1}, {'id': 2, 'track_title': 'baz', 'album_id': None}] In [9]: list(db["listens"].rows) Out[9]: [{'id': 1, 'track_id': 1}, {'id': 2, 'track_id': None}] ``` Extracting to the Changing the order of extracts doesn't help. I poked around in the source a bit and I believe this line (essentially comparing |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/423/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1212701569 | I_kwDOCGYnMM5ISFuB | 427 | sqlite-utils convert date parsing recipe complains about trying to parse "*" | wdccdw 1385831 | closed | 0 | 1 | 2022-04-22T19:27:10Z | 2022-07-02T13:59:59Z | 2022-07-02T13:59:32Z | NONE | Missing values in my dataset are denoted by a single asterisk. I am trying to parse string dates into dates. This works fine for columns without missing values, but, when the column contains "*", I get the following: ``` $ sqlite-utils convert ${dbfile} details dob 'r.parsedate(value)' [------------------------------------] 0%Traceback (most recent call last): File "/usr/local/Cellar/sqlite-utils/3.25.1/libexec/lib/python3.9/site-packages/sqlite_utils/db.py", line 2508, in convert_value return fn(v) File "<string>", line 2, in fn File "/usr/local/Cellar/sqlite-utils/3.25.1/libexec/lib/python3.9/site-packages/sqlite_utils/recipes.py", line 8, in parsedate parser.parse(value, dayfirst=dayfirst, yearfirst=yearfirst).date().isoformat() File "/usr/local/Cellar/sqlite-utils/3.25.1/libexec/lib/python3.9/site-packages/dateutil/parser/_parser.py", line 1368, in parse return DEFAULTPARSER.parse(timestr, **kwargs) File "/usr/local/Cellar/sqlite-utils/3.25.1/libexec/lib/python3.9/site-packages/dateutil/parser/_parser.py", line 643, in parse raise ParserError("Unknown string format: %s", timestr) dateutil.parser._parser.ParserError: Unknown string format: * Traceback (most recent call last): File "/usr/local/bin/sqlite-utils", line 33, in <module> sys.exit(load_entry_point('sqlite-utils==3.25.1', 'console_scripts', 'sqlite-utils')()) File "/usr/local/Cellar/sqlite-utils/3.25.1/libexec/lib/python3.9/site-packages/click/core.py", line 1128, in call return self.main(args, kwargs) File "/usr/local/Cellar/sqlite-utils/3.25.1/libexec/lib/python3.9/site-packages/click/core.py", line 1053, in main rv = self.invoke(ctx) File "/usr/local/Cellar/sqlite-utils/3.25.1/libexec/lib/python3.9/site-packages/click/core.py", line 1659, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/usr/local/Cellar/sqlite-utils/3.25.1/libexec/lib/python3.9/site-packages/click/core.py", line 1395, in invoke return ctx.invoke(self.callback, ctx.params) File "/usr/local/Cellar/sqlite-utils/3.25.1/libexec/lib/python3.9/site-packages/click/core.py", line 754, in invoke return __callback(args, **kwargs) File "/usr/local/Cellar/sqlite-utils/3.25.1/libexec/lib/python3.9/site-packages/sqlite_utils/cli.py", line 2698, in convert db[table].convert( File "/usr/local/Cellar/sqlite-utils/3.25.1/libexec/lib/python3.9/site-packages/sqlite_utils/db.py", line 2524, in convert self.db.execute(sql, where_args or []) File "/usr/local/Cellar/sqlite-utils/3.25.1/libexec/lib/python3.9/site-packages/sqlite_utils/db.py", line 458, in execute return self.conn.execute(sql, parameters) sqlite3.OperationalError: user-defined function raised exception ``` |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/427/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1303169663 | I_kwDOCGYnMM5NrMp_ | 453 | 'unclosed file' warning when using insert_upsert_implementation from Python | makkus 311257 | closed | 0 | 1 | 2022-07-13T09:34:35Z | 2022-07-15T21:52:25Z | 2022-07-15T21:52:21Z | NONE | I'm using the The warning goes away when wrapping the code from this line in a try/finally block like:
I suspect Python closes the reference automatically when the sqlite-utils cli run is done, but since my code doesn't exit, I'm getting the warning. Alternatively, it'd be cool if the 'import csv/tsv' functionality could be added directly to the Database class. |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/453/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1465194930 | PR_kwDOCGYnMM5DvZxa | 515 | upsert new rows with constraints, fixes #514 | cldellow 193185 | closed | 0 | 1 | 2022-11-26T16:15:21Z | 2023-05-08T21:27:11Z | 2023-05-08T21:27:10Z | NONE | simonw/sqlite-utils/pulls/515 | This fixes #514 by making the initial insert for upserts include all columns, so that new rows can be added to tables with non-pkey columns that have constraints. (aside: I'm not a python programmer. :books: Documentation preview :books:: https://sqlite-utils--515.org.readthedocs.build/en/515/ |
sqlite-utils 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/515/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
1622640374 | I_kwDOCGYnMM5gt4b2 | 534 | ResourceWarning: unclosed file | djhenderson 1244826 | closed | 0 | 1 | 2023-03-14T03:02:18Z | 2023-05-08T19:56:29Z | 2023-05-08T19:56:29Z | NONE | Issuing either
exhibits a ResourceWarning indicating that the CSV file being loaded is not closed. sqlite-utils --version sqlite-utils, version 3.30 py --version Python 3.11.2 Windows Version 10.0.19045 Build 19045 SQLite version 3.41.0 2023-02-21 18:09:37 |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/534/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1720096994 | I_kwDOCGYnMM5mhpji | 554 | `IndexError` when doing `.insert(..., pk='id')` after `insert_all` | xavdid 1231935 | open | 0 | 1 | 2023-05-22T17:13:02Z | 2023-05-22T17:18:33Z | NONE | I believe this is related to https://github.com/simonw/sqlite-utils/issues/98. When ```py from sqlite_utils import Database def test_pk_for_insert(fresh_db): user = {"id": "abc", "name": "david"}
if name == "main": db = Database("bug.db") if db["users"].exists(): raise ValueError( "bug only shows on a new database - remove bug.db before running the script" ) test_pk_for_insert(db) ``` The error is:
The issue is in this block: relevant locals are:
What's most interesting is the comment |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/554/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
1733198948 | I_kwDOCGYnMM5nToRk | 555 | Filter table by a large bunch of ids | redraw 10843208 | open | 0 | 1 | 2023-05-31T00:29:51Z | 2023-06-14T22:01:57Z | NONE | Hi! this might be a question related to both SQLite & sqlite-utils, and you might be more experienced with them. I have a large bunch of ids, and I'm wondering which is the best way to query them in terms of performance, and simplicity if possible. The naive approach would be something like Another approach might be creating a temp table, or in-memory db table, insert all ids in that table and then join with the target one. I failed to attach an in-memory db both using sqlite-utils, and plain sql's execute(), so my closest approach is something like,
That kinda worked, I couldn't find an option in sqlite-utils's |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/555/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
1907281675 | I_kwDOCGYnMM5xrs8L | 595 | Cascading DELETE not working with Table.delete(pk) | cycle-data 123451970 | closed | 0 | 1 | 2023-09-21T15:46:41Z | 2023-09-25T09:38:57Z | 2023-09-25T09:38:13Z | NONE | Hi ! I noticed that when I am trying to use the delete method of the Table object, the record get properly deleted from the table, but the cascading delete triggers on foreign keys do not activate.
I tried querying the database directly via DB Browser and the triggers work without any issue. Looked up the source code and behind the scene this method is just querying the database normally so I'm not exactly sure where this behavior comes from. Thank you in advance for your time ! |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/595/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1988525411 | I_kwDOCGYnMM52hn1j | 603 | Pyhton 3.12 Bug report | constantinedev 1324252 | open | 0 | 1 | 2023-11-10T22:57:48Z | 2023-12-08T05:10:31Z | NONE | I start with new python3 verison 3.12.0 Also have the error where connect DataBase
As well now of the resolved plan just keep the sqlite-utils version in python3.12 with v3.32.1 [tested] but where are the sqlite3.Connection problem.... This won't happen on python version down to 3.11[tested]
Just the python3.12.0, I have test this error are come from the sqlite3 connection
The error say from Let fix together. |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/603/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
2007893839 | I_kwDOCGYnMM53rgdP | 605 | Insert fails with `Error: Python int too large to convert to SQLite INTEGER`; can we use `NUMERIC` here? | Zac-HD 12229877 | closed | 0 | 1 | 2023-11-23T10:19:46Z | 2023-12-08T05:07:54Z | 2023-12-08T05:07:54Z | NONE | I'm currently working on a new feature for Hypothesis, where we can dump a tidy jsonlines table of all the test cases we tried - including arguments, outcomes, timings, coverage, etc. Exploring this seems like a perfect cases for I originally went to report this as a bug... and then found https://github.com/simonw/sqlite-utils/issues/309#issuecomment-895581038 almost exactly matched my repro 😅 https://github.com/simonw/sqlite-utils/issues/110#issuecomment-626391063 suggests that using After a bit more hacking, "manually cast large integers to float" seems like a decent solution for my particular case, but having written it up I thought I might as well post this issue anyway - I hope it's useful feedback, and won't mind at all if you close as wontfix if it's not. |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/605/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
349850687 | MDU6SXNzdWUzNDk4NTA2ODc= | 2 | Mechanism for adding foreign keys to an existing table | simonw 9599 | closed | 0 | 1 | 2018-08-12T22:50:56Z | 2019-02-24T21:34:41Z | 2019-02-24T21:34:41Z | OWNER | SQLite does not have ALTER TABLE support for adding new foreign keys... but it turns out it's possible to make these changes without having to duplicate the entire table by carefully running Here's how Django does it: https://github.com/django/django/blob/d3449faaa915a08c275b35de01e66a7ef6bdb2dc/django/db/backends/sqlite3/schema.py#L103-L125 And here's the official documentation about this: https://sqlite.org/lang_altertable.html#otheralter (scroll to the very bottom of the page) |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/2/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
351845423 | MDU6SXNzdWUzNTE4NDU0MjM= | 3 | Experiment with contentless FTS tables | simonw 9599 | closed | 0 | 1 | 2018-08-18T19:31:01Z | 2019-07-22T20:58:55Z | 2019-07-22T20:58:55Z | OWNER | Could greatly reduce size of resulting database for large datasets: http://cocoamine.net/blog/2015/09/07/contentless-fts4-for-large-immutable-documents/ |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/3/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
403624090 | MDU6SXNzdWU0MDM2MjQwOTA= | 6 | "sqlite-utils insert" should support newline-delimited JSON | simonw 9599 | closed | 0 | 1 | 2019-01-28T02:00:02Z | 2019-01-28T02:17:45Z | 2019-01-28T02:17:45Z | OWNER | We can already export newline delimited JSON. We should learn to import it as well. The neat thing about importing it is that you can import GBs of data without having to read the whole lot into memory in order to decode the wrapping JSON array. Datasette can export it now: https://github.com/simonw/datasette/issues/405 Demo: https://latest.datasette.io/fixtures/facetable.json?_shape=array&_nl=on It should be possible to do this:
|
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/6/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
413779210 | MDU6SXNzdWU0MTM3NzkyMTA= | 13 | Ability to automatically create IDs from content hash of row | simonw 9599 | closed | 0 | 1 | 2019-02-24T04:07:08Z | 2019-02-24T04:36:48Z | 2019-02-24T04:36:48Z | OWNER | Sometimes when you are importing data the underlying source provides records without IDs that can be uniquely identified by their contents. A utility mechanism for calculating a sha1 hash of the contents and using that as a unique ID would be useful. |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/13/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
455996809 | MDU6SXNzdWU0NTU5OTY4MDk= | 28 | Rearrange the docs by area, not CLI vs Python | simonw 9599 | closed | 0 | 1 | 2019-06-13T23:33:35Z | 2019-07-15T02:37:20Z | 2019-07-15T02:37:20Z | OWNER | The docs for eg inserting data should live on the same page, rather than being split across the API and CLI pages. |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/28/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
461215118 | MDU6SXNzdWU0NjEyMTUxMTg= | 30 | Option to open database in read-only mode | simonw 9599 | closed | 0 | 1 | 2019-06-26T22:50:38Z | 2020-05-11T19:17:17Z | 2020-05-11T19:17:17Z | OWNER | Would this make it 100% safe to run reads against a database file that is being written to by another process? |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/30/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
462094937 | MDExOlB1bGxSZXF1ZXN0MjkyODc5MjA0 | 32 | db.add_foreign_keys() method | simonw 9599 | closed | 0 | 1 | 2019-06-28T15:40:33Z | 2019-06-29T06:27:39Z | 2019-06-29T06:27:39Z | OWNER | simonw/sqlite-utils/pulls/32 | Refs #31. Still TODO:
|
sqlite-utils 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/32/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
467928674 | MDExOlB1bGxSZXF1ZXN0Mjk3NDU5Nzk3 | 40 | .get() method plus support for compound primary keys | simonw 9599 | closed | 0 | 1 | 2019-07-15T03:43:13Z | 2019-07-15T04:28:57Z | 2019-07-15T04:28:52Z | OWNER | simonw/sqlite-utils/pulls/40 |
|
sqlite-utils 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/40/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
473733752 | MDExOlB1bGxSZXF1ZXN0MzAxODI0MDk3 | 51 | Fix for too many SQL variables, closes #50 | simonw 9599 | closed | 0 | 1 | 2019-07-28T11:30:30Z | 2019-07-28T11:59:32Z | 2019-07-28T11:59:32Z | OWNER | simonw/sqlite-utils/pulls/51 | sqlite-utils 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/51/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||||
476413293 | MDU6SXNzdWU0NzY0MTMyOTM= | 52 | Throws error if .insert_all() / .upsert_all() called with empty list | simonw 9599 | closed | 0 | 1 | 2019-08-03T04:09:00Z | 2019-11-07T04:32:39Z | 2019-11-07T04:32:39Z | OWNER | sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/52/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||||
481887482 | MDExOlB1bGxSZXF1ZXN0MzA4MjkyNDQ3 | 55 | Ability to introspect and run queries against views | simonw 9599 | closed | 0 | 1 | 2019-08-17T13:40:56Z | 2019-08-23T12:19:42Z | 2019-08-23T12:19:42Z | OWNER | simonw/sqlite-utils/pulls/55 | See #54 |
sqlite-utils 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/55/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
517241040 | MDU6SXNzdWU1MTcyNDEwNDA= | 63 | ensure_index() method | simonw 9599 | closed | 0 | 1 | 2019-11-04T15:51:22Z | 2019-11-04T16:20:36Z | 2019-11-04T16:20:35Z | OWNER |
This will do the following: - if the specified table or column does not exist, do nothing - if they exist and already have an index, do nothing - otherwise, create the index I want this for tools like twitter-to-sqlite search where the |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/63/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
531583658 | MDU6SXNzdWU1MzE1ODM2NTg= | 68 | Add support for porter stemming in FTS | simonw 9599 | closed | 0 | 1 | 2019-12-02T22:35:52Z | 2020-09-20T04:25:53Z | 2020-09-20T04:25:47Z | OWNER | FTS5 can have porter stemming enabled. |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/68/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
559374410 | MDU6SXNzdWU1NTkzNzQ0MTA= | 83 | Make db["table"].exists a documented API | simonw 9599 | closed | 0 | 1 | 2020-02-03T22:31:44Z | 2020-02-08T23:58:35Z | 2020-02-08T23:56:23Z | OWNER | Right now it's a static thing which might get out-of-sync with the database. It should probably be a live check. Maybe call it |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/83/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
562911863 | MDU6SXNzdWU1NjI5MTE4NjM= | 85 | Create index doesn't work for columns containing spaces | simonw 9599 | closed | 0 | 1 | 2020-02-11T00:34:46Z | 2020-02-11T05:13:20Z | 2020-02-11T05:13:20Z | OWNER | sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/85/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||||
610853576 | MDU6SXNzdWU2MTA4NTM1NzY= | 105 | "sqlite-utils views" command | simonw 9599 | closed | 0 | 1 | 2020-05-01T16:56:11Z | 2020-05-01T20:40:07Z | 2020-05-01T20:38:36Z | OWNER | Similar to |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/105/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
611216862 | MDU6SXNzdWU2MTEyMTY4NjI= | 106 | create_view(..., ignore=True, replace=True) parameters | simonw 9599 | closed | 0 | 1 | 2020-05-02T15:45:21Z | 2020-05-02T16:04:51Z | 2020-05-02T16:02:10Z | OWNER | Two new parameters which specify what should happen if the view already exists. I want this for https://github.com/dogsheep/github-to-sqlite/issues/37 Here's the current
|
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/106/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
612658444 | MDU6SXNzdWU2MTI2NTg0NDQ= | 109 | table.create_index(..., ignore=True) | simonw 9599 | closed | 0 | 1 | 2020-05-05T14:44:21Z | 2020-05-05T14:46:53Z | 2020-05-05T14:46:53Z | OWNER | Option to silently do nothing if the index already exists. |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/109/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
637889964 | MDU6SXNzdWU2Mzc4ODk5NjQ= | 115 | Ability to execute insert/update statements with the CLI | simonw 9599 | closed | 0 | 1 | 2020-06-12T17:01:17Z | 2020-06-12T17:51:11Z | 2020-06-12T17:41:10Z | OWNER |
|
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/115/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
665802405 | MDU6SXNzdWU2NjU4MDI0MDU= | 124 | sqlite-utils query should support named parameters | simonw 9599 | closed | 0 | 1 | 2020-07-26T15:25:10Z | 2020-07-30T22:57:51Z | 2020-07-27T03:53:58Z | OWNER | To help out with escaping - so you can run this:
|
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/124/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
666639051 | MDU6SXNzdWU2NjY2MzkwNTE= | 128 | Support UUID and memoryview types | simonw 9599 | closed | 0 | 1 | 2020-07-27T23:08:34Z | 2020-07-30T01:10:43Z | 2020-07-30T01:10:43Z | OWNER |
|
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/128/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
677839979 | MDU6SXNzdWU2Nzc4Mzk5Nzk= | 133 | Release a sdist to PyPI | simonw 9599 | closed | 0 | 1 | 2020-08-12T16:55:09Z | 2020-08-12T17:05:06Z | 2020-08-12T17:05:06Z | OWNER | https://pypi.org/project/sqlite-utils/#files currently just has a wheel. I need this to package for homebrew: https://github.com/simonw/homebrew-datasette/issues/10 |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/133/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
683830416 | MDU6SXNzdWU2ODM4MzA0MTY= | 137 | --load-extension for other sqlite-utils commands | simonw 9599 | closed | 0 | 1 | 2020-08-21T21:12:56Z | 2020-10-16T19:14:32Z | 2020-10-16T19:14:32Z | OWNER | e.g. for this:
Follow-on from #134 |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/137/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
688351054 | MDU6SXNzdWU2ODgzNTEwNTQ= | 140 | Idea: insert-files mechanism for adding extra columns with fixed values | simonw 9599 | open | 0 | 1 | 2020-08-28T20:57:36Z | 2022-03-20T19:45:45Z | OWNER | Say for example you want to populate a
|
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/140/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
688389933 | MDU6SXNzdWU2ODgzODk5MzM= | 143 | Move to GitHub Actions CI | simonw 9599 | closed | 0 | 1 | 2020-08-28T22:34:11Z | 2020-08-28T22:41:35Z | 2020-08-28T22:41:35Z | OWNER | sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/143/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||||
695276328 | MDU6SXNzdWU2OTUyNzYzMjg= | 148 | More attractive indentation of created FTS table schema | simonw 9599 | closed | 0 | 1 | 2020-09-07T16:49:30Z | 2020-09-07T18:12:50Z | 2020-09-07T18:12:50Z | OWNER | On https://github-to-sqlite.dogsheep.net/github/licenses_fts the create table SQL is displayed as:
|
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/148/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
705190723 | MDU6SXNzdWU3MDUxOTA3MjM= | 160 | table.enable_fts(..., replace=True) | simonw 9599 | closed | 0 | 2.19 5896742 | 1 | 2020-09-20T21:36:23Z | 2020-09-24T20:35:47Z | 2020-09-20T22:05:51Z | OWNER | I noticed that https://til.simonwillison.net/ search doesn't use porter stemming. I'd like to add that, but since the build script always operates on an existing database (to avoid re-rendering markdown and re-building image thumbnails) I'd like it to only add porter stemming if it's not there already. So I'd like to be able to say "set up FTS to look like this, and fix it if it doesn't". I think the neatest way to do that is with a So the |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/160/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
706098005 | MDU6SXNzdWU3MDYwOTgwMDU= | 167 | Review the foreign key pragma stuff | simonw 9599 | closed | 0 | 2.20 5897911 | 1 | 2020-09-22T05:55:20Z | 2020-09-23T00:13:02Z | 2020-09-23T00:13:02Z | OWNER |
|
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/167/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
706768798 | MDU6SXNzdWU3MDY3Njg3OTg= | 170 | Release notes for 2.20 | simonw 9599 | closed | 0 | 2.20 5897911 | 1 | 2020-09-23T00:13:22Z | 2020-09-23T00:31:25Z | 2020-09-23T00:31:25Z | OWNER | sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/170/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
708301810 | MDU6SXNzdWU3MDgzMDE4MTA= | 177 | Simplify .transform(drop_foreign_keys=) and sqlite-transform --drop-foreign-key | simonw 9599 | closed | 0 | 1 | 2020-09-24T16:13:50Z | 2020-09-24T20:35:03Z | 2020-09-24T16:19:13Z | OWNER | These both currently require you to provide three strings, for Just providing |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/177/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
709861194 | MDU6SXNzdWU3MDk4NjExOTQ= | 180 | Try running some tests using Hypothesis | simonw 9599 | closed | 0 | 1 | 2020-09-28T01:11:30Z | 2020-10-19T04:51:55Z | 2020-10-19T04:51:55Z | OWNER | Inspired by this Twitter conversation: https://twitter.com/simonw/status/1310386009465479168 |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/180/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
709920027 | MDU6SXNzdWU3MDk5MjAwMjc= | 181 | pk=["id"] should have same effect as pk="id" | simonw 9599 | closed | 0 | 1 | 2020-09-28T04:28:07Z | 2020-10-14T21:59:47Z | 2020-10-14T21:59:47Z | OWNER | ``` In [11]: db['one'].insert({"id": 1, "name": "oentuh"}, pk="id") Out[11]: <Table one (id, name)> In [12]: db['two'].insert({"id": 1, "name": "oentuh"}, pk=["id"]) Out[12]: <Table two (id, name)> In [13]: db['one'].schema Out[13]: 'CREATE TABLE [one] (\n [id] INTEGER PRIMARY KEY,\n [name] TEXT\n)' In [14]: db['two'].schema Out[14]: 'CREATE TABLE [two] (\n [id] INTEGER,\n [name] TEXT\n)' ``` |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/181/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
712316959 | MDU6SXNzdWU3MTIzMTY5NTk= | 183 | Try out GitHub code scanning | simonw 9599 | closed | 0 | 1 | 2020-09-30T22:16:14Z | 2020-09-30T22:23:44Z | 2020-09-30T22:23:44Z | OWNER | https://github.blog/2020-09-30-code-scanning-is-now-available/ |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/183/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
737855731 | MDU6SXNzdWU3Mzc4NTU3MzE= | 199 | @db.register_function(..., replace=False) to avoid double-registering custom functions | simonw 9599 | closed | 0 | 1 | 2020-11-06T15:39:21Z | 2020-11-06T18:30:44Z | 2020-11-06T18:30:44Z | OWNER | I'd like a mechanism to optionally avoid registering a custom function if it has already been registered. SQLite doesn't seem to offer a way to introspect registered custom functions so I'll need to track what has already been registered in
Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/198#issuecomment-723145383 |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/199/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
738115165 | MDU6SXNzdWU3MzgxMTUxNjU= | 200 | sqlite-utils rows -c option | simonw 9599 | closed | 0 | 3.0 6079500 | 1 | 2020-11-07T00:22:12Z | 2020-11-07T00:28:48Z | 2020-11-07T00:28:47Z | OWNER | To let you specify the exact columns you want. Based on the |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/200/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
738128913 | MDU6SXNzdWU3MzgxMjg5MTM= | 201 | .search(columns=) and sqlite-utils search -c ... bug | simonw 9599 | closed | 0 | 3.0 6079500 | 1 | 2020-11-07T01:27:26Z | 2020-11-08T16:54:15Z | 2020-11-08T16:54:15Z | OWNER | Both This should be fixed before the 3.0 non-alpha release. |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/201/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
777392020 | MDU6SXNzdWU3NzczOTIwMjA= | 212 | Mechanism for maintaining cache of table counts using triggers | simonw 9599 | closed | 0 | 1 | 2021-01-02T02:58:53Z | 2021-01-02T21:40:27Z | 2021-01-02T21:40:27Z | OWNER | Counting all of the rows in a large table is expensive - this is one of the main causes of performance problems in Datasette when running against large databases. Carefully constructed SQL triggers could be used to maintain accurate cached counts for a table, by incrementing and decrementing a counter every time a row is inserted or deleted.
|
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/212/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
777540352 | MDU6SXNzdWU3Nzc1NDAzNTI= | 216 | database.triggers_dict introspection property | simonw 9599 | closed | 0 | 1 | 2021-01-02T23:13:00Z | 2021-01-03T04:27:14Z | 2021-01-03T04:25:36Z | OWNER | Following #211 |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/216/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
777560474 | MDU6SXNzdWU3Nzc1NjA0NzQ= | 218 | "sqlite-utils triggers" command | simonw 9599 | closed | 0 | 1 | 2021-01-03T02:34:50Z | 2021-01-03T03:49:51Z | 2021-01-03T03:03:35Z | OWNER | A command to list the triggers in the database.
Can optionally take one or more tables:
|
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/218/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
808046597 | MDU6SXNzdWU4MDgwNDY1OTc= | 234 | .insert_all() fails if subsequent chunks contain additional columns | simonw 9599 | closed | 0 | 1 | 2021-02-14T21:01:51Z | 2021-02-14T21:03:40Z | 2021-02-14T21:03:40Z | OWNER | Reported by @nieuwenhoven in #225 along with a proposed fix. |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/234/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
811680502 | MDU6SXNzdWU4MTE2ODA1MDI= | 236 | --attach command line option for attaching extra databases | simonw 9599 | closed | 0 | 1 | 2021-02-19T04:38:30Z | 2021-02-19T05:10:41Z | 2021-02-19T05:08:43Z | OWNER | This will enable cross-database joins, as seen in https://github.com/simonw/datasette/issues/283 Also refs #113 |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/236/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
816523763 | MDU6SXNzdWU4MTY1MjM3NjM= | 238 | .add_foreign_key() corrupts database if column contains a space | simonw 9599 | closed | 0 | 1 | 2021-02-25T15:07:20Z | 2021-02-25T16:54:02Z | 2021-02-25T16:54:02Z | OWNER | I ran this:
And got this: ``` ~/jupyter-venv/lib/python3.9/site-packages/sqlite_utils/db.py in add_foreign_keys(self, foreign_keys) 616 # Have to VACUUM outside the transaction to ensure .foreign_keys property 617 # can see the newly created foreign key. --> 618 self.vacuum() 619 620 def index_foreign_keys(self): ~/jupyter-venv/lib/python3.9/site-packages/sqlite_utils/db.py in vacuum(self) 629 630 def vacuum(self): --> 631 self.execute("VACUUM;") 632 633 ~/jupyter-venv/lib/python3.9/site-packages/sqlite_utils/db.py in execute(self, sql, parameters) 234 return self.conn.execute(sql, parameters) 235 else: --> 236 return self.conn.execute(sql) 237 238 def executescript(self, sql): DatabaseError: database disk image is malformed ``` |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/238/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
894948100 | MDU6SXNzdWU4OTQ5NDgxMDA= | 259 | Suggest the --alter option if a new column cannot be added | simonw 9599 | closed | 0 | 1 | 2021-05-19T03:17:38Z | 2021-05-19T03:27:33Z | 2021-05-19T03:26:26Z | OWNER | Refs #256. |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/259/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
919702451 | MDU6SXNzdWU5MTk3MDI0NTE= | 271 | table.upsert_all() fails if input has a single column that should be a primary key | simonw 9599 | closed | 0 | 1 | 2021-06-13T02:50:27Z | 2021-06-13T02:57:29Z | 2021-06-13T02:57:29Z | OWNER | This works: ```pycon <Table foo (name)> ``` But this fails: ``` >>> db['foo3'].upsert_all([{"name": "hello"}], pk="name") Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/Users/simon/.local/share/virtualenvs/datasette.io-TK86ygSO/lib/python3.9/site-packages/sqlite_utils/db.py", line 1837, in upsert_all return self.insert_all( File "/Users/simon/.local/share/virtualenvs/datasette.io-TK86ygSO/lib/python3.9/site-packages/sqlite_utils/db.py", line 1778, in insert_all self.insert_chunk( File "/Users/simon/.local/share/virtualenvs/datasette.io-TK86ygSO/lib/python3.9/site-packages/sqlite_utils/db.py", line 1588, in insert_chunk result = self.db.execute(query, params) File "/Users/simon/.local/share/virtualenvs/datasette.io-TK86ygSO/lib/python3.9/site-packages/sqlite_utils/db.py", line 213, in execute return self.conn.execute(sql, parameters) sqlite3.OperationalError: near "WHERE": syntax error ``` With the debugger: ``` >>> import pdb; pdb.pm() > /Users/simon/.local/share/virtualenvs/datasette.io-TK86ygSO/lib/python3.9/site-packages/sqlite_utils/db.py(213)execute() -> return self.conn.execute(sql, parameters) (Pdb) print(sql, parameters) UPDATE [foo3] SET WHERE [name] = ? ['hello'] ``` |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/271/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed |
Advanced export
JSON shape: default, array, newline-delimited, object
CREATE TABLE [issues] ( [id] INTEGER PRIMARY KEY, [node_id] TEXT, [number] INTEGER, [title] TEXT, [user] INTEGER REFERENCES [users]([id]), [state] TEXT, [locked] INTEGER, [assignee] INTEGER REFERENCES [users]([id]), [milestone] INTEGER REFERENCES [milestones]([id]), [comments] INTEGER, [created_at] TEXT, [updated_at] TEXT, [closed_at] TEXT, [author_association] TEXT, [pull_request] TEXT, [body] TEXT, [repo] INTEGER REFERENCES [repos]([id]), [type] TEXT , [active_lock_reason] TEXT, [performed_via_github_app] TEXT, [reactions] TEXT, [draft] INTEGER, [state_reason] TEXT); CREATE INDEX [idx_issues_repo] ON [issues] ([repo]); CREATE INDEX [idx_issues_milestone] ON [issues] ([milestone]); CREATE INDEX [idx_issues_assignee] ON [issues] ([assignee]); CREATE INDEX [idx_issues_user] ON [issues] ([user]);
comments 1 ✖