github
id | node_id | number | title | user | state | locked | assignee | milestone | comments | created_at | updated_at | closed_at | author_association | pull_request | body | repo | type | active_lock_reason | performed_via_github_app | reactions | draft | state_reason |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1066474200 | I_kwDOCGYnMM4_kRrY | 344 | Support STRICT tables | 9599 | closed | 0 | 14 | 2021-11-29T20:32:23Z | 2023-12-08T05:22:39Z | 2023-12-08T05:22:39Z | OWNER | New in SQLite 3.37.0, released a few days ago: https://www.sqlite.org/stricttables.html | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/344/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
2029161033 | I_kwDOCGYnMM548opJ | 606 | str and int as aliases for text and integer | 9599 | closed | 0 | 2 | 2023-12-06T18:35:49Z | 2023-12-06T19:44:04Z | 2023-12-06T18:49:32Z | OWNER | I keep making this mistake: ```bash sqlite-utils add-column content.db assets _since int ``` ``` Usage: sqlite-utils add-column [OPTIONS] PATH TABLE COL_NAME [[integer|float|b lob|text|INTEGER|FLOAT|BLOB|TEXT]] Try 'sqlite-utils add-column -h' for help. Error: Invalid value for '[[integer|float|blob|text|INTEGER|FLOAT|BLOB|TEXT]]': 'int' is not one of 'integer', 'float', 'blob', 'text', 'INTEGER', 'FLOAT', 'BLOB', 'TEXT'. ``` | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/606/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1886771493 | I_kwDOCGYnMM5wddkl | 592 | `table.transform()` should preserve `rowid` values | 9599 | closed | 0 | 6 | 2023-09-08T00:42:38Z | 2023-09-10T17:46:41Z | 2023-09-09T00:45:32Z | OWNER | I just spotted a bug when using https://datasette.io/plugins/datasette-configure-fts and https://datasette.io/plugins/datasette-edit-schema at the same time. Steps to reproduce: - Configure FTS for a table, then run a test search - Edit the schema for that table and change the order of columns - Run the test search again I got the wrong search results, which I think is because the `_fts` table pointed to the first table by `rowid` but those `rowid` values were entirely rewritten as a consequence of running `table.transform()` on the table. Reconfiguring FTS on the table fixed the problem. I think `table.transform()` should be able to preserve `rowid` values. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/592/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1857851384 | I_kwDOCGYnMM5uvI_4 | 587 | New .add_foreign_key() can break if PRAGMA legacy_alter_table=ON and there's an invalid foreign key reference | 9599 | closed | 0 | 3 | 2023-08-19T20:01:26Z | 2023-08-19T20:04:33Z | 2023-08-19T20:04:32Z | OWNER | Extremely detailed story of how I got to this point: - https://github.com/simonw/llm/issues/162 Steps to reproduce (only if that pragma is on though): ```bash python -c ' import sqlite_utils db = sqlite_utils.Database(memory=True) db.execute(""" CREATE TABLE "logs" ( [id] INTEGER PRIMARY KEY, [model] TEXT, [prompt] TEXT, [system] TEXT, [prompt_json] TEXT, [options_json] TEXT, [response] TEXT, [response_json] TEXT, [reply_to_id] INTEGER, [chat_id] INTEGER REFERENCES [log]([id]), [duration_ms] INTEGER, [datetime_utc] TEXT ); """) db["logs"].add_foreign_key("reply_to_id", "logs", "id") ' ``` This succeeds in some environments, fails in others. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/587/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1817289521 | I_kwDOCGYnMM5sUaMx | 577 | Get `add_foreign_keys()` to work without modifying `sqlite_master` | 9599 | closed | 0 | 9 | 2023-07-23T20:40:18Z | 2023-08-18T17:43:11Z | 2023-08-18T00:48:10Z | OWNER | https://github.com/simonw/sqlite-utils/blob/13ebcc575d2547c45e8d31288b71a3242c16b886/sqlite_utils/db.py#L1165-L1174 This is the only place in the code that attempts to modify `sqlite_master` directly, which fails on some Python installations. Could this use the `.transform()` trick instead? Or automatically switch to that trick if it hits an error? | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/577/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1855894222 | I_kwDOCGYnMM5unrLO | 585 | CLI equivalents to `transform(add_foreign_keys=)` | 9599 | closed | 0 | 7 | 2023-08-18T01:07:15Z | 2023-08-18T01:51:16Z | 2023-08-18T01:51:15Z | OWNER | The new options added in: - #577 Deserve consideration in the CLI as well. https://github.com/simonw/sqlite-utils/blob/d2bcdc00c6ecc01a6e8135e775ffdb87572b802b/sqlite_utils/db.py#L1706-L1708 | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/585/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1855836914 | I_kwDOCGYnMM5undLy | 583 | Get rid of test.utils.collapse_whitespace | 9599 | closed | 0 | 1 | 2023-08-17T23:31:09Z | 2023-08-18T00:59:19Z | 2023-08-18T00:59:19Z | OWNER | I have a neater pattern for this now - instead of: https://github.com/simonw/sqlite-utils/blob/1dc6b5aa644a92d3654f7068110ed7930989ce71/tests/test_create.py#L472-L475 I now prefer: https://github.com/simonw/sqlite-utils/blob/1dc6b5aa644a92d3654f7068110ed7930989ce71/tests/test_create.py#L1163-L1171 | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/583/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1823160748 | I_kwDOCGYnMM5sqzms | 581 | `sqlite-utils convert --pdb` option | 9599 | closed | 0 | 1 | 2023-07-26T21:02:50Z | 2023-07-26T21:07:45Z | 2023-07-26T21:06:10Z | OWNER | While using `sqlite-utils convert` I realized it would be handy if you could pass `--pdb` to have it open the debugger at the first instance of a failed conversion. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/581/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1816997390 | I_kwDOCGYnMM5sTS4O | 576 | Backfill the release notes prior to 0.4 | 9599 | closed | 0 | 2 | 2023-07-23T05:41:42Z | 2023-07-23T05:49:51Z | 2023-07-23T05:48:21Z | OWNER | Currently the changelog starts at 0.4: https://sqlite-utils.datasette.io/en/3.34/changelog.html#id115 I want the other releases - according to https://pypi.org/project/sqlite-utils/#history there are three missing: <img width="663" alt="image" src="https://github.com/simonw/sqlite-utils/assets/9599/4ebc036b-7bb1-477c-95c1-a2c7e26bcb62"> | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/576/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1816919568 | I_kwDOCGYnMM5sS_4Q | 575 | Python API ability to opt-out of connection plugins | 9599 | closed | 0 | 2 | 2023-07-22T23:01:13Z | 2023-07-22T23:17:22Z | 2023-07-22T23:08:22Z | OWNER | Plugins affecting the CLI by default makes sense to me. I'm less confident about them _always_ affecting users of the Python API. I'm going to have them apply by default, but I'm going to add a mechanism to opt-out on an individual database basis. Basically this: ```python from sqlite_utils import Database db = Database(memory=True, execute_plugins=False) # Anything using db from here on will not execute plugins ``` cc @asg017 Refs: - #567 - #574 | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/575/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1816918185 | I_kwDOCGYnMM5sS_ip | 574 | `prepare_connection()` plugin hook | 9599 | closed | 0 | 3 | 2023-07-22T22:52:47Z | 2023-07-22T23:13:14Z | 2023-07-22T22:59:10Z | OWNER | > Splitting off an issue for `prepare_connection()` since Alex got the PR in seconds before I shipped 3.34! _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/567#issuecomment-1646686424_ | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/574/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1816876211 | I_kwDOCGYnMM5sS1Sz | 571 | `.transform(keep_table=...)` option | 9599 | closed | 0 | 1 | 2023-07-22T19:49:29Z | 2023-07-22T22:32:18Z | 2023-07-22T22:32:18Z | OWNER | >> Also need a design for an option for the `.transform()` method to indicate that the new table should be created with a new name without dropping the old one. > > I think `keep_table="name_of_table"` is good for this. _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/565#issuecomment-1646657324_ | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/571/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1816877910 | I_kwDOCGYnMM5sS1tW | 572 | Don't test Python 3.7 against textual | 9599 | closed | 0 | 2 | 2023-07-22T19:57:03Z | 2023-07-22T22:16:50Z | 2023-07-22T22:16:50Z | OWNER | Spotted this in the GitHub Actions logs: ![IMG_5046](https://github.com/simonw/sqlite-utils/assets/9599/81fb1093-cd8a-4019-a612-2e49b500c933) | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/572/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1786243905 | I_kwDOCGYnMM5qd-tB | 564 | Document that running `db.transform()` tidies up the schema indentation | 9599 | closed | 0 | 0 | 2023-07-03T13:59:28Z | 2023-07-22T22:15:34Z | 2023-07-22T22:15:34Z | OWNER | > ... and it turns out running `.transform()` with no arguments still fixes the format of the schema! ```pycon >>> db["log"].add_column("foo", str) <Table log (id, name2, age, weight, foo)> >>> db["log"].add_column("bar", str) <Table log (id, name2, age, weight, foo, bar)> >>> db["log"].add_column("baz", str) <Table log (id, name2, age, weight, foo, bar, baz)> >>> print(db["log"].schema) CREATE TABLE "log" ( [id] INTEGER PRIMARY KEY, [name2] TEXT, [age] INTEGER, [weight] FLOAT , [foo] TEXT, [bar] TEXT, [baz] TEXT) >>> db["log"].transform() <Table log (id, name2, age, weight, foo, bar, baz)> >>> print(db["log"].schema) CREATE TABLE "log" ( [id] INTEGER PRIMARY KEY, [name2] TEXT, [age] INTEGER, [weight] FLOAT, [foo] TEXT, [bar] TEXT, [baz] TEXT ) ``` _Originally posted by @simonw in https://github.com/simonw/llm/issues/65#issuecomment-1618347727_ | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/564/reactions", "total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 1 } |
completed | ||||||
1205687423 | I_kwDOCGYnMM5H3VR_ | 426 | CLI docs should link to Python docs and vice versa | 9599 | closed | 0 | 9599 | 1 | 2022-04-15T16:05:15Z | 2023-07-22T22:13:22Z | 2023-07-22T22:13:22Z | OWNER | For every command/API method there should be a link to the equivalent in the other form factor. Maybe also link to the API and CLI reference pages too. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/426/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
1786258502 | I_kwDOCGYnMM5qeCRG | 565 | Table renaming: db.rename_table() and sqlite-utils rename-table | 9599 | closed | 0 | 6 | 2023-07-03T14:07:42Z | 2023-07-22T22:12:40Z | 2023-07-22T22:12:40Z | OWNER | > I find myself wanting two new features in `sqlite-utils`: > - The ability to have the new transformed table set to a specific name, while keeping the old table around > - The ability to rename a table (`sqlite-utils` doesn't have a table rename function at all right now) _Originally posted by @simonw in https://github.com/simonw/llm/issues/65#issuecomment-1618375042_ | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/565/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1816851056 | I_kwDOCGYnMM5sSvJw | 568 | table.create(..., replace=True) | 9599 | closed | 0 | 7 | 2023-07-22T18:12:22Z | 2023-07-22T19:25:35Z | 2023-07-22T19:15:44Z | OWNER | Found myself using this pattern to quickly prototype a schema: ```python import sqlite_utils db = sqlite_utils.Database(memory=True) print(db["answers_chunks"].create({ "id": int, "content": str, "embedding_type_id": int, "embedding": bytes, "embedding_content_md5": str, "source": str, }, pk="id", transform=True).schema) ``` <img width="472" alt="image" src="https://github.com/simonw/datasette-answers/assets/9599/30237f7c-4964-4190-984c-d824c57939a9"> Using `replace=True` to drop and then recreate the table would be neat here, and would be consistent with other places that use `replace=True`. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/568/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1816852402 | I_kwDOCGYnMM5sSvey | 569 | register_command plugin hook | 9599 | closed | 0 | 3 | 2023-07-22T18:17:27Z | 2023-07-22T19:19:35Z | 2023-07-22T19:19:35Z | OWNER | > I'm going to start by adding the `register_command` hook using the exact same pattern as Datasette and LLM. _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/567#issuecomment-1646643450_ | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/569/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1816857105 | I_kwDOCGYnMM5sSwoR | 570 | `sqlite-utils install -e` option | 9599 | closed | 0 | 0 | 2023-07-22T18:32:23Z | 2023-07-22T18:55:59Z | 2023-07-22T18:32:56Z | OWNER | As seen in LLM. Needed while working on: - #567 | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/570/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1785360409 | I_kwDOCGYnMM5qanAZ | 563 | `--empty-null` option when importing CSV | 9599 | closed | 0 | 1 | 2023-07-03T05:23:36Z | 2023-07-03T05:44:43Z | 2023-07-03T05:42:30Z | OWNER | CSV files with empty cells in (which come through as the empty string) are common and a bit gross. Having an option that means "and if it's an empty string store `null` instead) would be cool. I brainstormed name options here https://chat.openai.com/share/c947b738-ee7d-419c-af90-bc84e90987da | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/563/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1777548699 | I_kwDOCGYnMM5p8z2b | 561 | `--stop-after` option for `insert` and `upsert` commands | 9599 | closed | 0 | 1 | 2023-06-27T18:44:15Z | 2023-06-27T18:50:09Z | 2023-06-27T18:50:08Z | OWNER | I found myself wanting to insert rows from a 849MB CSV file without processing the whole thing: https://huggingface.co/datasets/jerpint-org/HackAPrompt-Playground-Submissions/tree/main | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/561/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1773450152 | I_kwDOCGYnMM5ptLOo | 559 | sqlean support | 9599 | closed | 0 | 0 | 2023-06-25T19:27:26Z | 2023-06-25T23:25:53Z | 2023-06-25T23:25:53Z | OWNER | If sqlean is available, use that. Refs: - https://github.com/nalgeon/sqlean.py/issues/1#issuecomment-1605707788 This will provide a good workaround for: - #235 | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/559/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1718612569 | I_kwDOCGYnMM5mb_JZ | 552 | Document how to setup shell auto-completion | 9599 | closed | 0 | 1 | 2023-05-21T19:20:41Z | 2023-05-21T21:05:16Z | 2023-05-21T21:03:40Z | OWNER | https://click.palletsprojects.com/en/8.1.x/shell-completion/ This works for `zsh`: eval "$(_SQLITE_UTILS_COMPLETE=zsh_source sqlite-utils)" This will probably work for `bash`: eval "$(_SQLITE_UTILS_COMPLETE=bash_source sqlite-utils)" Need to add this to the installation docs here: https://sqlite-utils.datasette.io/en/stable/installation.html - along with the pattern for adding that to `.zshrc` or whatever. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/552/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1718607907 | I_kwDOCGYnMM5mb-Aj | 551 | Make as many examples in the CLI docs as possible copy-and-pastable | 9599 | closed | 0 | 6 | 2023-05-21T19:04:10Z | 2023-05-21T21:04:04Z | 2023-05-21T20:57:24Z | OWNER | e.g. in this section: https://sqlite-utils.datasette.io/en/stable/cli.html#running-queries-directly-against-csv-or-json <img width="771" alt="image" src="https://github.com/simonw/sqlite-utils/assets/9599/3935111d-441e-4edb-9100-ce210117dbe5"> The little copy button will also copy the `$ ` which breaks the examples when copied. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/551/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1718517882 | I_kwDOCGYnMM5mboB6 | 545 | Try out Trogon for a tui interface | 9599 | closed | 0 | 6 | 2023-05-21T14:08:25Z | 2023-05-21T19:33:13Z | 2023-05-21T18:41:58Z | OWNER | https://github.com/Textualize/trogon | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/545/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1718595700 | I_kwDOCGYnMM5mb7B0 | 550 | AttributeError: 'EntryPoints' object has no attribute 'get' for flake8 on Python 3.7 | 9599 | closed | 0 | 3 | 2023-05-21T18:24:39Z | 2023-05-21T18:42:25Z | 2023-05-21T18:41:58Z | OWNER | https://github.com/simonw/sqlite-utils/actions/runs/5039064797/jobs/9036965488 ``` Traceback (most recent call last): File "/opt/hostedtoolcache/Python/3.7.16/x64/bin/flake8", line 8, in <module> sys.exit(main()) File "/opt/hostedtoolcache/Python/3.7.16/x64/lib/python3.7/site-packages/flake8/main/cli.py", line 22, in main app.run(argv) File "/opt/hostedtoolcache/Python/3.7.16/x64/lib/python3.7/site-packages/flake8/main/application.py", line 363, in run self._run(argv) File "/opt/hostedtoolcache/Python/3.7.16/x64/lib/python3.7/site-packages/flake8/main/application.py", line 350, in _run self.initialize(argv) File "/opt/hostedtoolcache/Python/3.7.16/x64/lib/python3.7/site-packages/flake8/main/application.py", line 330, in initialize self.find_plugins(config_finder) File "/opt/hostedtoolcache/Python/3.7.16/x64/lib/python3.7/site-packages/flake8/main/application.py", line 153, in find_plugins self.check_plugins = plugin_manager.Checkers(local_plugins.extension) File "/opt/hostedtoolcache/Python/3.7.16/x64/lib/python3.7/site-packages/flake8/plugins/manager.py", line 357, in __init__ self.namespace, local_plugins=local_plugins File "/opt/hostedtoolcache/Python/3.7.16/x64/lib/python3.7/site-packages/flake8/plugins/manager.py", line 238, in __init__ self._load_entrypoint_plugins() File "/opt/hostedtoolcache/Python/3.7.16/x64/lib/python3.7/site-packages/flake8/plugins/manager.py", line 254, in _load_entrypoint_plugins eps = importlib_metadata.entry_points().get(self.namespace, ()) AttributeError: 'EntryPoints' object has no attribute 'get' Error: Process completed with exit code 1. ``` | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/550/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1718576761 | I_kwDOCGYnMM5mb2Z5 | 548 | analyze-tables should validate provide --column names | 9599 | closed | 0 | 1 | 2023-05-21T17:20:24Z | 2023-05-21T17:35:52Z | 2023-05-21T17:35:52Z | OWNER | Noticed this while testing: - #547 If you pass a non-existent column to `-c/--column` you don't get an error message. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/548/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1718572201 | I_kwDOCGYnMM5mb1Sp | 547 | No need to show common values if everything is null | 9599 | closed | 0 | 1 | 2023-05-21T17:05:07Z | 2023-05-21T17:19:21Z | 2023-05-21T17:19:21Z | OWNER | Noticed this: ``` % sqlite-utils analyze-tables content.db repos -c delete_branch_on_merge --common-limit 20 --no-least repos.delete_branch_on_merge: (1/1) Total rows: 158 Null rows: 158 Blank rows: 0 Distinct values: 0 Most common: 158: None ``` The `158: None` there is duplicate information considering we already know there are 158/158 null rows. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/547/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1718515590 | I_kwDOCGYnMM5mbneG | 544 | New options for analyze-tables --common-limit --no-most and --no-least | 9599 | closed | 0 | 2 | 2023-05-21T14:03:19Z | 2023-05-21T17:03:06Z | 2023-05-21T16:19:31Z | OWNER | The "least common" section is frequently uninteresting, especially for huge tables with a large number of repeated-once values. sqlite-utils analyze-tables content.db repos --common-limit 20 --no-least | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/544/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1701018909 | I_kwDOCGYnMM5lY30d | 543 | Tests broken on Windows due to new convert() lambda names | 9599 | closed | 0 | 0 | 2023-05-08T22:11:29Z | 2023-05-08T22:19:04Z | 2023-05-08T22:19:04Z | OWNER | https://github.com/simonw/sqlite-utils/actions/runs/4920084038/jobs/8788501314 ```python sql = 'update [example] set [dt] = lambda_-9223371942137158589([dt]);' ``` From: - #526 | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/543/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1516644980 | I_kwDOCGYnMM5aZip0 | 520 | rows_from_file() raises confusing error if file-like object is not in binary mode | 9599 | closed | 0 | 3 | 2023-01-02T19:00:14Z | 2023-05-08T22:08:07Z | 2023-05-08T22:08:07Z | OWNER | I got this error: ``` File "/Users/simon/Dropbox/Development/openai-to-sqlite/openai_to_sqlite/cli.py", line 27, in embeddings rows, _ = rows_from_file(input) ^^^^^^^^^^^^^^^^^^^^^ File "/Users/simon/.local/share/virtualenvs/openai-to-sqlite-jt4obeb2/lib/python3.11/site-packages/sqlite_utils/utils.py", line 305, in rows_from_file first_bytes = buffered.peek(2048).strip() ^^^^^^^^^^^^^^^^^^^ ``` From this code: ```python @cli.command() @click.argument( "db_path", type=click.Path(file_okay=True, dir_okay=False, allow_dash=False), ) @click.option( "-i", "--input", type=click.File("r"), default="-", ) def embeddings(db_path, input): "Store embeddings for one or more text documents" click.echo("Here is some output") db = sqlite_utils.Database(db_path) rows, _ = rows_from_file(input) print(list(rows)) ``` The error went away when I changed it to `type=click.File("rb")`. This should either be called out in the documentation or `rows_from_file()` should be fixed to handle text-mode files in addition to binary files. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/520/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1699184583 | I_kwDOCGYnMM5lR3_H | 540 | sphinx.builders.linkcheck build error | 9599 | closed | 0 | 4 | 2023-05-07T18:37:09Z | 2023-05-08T04:56:13Z | 2023-05-07T18:42:36Z | OWNER | https://readthedocs.org/projects/sqlite-utils/builds/20512693/ ``` Running Sphinx v6.2.1 Traceback (most recent call last): File "/home/docs/checkouts/readthedocs.org/user_builds/sqlite-utils/envs/latest/lib/python3.8/site-packages/sphinx/registry.py", line 442, in load_extension mod = import_module(extname) File "/home/docs/checkouts/readthedocs.org/user_builds/sqlite-utils/envs/latest/lib/python3.8/importlib/__init__.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "<frozen importlib._bootstrap>", line 1014, in _gcd_import File "<frozen importlib._bootstrap>", line 991, in _find_and_load File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 671, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 783, in exec_module File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed File "/home/docs/checkouts/readthedocs.org/user_builds/sqlite-utils/envs/latest/lib/python3.8/site-packages/sphinx/builders/linkcheck.py", line 20, in <module> from requests import Response File "/home/docs/checkouts/readthedocs.org/user_builds/sqlite-utils/envs/latest/lib/python3.8/site-packages/requests/__init__.py", line 43, in <module> import urllib3 File "/home/docs/checkouts/readthedocs.org/user_builds/sqlite-utils/envs/latest/lib/python3.8/site-packages/urllib3/__init__.py", line 38, in <module> raise ImportError( ImportError: urllib3 v2.0 only supports OpenSSL 1.1.1+, currently the 'ssl' module is compiled with OpenSSL 1.0.2n 7 Dec 2017. See: https://github.com/urllib3/urllib3/issues/2168 The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/home/docs/checkouts/readthedocs.org/user_builds/sqlite-utils/envs/latest/lib/python3.8/site-packages/sphinx/cmd/build.py", line 280, in build_main app = Sphinx(args.sourcedir, args.confdir, args.outputdir, File "/… | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/540/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1699174055 | I_kwDOCGYnMM5lR1an | 539 | `--raw-lines` option, like `--raw` for multiple lines | 9599 | closed | 0 | 4 | 2023-05-07T18:07:46Z | 2023-05-07T18:43:24Z | 2023-05-07T18:26:18Z | OWNER | I wanted to output newline-separated output of the first column of every row in the results - like `--row` but for more than one line. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/539/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1203842656 | I_kwDOCGYnMM5HwS5g | 425 | `sqlite3.NotSupportedError`: deterministic=True requires SQLite 3.8.3 or higher | 9599 | closed | 0 | 5 | 2022-04-13T22:16:53Z | 2023-04-15T20:14:58Z | 2022-04-13T22:48:57Z | OWNER | Got this error while investigating: - #421 Even though I was using the `LD_PRELOAD` trick from https://til.simonwillison.net/sqlite/ld-preload to use a newer version of SQLite. _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/421#issuecomment-1098531354_ | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/425/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1114543475 | I_kwDOCGYnMM5CbpVz | 388 | Link to stable docs from older versions | 9599 | closed | 0 | 7 | 2022-01-26T01:55:46Z | 2023-03-26T23:43:12Z | 2022-01-26T02:00:22Z | OWNER | https://sqlite-utils.datasette.io/en/2.14.1/ isn't showing a link to the stable release right now. I should also apply the same fix I used for Datasette in: - https://github.com/simonw/datasette/issues/1608 TIL: https://til.simonwillison.net/readthedocs/link-from-latest-to-stable | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/388/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1620516340 | I_kwDOCGYnMM5glx30 | 533 | ReadTheDocs error: not all arguments converted during string formatting | 9599 | closed | 0 | 2 | 2023-03-12T21:21:05Z | 2023-03-12T21:25:33Z | 2023-03-12T21:25:33Z | OWNER | This came up as a failure running tests for: - #531 Traceback on https://readthedocs.org/projects/sqlite-utils/builds/19749348/ ``` File "/home/docs/checkouts/readthedocs.org/user_builds/sqlite-utils/envs/531/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 889, in interpreted nodes, messages2 = role_fn(role, rawsource, text, lineno, self) File "/home/docs/checkouts/readthedocs.org/user_builds/sqlite-utils/envs/531/lib/python3.8/site-packages/sphinx/ext/extlinks.py", line 103, in role title = caption % part TypeError: not all arguments converted during string formatting Exception occurred: File "/home/docs/checkouts/readthedocs.org/user_builds/sqlite-utils/envs/531/lib/python3.8/site-packages/sphinx/ext/extlinks.py", line 103, in role title = caption % part TypeError: not all arguments converted during string formatting ``` | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/533/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1487764628 | I_kwDOCGYnMM5YrXyU | 518 | flake8 ValueError: Error code '#' supplied to 'extend-ignore' option... | 9599 | closed | 0 | 0 | 2022-12-10T01:30:24Z | 2022-12-10T01:36:46Z | 2022-12-10T01:36:46Z | OWNER | > `Error code '#' supplied to 'extend-ignore' option does not match '^[A-Z]{1,3}[0-9]{0,3}$'` https://github.com/simonw/sqlite-utils/actions/runs/3662011265/jobs/6190770361 I think from this: https://github.com/simonw/sqlite-utils/blob/e660635cea6c32f4022818380b1e1ee88e7c93a6/setup.cfg#L1-L3 | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/518/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1487757143 | I_kwDOCGYnMM5YrV9X | 517 | Drop support for Python 3.6 | 9599 | closed | 0 | 1 | 2022-12-10T01:23:31Z | 2022-12-10T01:36:36Z | 2022-12-10T01:36:36Z | OWNER | CI has started failing for Python 3.6: https://github.com/simonw/sqlite-utils/actions/runs/3576322798 <img width="945" alt="image" src="https://user-images.githubusercontent.com/9599/206821688-e4176cb3-ea82-460d-8e93-626fb4fa0269.png"> It's fixable by swiching away from `ubuntu-latest` according to: - https://github.com/actions/setup-python/issues/355#issuecomment-1335042510 But https://endoflife.date/python says that 3.6 end of life was almost 6 years ago, and end of security support nearly 1 year ago. So I'm OK dropping support entirely - Python 3.6 users will still be able to install version 3.30, just not any releases that come next. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/517/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1450952393 | I_kwDOCGYnMM5We8bJ | 512 | mypy failures in CI | 9599 | closed | 0 | 3 | 2022-11-16T06:22:48Z | 2022-11-16T07:49:51Z | 2022-11-16T07:49:50Z | OWNER | https://github.com/simonw/sqlite-utils/actions/runs/3472012235 failed on Python 3.11: Truncated output: ``` sqlite_utils/db.py:2467: note: PEP 484 prohibits implicit Optional. Accordingly, mypy has changed its default to no_implicit_optional=True sqlite_utils/db.py:2467: note: Use https://github.com/hauntsaninja/no_implicit_optional to automatically upgrade your codebase sqlite_utils/db.py:2530: error: Incompatible default for argument "where" (default has type "None", argument has type "str") [assignment] sqlite_utils/db.py:2530: note: PEP 484 prohibits implicit Optional. Accordingly, mypy has changed its default to no_implicit_optional=True sqlite_utils/db.py:2530: note: Use https://github.com/hauntsaninja/no_implicit_optional to automatically upgrade your codebase sqlite_utils/db.py:2658: error: Argument 1 to "count_where" of "Queryable" has incompatible type "Optional[str]"; expected "str" [arg-type] Found 23 errors in 1 file (checked 51 source files) ``` Best look at https://github.com/hauntsaninja/no_implicit_optional | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/512/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
473083260 | MDU6SXNzdWU0NzMwODMyNjA= | 50 | "Too many SQL variables" on large inserts | 9599 | closed | 0 | 4 | 2019-07-25T21:43:31Z | 2022-11-04T14:38:36Z | 2019-07-28T11:59:33Z | OWNER | Reported here: https://github.com/dogsheep/healthkit-to-sqlite/issues/9 It looks like there's a default limit of 999 variables - we need to be smart about that, maybe dynamically lower the batch size based on the number of columns. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/50/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1429029604 | I_kwDOCGYnMM5VLULk | 506 | Make `cursor.rowcount` accessible (wontfix) | 9599 | closed | 0 | 3 | 2022-10-30T21:51:55Z | 2022-11-01T17:37:47Z | 2022-11-01T17:37:13Z | OWNER | In building this Datasette feature on top of `sqlite-utils` I thought it might be useful to expose the number of rows that had been affected by a bulk insert or update - the `cursor.rowcount`: - https://github.com/simonw/datasette/issues/1866 This isn't currently exposed by `sqlite-utils`. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/506/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1423182778 | I_kwDOCGYnMM5U1Au6 | 505 | Release sqlite-utils 3.30 | 9599 | closed | 0 | 2 | 2022-10-25T22:20:05Z | 2022-10-25T22:41:26Z | 2022-10-25T22:41:16Z | OWNER | https://github.com/simonw/sqlite-utils/compare/3.29...defa2974c6d3abc19be28d6b319649b8028dc966 | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/505/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1423069384 | I_kwDOCGYnMM5U0lDI | 504 | db.close() method, calling db.conn.close() | 9599 | closed | 0 | 1 | 2022-10-25T20:50:50Z | 2022-10-25T21:00:29Z | 2022-10-25T20:57:47Z | OWNER | I ended up needing to use `db.conn.close()` to fix this issue: - #503 I think `.close()` should be a method on `Database` itself. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/504/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1423000702 | I_kwDOCGYnMM5U0UR- | 503 | test_recreate failing on Windows Python 3.11 | 9599 | closed | 0 | 10 | 2022-10-25T20:01:41Z | 2022-10-25T20:47:34Z | 2022-10-25T20:45:43Z | OWNER | https://github.com/simonw/sqlite-utils/actions/runs/3323672128/jobs/5494726927 Related: - #502 ``` FAILED tests/test_recreate.py::test_recreate[True-True] - PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\runneradmin\\AppData\\Local\\Temp\\pytest-of-runneradmin\\pytest-0\\test_recreate_True_True_0\\data.db' FAILED tests/test_recreate.py::test_recreate[False-True] - PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\runneradmin\\AppData\\Local\\Temp\\pytest-of-runneradmin\\pytest-0\\test_recreate_False_True_0\\data.db' ``` | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/503/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1422954582 | I_kwDOCGYnMM5U0JBW | 502 | Fix tests for Python 3.11 | 9599 | closed | 0 | 1 | 2022-10-25T19:20:31Z | 2022-10-25T19:23:47Z | 2022-10-25T19:23:47Z | OWNER | The way errors are represented has changed: https://github.com/simonw/sqlite-utils/actions/runs/3323588047/jobs/5494127154 ``` _________________________ test_query_invalid_function __________________________ db_path = '/tmp/pytest-of-runner/pytest-0/test_query_invalid_function0/test.db' def test_query_invalid_function(db_path): result = CliRunner().invoke( cli.cli, [db_path, "select bad()", "--functions", "def invalid_python"] ) assert result.exit_code == 1 > assert ( result.output.strip() == "Error: Error in functions definition: invalid syntax (<string>, line 1)" ) E AssertionError: assert 'Error: Error...ing>, line 1)' == 'Error: Error...ing>, line 1)' E - Error: Error in functions definition: invalid syntax (<string>, line 1) E ? ^^^^^^ ^^^^^^ E + Error: Error in functions definition: expected '(' (<string>, line 1) E ? ^^^^^^^ ^^^ ``` | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/502/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1413641049 | I_kwDOCGYnMM5UQnNZ | 501 | Tests failing due to updated tabulate library | 9599 | closed | 0 | 4 | 2022-10-18T18:07:52Z | 2022-10-18T18:23:40Z | 2022-10-18T18:23:40Z | OWNER | Failure here: https://github.com/simonw/sqlite-utils/actions/runs/3275786702/jobs/5391063221 I figured out the problem: ```diff diff --git a/docs/cli-reference.rst b/docs/cli-reference.rst index b88e38a..82b4b6c 100644 --- a/docs/cli-reference.rst +++ b/docs/cli-reference.rst @@ -112,11 +112,15 @@ See :ref:`cli_query`. --tsv Output TSV --no-headers Omit CSV headers -t, --table Output as a formatted table - --fmt TEXT Table format - one of fancy_grid, fancy_outline, - github, grid, html, jira, latex, latex_booktabs, - latex_longtable, latex_raw, mediawiki, moinmoin, - orgtbl, pipe, plain, presto, pretty, psql, rst, - simple, textile, tsv, unsafehtml, youtrack + --fmt TEXT Table format - one of asciidoc, double_grid, + double_outline, fancy_grid, fancy_outline, github, + grid, heavy_grid, heavy_outline, html, jira, + latex, latex_booktabs, latex_longtable, latex_raw, + mediawiki, mixed_grid, mixed_outline, moinmoin, + orgtbl, outline, pipe, plain, presto, pretty, + psql, rounded_grid, rounded_outline, rst, simple, + simple_grid, simple_outline, textile, tsv, + unsafehtml, youtrack --json-cols Detect JSON cols and output them as JSON, not escaped strings -r, --raw Raw output, first column of first row @@ -176,11 +180,15 @@ See :ref:`cli_memory`. --tsv Output TSV --no-headers Omit CSV headers -t, --table Output as a formatte… | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/501/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1413610718 | I_kwDOCGYnMM5UQfze | 500 | Turn --flatten into a documented utility function | 9599 | closed | 0 | 4 | 2022-10-18T17:43:36Z | 2022-10-18T18:02:10Z | 2022-10-18T18:00:40Z | OWNER | The `--flatten` implementation isn't currently available to Python code - people have to roll their own implementation. Feedback from a conversation at DjangoCon. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/500/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1386593843 | I_kwDOCGYnMM5Spb4z | 494 | Document how to use Just | 9599 | closed | 0 | 2 | 2022-09-26T19:25:12Z | 2022-09-26T19:32:36Z | 2022-09-26T19:26:39Z | OWNER | I'm using `just` a lot know, based on this file - I should add that to https://sqlite-utils.datasette.io/en/latest/contributing.html https://github.com/simonw/sqlite-utils/blob/afbd2b2cba45cccb305c3d4638d18db4dd3d4bbd/Justfile#L1-L24 | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/494/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1363765916 | I_kwDOCGYnMM5RSWqc | 483 | `sqlite-utils install` command | 9599 | closed | 0 | 2 | 2022-09-06T20:13:55Z | 2022-09-26T19:04:43Z | 2022-09-26T18:57:15Z | OWNER | With the addition of `--functions` in: - #471 In addition to the existing `convert` command, there are now very good reasons to want to install additional packages into the same virtual environment as `sqlite-utils` itself, to allow them to be used with those features. This isn't easy if you installed the tool with `pipx` or `brew install sqlite-utils`. Datasette solved this problem with the `datasette install` command: - https://github.com/simonw/datasette/issues/925 `sqlite-utils` could benefit from the same idea. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/483/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1352932716 | I_kwDOCGYnMM5QpB1s | 471 | sqlite-utils query --functions mechanism for registering extra functions | 9599 | closed | 0 | 8355157 | 12 | 2022-08-27T03:57:53Z | 2022-09-07T03:46:26Z | 2022-08-27T05:10:57Z | OWNER | It would be really cool if you could register additional custom SQL functions for use with the `sqlite-utils query` command - something like this: ``` sqlite-utils data.db 'update images set domain = extract_domain(url)' --functions ' from urllib.parse import urlparse def extract_domain(url): return urlparse(url).netloc ' ``` Every function defined in that code block would be registered with the connection, unless the name began with an underscore. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/471/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
1353196970 | I_kwDOCGYnMM5QqCWq | 476 | Release notes for 3.29 | 9599 | closed | 0 | 8355157 | 2 | 2022-08-27T23:21:21Z | 2022-08-28T04:07:15Z | 2022-08-28T04:07:03Z | OWNER | https://github.com/simonw/sqlite-utils/compare/3.28...104f37fa4d2e7e5999c1d829267b62c737f74d3e | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/476/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
1348169997 | I_kwDOCGYnMM5QW3EN | 467 | Mechanism for ensuring a table has all the columns | 9599 | closed | 0 | 8355157 | 13 | 2022-08-23T15:50:23Z | 2022-08-27T23:19:41Z | 2022-08-27T23:17:56Z | OWNER | Suggested by @jefftriplett on Discord: https://discord.com/channels/823971286308356157/997738192360964156/1011655389063958600 | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/467/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
1353189941 | I_kwDOCGYnMM5QqAo1 | 475 | table.default_values introspection property | 9599 | closed | 0 | 8355157 | 1 | 2022-08-27T22:33:31Z | 2022-08-27T22:44:46Z | 2022-08-27T22:43:02Z | OWNER | > Interesting challenge with `default_value`: I need to be able to tell if the default values passed to `.create()` differ from those in the database already. > > Introspecting that is a bit tricky: > > ```pycon > >>> import sqlite_utils > >>> db = sqlite_utils.Database(memory=True) > >>> db["blah"].create({"id": int, "name": str}, not_null=("name",), defaults={"name": "bob"}) > <Table blah (id, name)> > >>> db["blah"].columns > [Column(cid=0, name='id', type='INTEGER', notnull=0, default_value=None, is_pk=0), Column(cid=1, name='name', type='TEXT', notnull=1, default_value="'bob'", is_pk=0)] > ``` > Note how a default value of the Python string `bob` is represented in the results of `PRAGMA table_info()` as `default_value="'bob'"` - it's got single quotes added to it! > > So comparing default values from introspecting the database needs me to first parse that syntax. This may require a new table introspection method. _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/468#issuecomment-1229279539_ | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/475/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
1352932038 | I_kwDOCGYnMM5QpBrG | 470 | Upgrade `--load-extension` to accept entrypoints like Datasette | 9599 | closed | 0 | 8355157 | 6 | 2022-08-27T03:53:20Z | 2022-08-27T05:55:49Z | 2022-08-27T05:55:48Z | OWNER | Imitate: - https://github.com/simonw/datasette/pull/1789 ``` # would load default entrypoint like before datasette data.db --load-extension ext # loads the extensions with the "sqlite3_foo_init" entrpoint datasette data.db --load-extension ext:sqlite3_foo_init # loads the extensions with the "sqlite3_bar_init" entrpoint datasette data.db --load-extension ext:sqlite3_bar_init ``` | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/470/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
1352946135 | I_kwDOCGYnMM5QpFHX | 472 | Reuse the locals/globals fix from --functions for other code accepting options | 9599 | closed | 0 | 8355157 | 2 | 2022-08-27T05:12:05Z | 2022-08-27T05:20:12Z | 2022-08-27T05:20:12Z | OWNER | I figured out a workaround for the ugly `global x` hack here: - https://github.com/simonw/sqlite-utils/issues/471#issuecomment-1229120653 | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/472/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
1352931464 | I_kwDOCGYnMM5QpBiI | 469 | sqlite-utils rows --order option | 9599 | closed | 0 | 8355157 | 1 | 2022-08-27T03:49:51Z | 2022-08-27T04:30:49Z | 2022-08-27T04:10:32Z | OWNER | For consistency with `search`: https://sqlite-utils.datasette.io/en/stable/cli-reference.html#search ``` -o, --order TEXT Order by ('column' or 'column desc') ``` I wanted to run `sqlite-utils rows db.db mytable --order 'rowid desc'` to see the most recently imported rows. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/469/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
1320243134 | I_kwDOCGYnMM5OsU-- | 458 | Support custom names for registered functions | 9599 | closed | 0 | 8355157 | 1 | 2022-07-28T00:13:00Z | 2022-08-27T03:56:01Z | 2022-07-28T00:13:57Z | OWNER | In this example: ```python @db.register_function def reverse_string(s): return "".join(reversed(list(s))) print(db.execute('select reverse_string("hello")').fetchone()[0]) ``` There's currently no way to over-ride the automatically selected name for the SQL function. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/458/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
1338001039 | I_kwDOCGYnMM5PwEaP | 464 | Link from documentation to source code | 9599 | closed | 0 | 5 | 2022-08-13T16:19:57Z | 2022-08-17T23:38:03Z | 2022-08-17T23:38:03Z | OWNER | Twitter conversation asking for ways to automate this here: https://twitter.com/simonw/status/1558260492015046656 | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/464/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1292060682 | I_kwDOCGYnMM5NA0gK | 450 | Add --ignore option to more commands | 9599 | closed | 0 | 9 | 2022-07-02T13:52:02Z | 2022-07-15T22:39:09Z | 2022-07-15T22:37:45Z | OWNER | As seen in https://sqlite-utils.datasette.io/en/stable/cli-reference.html#add-foreign-key Could make this TIL trick unnecessary: https://til.simonwillison.net/bash/ignore-errors | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/450/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1298531653 | I_kwDOCGYnMM5NZgVF | 451 | Make sqlite_utils.utils.chunks a documented function | 9599 | closed | 0 | 2 | 2022-07-08T06:01:04Z | 2022-07-15T22:09:34Z | 2022-07-15T21:59:33Z | OWNER | I want to use it in another project: https://github.com/simonw/sqlite-utils/blob/8a9fe6498faf783a1fdeb1793e661ad194a05267/sqlite_utils/utils.py#L471-L474 | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/451/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1306548397 | I_kwDOCGYnMM5N4Fit | 454 | CLI command for duplicating tables | 9599 | closed | 0 | 1 | 2022-07-15T21:31:27Z | 2022-07-15T21:48:23Z | 2022-07-15T21:45:51Z | OWNER | CLI equivalent of: - #449 | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/454/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1277328147 | I_kwDOCGYnMM5MInsT | 446 | Use Just to automate running tests and linters locally | 9599 | closed | 0 | 2 | 2022-06-20T19:51:09Z | 2022-06-21T19:28:35Z | 2022-06-20T19:54:50Z | OWNER | I keep committing code that fails additional tests like `mypy` and `flake8` and `black`. Automate those using Just. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/446/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1278571700 | I_kwDOCGYnMM5MNXS0 | 447 | Incorrect syntax highlighting in docs CLI reference | 9599 | closed | 0 | 3 | 2022-06-21T14:53:10Z | 2022-06-21T18:48:47Z | 2022-06-21T18:48:46Z | OWNER | https://sqlite-utils.datasette.io/en/stable/cli-reference.html#insert ![CE020DDA-27FB-49C3-9EA6-37457DC4C321](https://user-images.githubusercontent.com/9599/174830380-06530537-b870-41c0-a8af-03c7fa720c6f.jpeg) It looks like Python keywords are being incorrectly highlighted here. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/447/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1269998342 | I_kwDOCGYnMM5LsqMG | 443 | Make `utils.rows_from_file()` a documented API | 9599 | closed | 0 | 2 | 2022-06-13T21:53:24Z | 2022-06-20T19:49:37Z | 2022-06-14T20:12:46Z | OWNER | > `rows_from_file()` isn't part of the documented API but maybe it should be! _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/440#issuecomment-1154385916_ | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/443/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1277295119 | I_kwDOCGYnMM5MIfoP | 445 | `sqlite_utils.utils.TypeTracker` should be a documented API | 9599 | closed | 0 | 3 | 2022-06-20T19:08:28Z | 2022-06-20T19:49:02Z | 2022-06-20T19:46:58Z | OWNER | I've used it in a couple of external places now: - https://github.com/simonw/datasette-socrata/blob/32fb256a461bf0e790eca10bdc7dd9d96c20f7c4/datasette_socrata/__init__.py#L264-L280 - https://github.com/simonw/datasette-lite/blob/caa8eade10f0321c64f9f65c4561186f02d57c5b/webworker.js#L55-L64 Refs: - https://github.com/simonw/datasette-lite/issues/32 | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/445/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1269886084 | I_kwDOCGYnMM5LsOyE | 442 | `maximize_csv_field_size_limit()` utility function | 9599 | closed | 0 | 2 | 2022-06-13T19:54:54Z | 2022-06-14T21:55:15Z | 2022-06-14T21:31:49Z | OWNER | This code here runs only if `cli.py` is imported: https://github.com/simonw/sqlite-utils/blob/7ddf5300886a32d6daf60cf1d71efe492b65c87e/sqlite_utils/cli.py#L50-L59 I found myself needing the same fix in another library: - https://github.com/simonw/datasette-socrata/issues/13 It should be a documented utility function. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/442/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1243715381 | I_kwDOCGYnMM5KIZc1 | 436 | Add "copy to clipboard" button to code examples in documentation | 9599 | closed | 0 | 0 | 2022-05-20T21:53:23Z | 2022-05-20T21:57:53Z | 2022-05-20T21:57:53Z | OWNER | Follows: - #435 Imitates: - https://github.com/simonw/datasette/issues/1748 I'll use https://github.com/executablebooks/sphinx-copybutton - here's the Datasette commit: https://github.com/simonw/datasette/commit/1465fea4798599eccfe7e8f012bd8d9adfac3039 | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/436/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1243704847 | I_kwDOCGYnMM5KIW4P | 435 | Switch to Furo documentation theme | 9599 | closed | 0 | 2 | 2022-05-20T21:46:39Z | 2022-05-20T21:56:10Z | 2022-05-20T21:54:43Z | OWNER | As seen in: - https://github.com/simonw/datasette/issues/1746 - https://github.com/simonw/shot-scraper/issues/77 | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/435/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1200866134 | I_kwDOCGYnMM5Hk8NW | 424 | Better error message if you try to create a table with no columns | 9599 | closed | 0 | 1 | 2022-04-12T02:43:20Z | 2022-04-13T22:40:15Z | 2022-04-13T22:40:10Z | OWNER | Seen here: - https://github.com/simonw/geojson-to-sqlite/issues/30 Attempting to create a table with no columns produced this confusing error: ``` File "/Users/simon/.local/pipx/venvs/geojson-to-sqlite/lib/python3.9/site-packages/geojson_to_sqlite/utils.py", line 69, in import_features db[table].create(column_types, pk=pk) File "/Users/simon/.local/pipx/venvs/geojson-to-sqlite/lib/python3.9/site-packages/sqlite_utils/db.py", line 863, in create self.db.create_table( File "/Users/simon/.local/pipx/venvs/geojson-to-sqlite/lib/python3.9/site-packages/sqlite_utils/db.py", line 517, in create_table self.execute(sql) File "/Users/simon/.local/pipx/venvs/geojson-to-sqlite/lib/python3.9/site-packages/sqlite_utils/db.py", line 236, in execute return self.conn.execute(sql) sqlite3.OperationalError: near ")": syntax error ``` | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/424/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1166731361 | I_kwDOCGYnMM5Fiuhh | 414 | I forgot to include the changelog in the 3.25.1 release | 9599 | closed | 0 | 7 | 2022-03-11T18:32:36Z | 2022-03-11T18:40:39Z | 2022-03-11T18:40:39Z | OWNER | I pushed a release for https://github.com/simonw/sqlite-utils/releases/tag/3.25.1 but forgot to include the release notes in `docs/changelog.rst` This means https://sqlite-utils.datasette.io/en/stable/changelog.html isn't showing them. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/414/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1166587040 | I_kwDOCGYnMM5FiLSg | 413 | Display autodoc type information more legibly | 9599 | closed | 0 | 5 | 2022-03-11T15:58:20Z | 2022-03-11T18:07:10Z | 2022-03-11T18:07:10Z | OWNER | https://sqlite-utils.datasette.io/en/3.25/reference.html#sqlite_utils.db.Table.insert looks like this at the moment: <img width="703" alt="image" src="https://user-images.githubusercontent.com/9599/157902622-368935a8-93f2-42e9-98ad-94a45c818e80.png"> | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/413/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1128139375 | I_kwDOCGYnMM5DPgpv | 405 | `Database(memory_name="name")` constructor argument | 9599 | closed | 0 | 2 | 2022-02-09T07:15:03Z | 2022-02-16T01:23:16Z | 2022-02-16T01:23:16Z | OWNER | SQLite in-memory databases can be named, in which case multiple connections can be opened to a shared in-memory database running within the same process. Datasette supports this - SQLite could support it too. https://docs.datasette.io/en/0.60.2/internals.html#database-ds-path-none-is-mutable-false-is-memory-false-memory-name-none | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/405/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1128120451 | I_kwDOCGYnMM5DPcCD | 404 | Add example of `--convert` to the help for `sqlite-utils insert` | 9599 | closed | 0 | 2 | 2022-02-09T06:49:09Z | 2022-02-09T06:56:35Z | 2022-02-09T06:55:16Z | OWNER | https://sqlite-utils.datasette.io/en/3.23/cli-reference.html#insert would be more useful if it included an example of `--convert` in action. I can maybe use an example from https://simonwillison.net/2022/Jan/11/sqlite-utils/ | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/404/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1125081640 | I_kwDOCGYnMM5DD2Io | 401 | Update SpatiaLite example in the documentation | 9599 | closed | 0 | 2 | 2022-02-06T02:02:07Z | 2022-02-06T02:05:03Z | 2022-02-06T02:03:24Z | OWNER | This one here: https://sqlite-utils.datasette.io/en/3.23/python-api.html#converting-column-values-using-sql-functions It should take advantage of the new methods from: - #79 | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/401/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1125077063 | I_kwDOCGYnMM5DD1BH | 400 | `sqlite-utils create-table` ... `--if-not-exists` | 9599 | closed | 0 | 1 | 2022-02-06T01:32:53Z | 2022-02-06T01:34:53Z | 2022-02-06T01:34:46Z | OWNER | Inspired by: - #397 To match the option on `create-index`: https://sqlite-utils.datasette.io/en/stable/cli-reference.html#create-index ``` --if-not-exists Ignore if index already exists ``` | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/400/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
683805434 | MDU6SXNzdWU2ODM4MDU0MzQ= | 135 | Code for finding SpatiaLite in the usual locations | 9599 | closed | 0 | 3 | 2020-08-21T20:15:34Z | 2022-02-05T00:04:26Z | 2020-08-21T20:30:13Z | OWNER | I built this for `shapefile-to-sqlite` but it would be useful in `sqlite-utils` too: https://github.com/simonw/shapefile-to-sqlite/blob/e754d0747ca2facf9a7433e2d5d15a6a37a9cf6e/shapefile_to_sqlite/utils.py#L16-L19 ```python SPATIALITE_PATHS = ( "/usr/lib/x86_64-linux-gnu/mod_spatialite.so", "/usr/local/lib/mod_spatialite.dylib", ) ``` https://github.com/simonw/shapefile-to-sqlite/blob/e754d0747ca2facf9a7433e2d5d15a6a37a9cf6e/shapefile_to_sqlite/utils.py#L105-L109 ```python def find_spatialite(): for path in SPATIALITE_PATHS: if os.path.exists(path): return path return None ``` | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/135/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
683812642 | MDU6SXNzdWU2ODM4MTI2NDI= | 136 | --load-extension=spatialite shortcut option | 9599 | closed | 0 | 3 | 2020-08-21T20:31:25Z | 2022-02-05T00:04:26Z | 2020-10-16T19:14:32Z | OWNER | In conjunction with #135 - this would do the same thing as `--load-extension=path-to-spatialite` (see #134) | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/136/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
723460107 | MDU6SXNzdWU3MjM0NjAxMDc= | 187 | Maybe: Utility method / CLI tool for initializing SpatiaLite | 9599 | closed | 0 | 2 | 2020-10-16T19:04:03Z | 2022-02-05T00:04:26Z | 2020-10-16T19:15:13Z | OWNER | > I think this should initialize SpatiaLite against the current database if it has not been initialized already. > > Relevant code: https://github.com/simonw/shapefile-to-sqlite/blob/e754d0747ca2facf9a7433e2d5d15a6a37a9cf6e/shapefile_to_sqlite/utils.py#L112-L126 | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/187/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
557825032 | MDU6SXNzdWU1NTc4MjUwMzI= | 77 | Ability to insert data that is transformed by a SQL function | 9599 | closed | 0 | 2 | 2020-01-30T23:45:55Z | 2022-02-05T00:04:25Z | 2020-01-31T00:24:32Z | OWNER | I want to be able to run the equivalent of this SQL insert: ```python # Convert to "Well Known Text" format wkt = shape(geojson['geometry']).wkt # Insert and commit the record conn.execute("INSERT INTO places (id, name, geom) VALUES(null, ?, GeomFromText(?, 4326))", ( "Wales", wkt )) conn.commit() ``` From the Datasette SpatiaLite docs: https://datasette.readthedocs.io/en/stable/spatialite.html To do this, I need a way of telling `sqlite-utils` that a specific column should be wrapped in `GeomFromText(?, 4326)`. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/77/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
557842245 | MDU6SXNzdWU1NTc4NDIyNDU= | 79 | Helper methods for working with SpatiaLite | 9599 | closed | 0 | 8 | 2020-01-31T00:39:19Z | 2022-02-05T00:04:25Z | 2022-02-04T05:55:11Z | OWNER | As demonstrated by this piece of documentation, using SpatiaLite with sqlite-utils requires a fair bit of boilerplate: https://github.com/simonw/sqlite-utils/blob/f7289174e66ae4d91d57de94bbd9d09fabf7aff4/docs/python-api.rst#L880-L909 | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/79/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1123849278 | I_kwDOCGYnMM5C_JQ- | 395 | "apt-get: command not found" error on macOS | 9599 | closed | 0 | 1 | 2022-02-04T06:03:42Z | 2022-02-04T06:10:58Z | 2022-02-04T06:10:58Z | OWNER | Yeah, `apt-get` isn't a thing on macOS so 4a2a3e2fd0d5534f446b3f1fee34cb165e4d86d2 (to test #79 against real SpatiaLite) broke. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/395/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1123851690 | I_kwDOCGYnMM5C_J2q | 396 | mypy failure, sqlite_utils/utils.py:56 | 9599 | closed | 0 | 0 | 2022-02-04T06:08:09Z | 2022-02-04T06:10:33Z | 2022-02-04T06:10:33Z | OWNER | https://github.com/simonw/sqlite-utils/runs/5062725880?check_suite_focus=true > `sqlite_utils/utils.py:56: error: Incompatible return value type (got "None", expected "str")` | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/396/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1094981339 | I_kwDOCGYnMM5BRBbb | 363 | Better error message if `--convert` code fails to return a dict | 9599 | closed | 0 | 4 | 2022-01-06T05:26:28Z | 2022-02-03T22:52:30Z | 2022-02-03T22:51:30Z | OWNER | Here's the traceback if your `--convert` function doesn't return a dict right now: ``` % sqlite-utils insert /tmp/all.db blah /tmp/log.log --convert 'all.upper()' --all Traceback (most recent call last): File "/Users/simon/.local/share/virtualenvs/sqlite-utils-C4Ilevlm/bin/sqlite-utils", line 33, in <module> sys.exit(load_entry_point('sqlite-utils', 'console_scripts', 'sqlite-utils')()) File "/Users/simon/.local/share/virtualenvs/sqlite-utils-C4Ilevlm/lib/python3.8/site-packages/click/core.py", line 1137, in __call__ return self.main(*args, **kwargs) File "/Users/simon/.local/share/virtualenvs/sqlite-utils-C4Ilevlm/lib/python3.8/site-packages/click/core.py", line 1062, in main rv = self.invoke(ctx) File "/Users/simon/.local/share/virtualenvs/sqlite-utils-C4Ilevlm/lib/python3.8/site-packages/click/core.py", line 1668, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/Users/simon/.local/share/virtualenvs/sqlite-utils-C4Ilevlm/lib/python3.8/site-packages/click/core.py", line 1404, in invoke return ctx.invoke(self.callback, **ctx.params) File "/Users/simon/.local/share/virtualenvs/sqlite-utils-C4Ilevlm/lib/python3.8/site-packages/click/core.py", line 763, in invoke return __callback(*args, **kwargs) File "/Users/simon/Dropbox/Development/sqlite-utils/sqlite_utils/cli.py", line 949, in insert insert_upsert_implementation( File "/Users/simon/Dropbox/Development/sqlite-utils/sqlite_utils/cli.py", line 834, in insert_upsert_implementation db[table].insert_all( File "/Users/simon/Dropbox/Development/sqlite-utils/sqlite_utils/db.py", line 2602, in insert_all first_record = next(records) File "/Users/simon/Dropbox/Development/sqlite-utils/sqlite_utils/db.py", line 3044, in fix_square_braces for record in records: File "/Users/simon/Dropbox/Development/sqlite-utils/sqlite_utils/cli.py", line 831, in <genexpr> docs = (decode_base64_values(doc) for doc in docs) File "/Users/simon/Dropbox/Development/s… | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/363/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1118585417 | I_kwDOCGYnMM5CrEJJ | 393 | Better documentation for insert-replace | 9599 | closed | 0 | 1 | 2022-01-30T15:40:23Z | 2022-02-03T22:13:24Z | 2022-02-03T22:13:24Z | OWNER | Currently: https://sqlite-utils.datasette.io/en/stable/python-api.html#insert-replacing-data > If you want to insert a record or replace an existing record with the same primary key, using the replace=True argument to .insert() or .insert_all(): Should describe the exception you get first, then how to use replace to avoid it. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/393/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1097091527 | I_kwDOCGYnMM5BZEnH | 369 | Research how much of a difference analyze / sqlite_stat1 makes | 9599 | closed | 0 | 11 | 2022-01-09T03:03:36Z | 2022-02-03T21:07:41Z | 2022-02-03T21:07:35Z | OWNER | > Is there a downside to having a `sqlite_stat1` table if it has wildly incorrect statistics in it? _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/365#issuecomment-1008163050_ More generally: how much of a difference does the `sqlite_stat1` table created by `ANALYZE` make to queries? I'm particularly interested in `group by` / `count *` queries since Datasette uses those for faceting. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/369/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1114640101 | I_kwDOCGYnMM5CcA7l | 392 | `sqlite-utils bulk --batch-size` option | 9599 | closed | 0 | 4 | 2022-01-26T05:17:11Z | 2022-01-26T18:17:59Z | 2022-01-26T18:17:59Z | OWNER | > Could add support for `--batch-size` as seen in `insert`/`upsert` too - causing it to break the list up into batches and commit for each one. _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/391#issuecomment-1021876055_ | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/392/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1114638930 | I_kwDOCGYnMM5CcApS | 391 | `sqlite-utils bulk` progress bar | 9599 | closed | 0 | 2 | 2022-01-26T05:14:49Z | 2022-01-26T05:17:20Z | 2022-01-26T05:16:51Z | OWNER | It can easily have a progress bar because it works by looping through an iterator: https://github.com/simonw/sqlite-utils/blob/a9fca7efa4184fbb2a65ca1275c326950ed9d3c1/sqlite_utils/cli.py#L1014-L1018 Should also support the `--silent` option if I add this. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/391/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1114557284 | I_kwDOCGYnMM5Cbstk | 390 | `sqlite-utils upsert` should require `--pk` more elegantly | 9599 | closed | 0 | 1 | 2022-01-26T02:20:31Z | 2022-01-26T03:20:25Z | 2022-01-26T03:19:43Z | OWNER | Currently throws an ugly traceback: ``` % echo '[ {"id": 1, "name": "Lila"}, {"id": 1, "name": "Lila"} ]' | sqlite-utils upsert data.db chickens - Traceback (most recent call last): File "/Users/simon/.local/share/virtualenvs/sqlite-utils-C4Ilevlm/bin/sqlite-utils", line 33, in <module> sys.exit(load_entry_point('sqlite-utils', 'console_scripts', 'sqlite-utils')()) File "/Users/simon/.local/share/virtualenvs/sqlite-utils-C4Ilevlm/lib/python3.8/site-packages/click/core.py", line 1137, in __call__ return self.main(*args, **kwargs) File "/Users/simon/.local/share/virtualenvs/sqlite-utils-C4Ilevlm/lib/python3.8/site-packages/click/core.py", line 1062, in main rv = self.invoke(ctx) File "/Users/simon/.local/share/virtualenvs/sqlite-utils-C4Ilevlm/lib/python3.8/site-packages/click/core.py", line 1668, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/Users/simon/.local/share/virtualenvs/sqlite-utils-C4Ilevlm/lib/python3.8/site-packages/click/core.py", line 1404, in invoke return ctx.invoke(self.callback, **ctx.params) File "/Users/simon/.local/share/virtualenvs/sqlite-utils-C4Ilevlm/lib/python3.8/site-packages/click/core.py", line 763, in invoke return __callback(*args, **kwargs) File "/Users/simon/Dropbox/Development/sqlite-utils/sqlite_utils/cli.py", line 1104, in upsert insert_upsert_implementation( File "/Users/simon/Dropbox/Development/sqlite-utils/sqlite_utils/cli.py", line 906, in insert_upsert_implementation db[table].insert_all( File "/Users/simon/Dropbox/Development/sqlite-utils/sqlite_utils/db.py", line 2615, in insert_all raise PrimaryKeyRequired("upsert() requires a pk") sqlite_utils.db.PrimaryKeyRequired: upsert() requires a pk ``` | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/390/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1099897648 | I_kwDOCGYnMM5Bjxsw | 384 | Add examples to every `--help` | 9599 | closed | 0 | 0 | 2022-01-12T05:31:25Z | 2022-01-26T03:15:02Z | 2022-01-26T03:15:02Z | OWNER | Everything on https://sqlite-utils.datasette.io/en/stable/cli-reference.html would benefit from an example. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/384/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
471818939 | MDU6SXNzdWU0NzE4MTg5Mzk= | 48 | Jupyter notebook demo of the library, launchable on Binder | 9599 | closed | 0 | 2 | 2019-07-23T17:05:05Z | 2022-01-26T02:08:46Z | 2022-01-26T02:08:39Z | OWNER | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/48/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||||
1114544727 | I_kwDOCGYnMM5CbppX | 389 | Plausible analytics for documentation | 9599 | closed | 0 | 2 | 2022-01-26T01:58:35Z | 2022-01-26T02:07:41Z | 2022-01-26T02:07:41Z | OWNER | ```html <script defer data-domain="sqlite-utils.datasette.io" src="https://plausible.io/js/plausible.js"></script> ``` _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/388#issuecomment-1021785268_ | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/389/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1111293050 | I_kwDOCGYnMM5CPPx6 | 387 | Python library docs should start with a self contained example | 9599 | closed | 0 | 1 | 2022-01-22T06:23:56Z | 2022-01-26T01:37:17Z | 2022-01-26T01:35:30Z | OWNER | You have to read a lot of stuff in a lot of different places to get started with the Python library. Add a getting started introduction to https://sqlite-utils.datasette.io/en/stable/python-api.html | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/387/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1107557831 | I_kwDOCGYnMM5CA_3H | 386 | Better "contributing" documentation | 9599 | closed | 0 | 0 | 2022-01-19T02:11:48Z | 2022-01-19T02:15:21Z | 2022-01-19T02:15:21Z | OWNER | This page jumps straight into running the tests: https://sqlite-utils.datasette.io/en/latest/contributing.html It should add a little more about expected collaboration styles - opening an issue before filing a pull request - and probably link to https://simonwillison.net/2022/Jan/12/how-i-build-a-feature/ | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/386/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1067771698 | I_kwDOCGYnMM4_pOcy | 348 | Command for creating an empty database | 9599 | closed | 0 | 7558727 | 6 | 2021-11-30T23:24:27Z | 2022-01-13T07:06:59Z | 2022-01-09T20:33:20Z | OWNER | I sometimes find the need to create an empty SQLite database file - for example if I want to enable WAL on it before using it with another script. I currently do that like this: sqlite3 my.db vacuum sqlite-utils enable-wal my.db It would be nice if `sqlite-utils` had a convenience command for doing this. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/348/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
1099584685 | I_kwDOCGYnMM5BilSt | 381 | `sqlite-utils rows` options `--limit` and `--offset` | 9599 | closed | 0 | 2 | 2022-01-11T20:23:12Z | 2022-01-11T23:33:37Z | 2022-01-11T23:19:36Z | OWNER | Because I often want to use it just to preview a few rows from the database. Piping through `| head -n 20` works for JSON and CSV (they stream) but not for `--table`. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/381/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1099585611 | I_kwDOCGYnMM5BilhL | 382 | `--where` option for `sqlite-rows` | 9599 | closed | 0 | 1 | 2022-01-11T20:24:23Z | 2022-01-11T23:33:14Z | 2022-01-11T23:32:47Z | OWNER | CLI equivalent of `table.rows_where()` - should accept parameters too. Work on this at the same time as #381. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/382/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1099586786 | I_kwDOCGYnMM5Bilzi | 383 | Add documentation page with the output of `--help` | 9599 | closed | 0 | 4 | 2022-01-11T20:25:58Z | 2022-01-11T22:55:05Z | 2022-01-11T21:44:05Z | OWNER | Can be maintained using `cog` from #373. Similar in purpose to the API reference page, but this is for the CLI. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/383/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1098574572 | I_kwDOCGYnMM5Beurs | 380 | Release notes for 3.21 | 9599 | closed | 0 | 7558727 | 1 | 2022-01-11T02:12:30Z | 2022-01-11T02:34:26Z | 2022-01-11T02:34:26Z | OWNER | For these commits: https://github.com/simonw/sqlite-utils/compare/3.20...129141572f249ea290e2a075437e2ebaad215859 | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/380/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
1097251014 | I_kwDOCGYnMM5BZrjG | 375 | `sqlite-utils bulk` command | 9599 | closed | 0 | 7558727 | 3 | 2022-01-09T17:12:38Z | 2022-01-11T02:12:58Z | 2022-01-11T02:10:55Z | OWNER | The `.executemany()` method is a very efficient way to execute the same SQL query against a huge list of parameters. `sqlite-utils insert` supports a bunch of ways of loading a list of dictionaries - from CSV, TSV, JSON, newline JSON and more thanks to: - #361 What if you could load a list of dictionaries and provide a SQL query with `:named` parameters that correspond to keys in those dictionaries instead? This would need to be a new command - I thought about adding a `--sql` option to `insert` but that doesn't make sense as that command already requires a table name. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/375/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
1098544628 | I_kwDOCGYnMM5BenX0 | 379 | CLI options for running ANALYZE | 9599 | closed | 0 | 7558727 | 0 | 2022-01-11T01:09:16Z | 2022-01-11T01:38:01Z | 2022-01-11T01:36:48Z | OWNER | > The Python methods are all done now, next step is the CLI options. I'll do those in a separate issue. _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/366#issuecomment-1009508865_ - [x] `sqlite-utils analyze` command - [x] `sqlite-utils create-index --analyze` option (see #365) - [x] `sqlite-utils insert --analyze` option - [x] `sqlite-utils upsert --analyze` option In #378 I also added `.delete_where(..., analyze=True)` but there isn't currently a `sqlite-utils delete-where` CLI command - deletions via CLI are expected to be handled using SQL queries. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/379/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
1096563265 | I_kwDOCGYnMM5BXDpB | 366 | Python library methods for calling ANALYZE | 9599 | closed | 0 | 7558727 | 10 | 2022-01-07T18:28:01Z | 2022-01-11T01:09:33Z | 2022-01-11T01:09:33Z | OWNER | > Relevant documentation: https://www.sqlite.org/lang_analyze.html _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/365#issuecomment-1007633376_ | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/366/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed |