github
id | node_id | number | title | user | state | locked | assignee | milestone | comments | created_at | updated_at | closed_at | author_association | pull_request | body | repo | type | active_lock_reason | performed_via_github_app | reactions | draft | state_reason |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
970320615 | MDU6SXNzdWU5NzAzMjA2MTU= | 316 | Fix visible backticks on reference page | 9599 | closed | 0 | 1 | 2021-08-13T11:37:46Z | 2021-08-14T05:12:23Z | 2021-08-14T05:10:48Z | OWNER | https://sqlite-utils.datasette.io/en/latest/reference.html Search for backtick to reveal various minor markup bugs. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/316/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
965102534 | MDU6SXNzdWU5NjUxMDI1MzQ= | 311 | Add reference documentation generated from docstrings | 9599 | closed | 0 | 4 | 2021-08-10T16:04:00Z | 2021-08-11T12:03:50Z | 2021-08-11T12:03:50Z | OWNER | Using https://www.sphinx-doc.org/en/master/usage/extensions/autodoc.html I'm not a big fan of this kind of documentation because it so often comes in place of narrative documentation - but the library has great narrative documentation now, so the reference documentation can link to it in places. This will also encourage me to add good docstrings everywhere, useful for IDEs and suchlike. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/311/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
965143346 | MDExOlB1bGxSZXF1ZXN0NzA3NDkwNzg5 | 312 | Add reference page to documentation using Sphinx autodoc | 9599 | closed | 0 | 10 | 2021-08-10T16:59:17Z | 2021-08-10T23:09:32Z | 2021-08-10T23:09:28Z | OWNER | simonw/sqlite-utils/pulls/312 | Refs #311. | 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/312/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
965440017 | MDU6SXNzdWU5NjU0NDAwMTc= | 315 | `.delete_where()` returns `[]` when it should return self | 9599 | closed | 0 | 1 | 2021-08-10T21:54:55Z | 2021-08-10T23:09:29Z | 2021-08-10T23:09:29Z | OWNER | If the table doesn't exist it should still return `self`, not `[]`: https://github.com/simonw/sqlite-utils/blob/ee469e3122d6f5973ec2584c1580d930daca2e7c/sqlite_utils/db.py#L1676-L1683 Spotted with `mypy` while working on #312. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/315/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
965166058 | MDU6SXNzdWU5NjUxNjYwNTg= | 313 | `.add_foreign_keys()` doesn't reject being called with a View | 9599 | closed | 0 | 0 | 2021-08-10T17:22:17Z | 2021-08-10T17:25:34Z | 2021-08-10T17:25:34Z | OWNER | Spotted this bug using `mypy` while working on #311 / #312! ``` % mypy sqlite_utils sqlite_utils/db.py:725: error: Item "View" of "Union[Table, View]" has no attribute "foreign_keys" Found 1 error in 1 file (checked 5 source files) ``` Refers to this code: https://github.com/simonw/sqlite-utils/blob/c11ff89894727270d4a9eb554d3a006f5b0d8d9d/sqlite_utils/db.py#L710-L720 It's a bug! We run some checks earlier but none of them ensure that it's a view: https://github.com/simonw/sqlite-utils/blob/c11ff89894727270d4a9eb554d3a006f5b0d8d9d/sqlite_utils/db.py#L697-L709 | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/313/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
959305209 | MDU6SXNzdWU5NTkzMDUyMDk= | 307 | codespell to spell check documentation | 9599 | closed | 0 | 0 | 2021-08-03T16:48:19Z | 2021-08-03T16:48:53Z | 2021-08-03T16:48:53Z | OWNER | As seen in https://github.com/simonw/datasette/issues/1417 and https://til.simonwillison.net/python/codespell | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/307/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
841377702 | MDU6SXNzdWU4NDEzNzc3MDI= | 251 | "sqlite-utils convert" command to replace the separate "sqlite-transform" tool | 9599 | closed | 0 | 15 | 2021-03-25T22:36:36Z | 2021-08-02T22:39:46Z | 2021-08-02T04:47:40Z | OWNER | See https://github.com/simonw/sqlite-transform/issues/11 - I built a separate `sqlite-transform` tool a while ago that uses the word "transform" to means something entirely different from `sqlite-utils transform` - I'd like to resolve this by merging the two tools. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/251/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
958516743 | MDU6SXNzdWU5NTg1MTY3NDM= | 306 | Configure sphinx.ext.extlinks for issues | 9599 | closed | 0 | 2 | 2021-08-02T21:19:19Z | 2021-08-02T21:39:34Z | 2021-08-02T21:29:22Z | OWNER | As seen in Datasette: https://github.com/simonw/datasette/issues/1227 | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/306/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
957383814 | MDU6SXNzdWU5NTczODM4MTQ= | 301 | insert-files should get a --silent option | 9599 | closed | 0 | 0 | 2021-08-01T04:11:03Z | 2021-08-02T19:12:21Z | 2021-08-02T19:12:21Z | OWNER | The new `sqlite-utils convert` command I'm adding in #251 will have a `--silent` option for turning off the progress bars. The only other command that has progress bars right now is `insert-files` so it should get this option too, for consistency. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/301/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
957731178 | MDU6SXNzdWU5NTc3MzExNzg= | 304 | `table.convert(..., where=)` and `sqlite-utils convert ... --where=` | 9599 | closed | 0 | 3 | 2021-08-02T04:27:23Z | 2021-08-02T19:00:00Z | 2021-08-02T18:58:10Z | OWNER | For applying the conversion to a subset of rows selected using the where clause. Should also take optional arguments, as seen in `db["dogs"].delete_where("age < ?", [3])`. Follows #302 and #251. This was originally https://github.com/simonw/sqlite-transform/issues/9 | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/304/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
957741820 | MDU6SXNzdWU5NTc3NDE4MjA= | 305 | Python: need a way to execute a count with an extra where clause | 9599 | closed | 0 | 1 | 2021-08-02T04:52:02Z | 2021-08-02T05:08:22Z | 2021-08-02T05:08:22Z | OWNER | I need this for #304. I'll probably add this to the `.execute_count()` method as `where=` and `where_args=`. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/305/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
957536983 | MDExOlB1bGxSZXF1ZXN0NzAwOTQ0NjQ0 | 303 | sqlite-utils convert command and db[table].convert(...) method | 9599 | closed | 0 | 1 | 2021-08-01T16:52:42Z | 2021-08-02T04:47:42Z | 2021-08-02T04:47:39Z | OWNER | simonw/sqlite-utils/pulls/303 | Refs #251, #302. - [x] Get recipes working - [x] Document recipes - [x] Implement `db[table].convert(...)` method - [x] Add tests for recipes that use the new Python method - [x] Implement `db[table].convert(..., multi=True)` mechanism - [x] Documentation for `db[table].convert(...)` - [x] Refactor `sqlite-utils convert` to use the new method | 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/303/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
957529248 | MDU6SXNzdWU5NTc1MjkyNDg= | 302 | Python library version of `sqlite-utils convert` | 9599 | closed | 0 | 9599 | 1 | 2021-08-01T16:11:02Z | 2021-08-02T04:47:40Z | 2021-08-02T04:47:40Z | OWNER | Spin off from #251. The ability to execute Python functions to convert and split columns should be part of the library too, not just the CLI. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/302/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
952154468 | MDU6SXNzdWU5NTIxNTQ0Njg= | 299 | Ability to see just specific table schemas with `sqlite-utils schema` | 9599 | closed | 0 | 1 | 2021-07-24T22:00:05Z | 2021-07-24T22:12:01Z | 2021-07-24T22:08:46Z | OWNER | It currently accepts no arguments. Allowing for optional arguments specifying tables would be useful: sqlite-utils schema fixtures.db facetable searchable | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/299/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
927789811 | MDU6SXNzdWU5Mjc3ODk4MTE= | 292 | Add contributing documentation | 9599 | closed | 0 | 0 | 2021-06-23T02:13:05Z | 2021-06-25T17:53:51Z | 2021-06-25T17:53:51Z | OWNER | Like https://docs.datasette.io/en/latest/contributing.html (but simpler) - should cover how to run `black` and `flake8` and `mypy` and how to run the tests. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/292/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
926777310 | MDU6SXNzdWU5MjY3NzczMTA= | 290 | `db.query()` method (renamed `db.execute_returning_dicts()`) | 9599 | closed | 0 | 6 | 2021-06-22T03:03:54Z | 2021-06-24T23:17:38Z | 2021-06-24T22:54:43Z | OWNER | Most of this library deals with lists of Python dictionaries - `.insert_all()`, `.rows`, `.rows_where()`, `.search()`. The `db.execute()` method is the only thing that returns a `sqlite3` cursor. There is a clumsily named `db.execute_returning_dicts(sql)` method but it's not currently mentioned in the documentation. It needs a better name, and needs to be properly documented. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/290/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
927766296 | MDU6SXNzdWU5Mjc3NjYyOTY= | 291 | Adopt flake8 | 9599 | closed | 0 | 2 | 2021-06-23T01:19:37Z | 2021-06-24T17:50:27Z | 2021-06-24T17:50:27Z | OWNER | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/291/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||||
925487946 | MDU6SXNzdWU5MjU0ODc5NDY= | 286 | Add installation instructions | 9599 | closed | 0 | 1 | 2021-06-19T23:55:36Z | 2021-06-20T18:47:13Z | 2021-06-20T18:47:13Z | OWNER | `pip install sqlite-utils`, `pipx install sqlite-utils` and `brew install sqlite-utils` | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/286/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
925544070 | MDU6SXNzdWU5MjU1NDQwNzA= | 287 | Update rowid examples in the docs | 9599 | closed | 0 | 0 | 2021-06-20T08:03:00Z | 2021-06-20T18:26:21Z | 2021-06-20T18:26:21Z | OWNER | Changed in #284 - a couple of examples need updating on https://github.com/simonw/sqlite-utils/blob/3.10/docs/cli.rst. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/287/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
925545468 | MDU6SXNzdWU5MjU1NDU0Njg= | 288 | sqlite-utils memory blah.json --schema | 9599 | closed | 0 | 0 | 2021-06-20T08:10:40Z | 2021-06-20T18:26:21Z | 2021-06-20T18:26:21Z | OWNER | Like `--dump` but only outputs the schema - useful for understanding what you are about to run queries against. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/288/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
921878733 | MDU6SXNzdWU5MjE4Nzg3MzM= | 272 | Idea: import CSV to memory, run SQL, export in a single command | 9599 | closed | 0 | 22 | 2021-06-15T23:02:48Z | 2021-06-19T23:36:48Z | 2021-06-18T15:05:03Z | OWNER | I quite often load a CSV file into a SQLite DB, then do stuff with it (like export results back out again as a new CSV) without any intention of keeping the CSV file around afterwards. What if `sqlite-utils` could do this for me? Something like this: sqlite-utils --csv blah.csv --csv baz.csv "select * from blah join baz ..." | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/272/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
925320167 | MDU6SXNzdWU5MjUzMjAxNjc= | 284 | .transform(types=) turns rowid into a concrete column | 9599 | closed | 0 | 5 | 2021-06-19T05:25:27Z | 2021-06-19T15:28:30Z | 2021-06-19T15:28:30Z | OWNER | Noticed this in the tests for `sqlite-utils memory` in #282 - is it possible to fix this? https://github.com/simonw/sqlite-utils/commit/ec5174ed40fa283cb06f25ee0c0136297ec313ae | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/284/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
925410305 | MDU6SXNzdWU5MjU0MTAzMDU= | 285 | Introspection property for telling if a table is a rowid table | 9599 | closed | 0 | 7 | 2021-06-19T14:56:16Z | 2021-06-19T15:12:33Z | 2021-06-19T15:12:33Z | OWNER | _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/284#issuecomment-864416785_ | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/285/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
925319214 | MDU6SXNzdWU5MjUzMTkyMTQ= | 283 | memory: Shouldn't detect types for JSON | 9599 | closed | 0 | 1 | 2021-06-19T05:17:35Z | 2021-06-19T14:52:48Z | 2021-06-19T14:52:48Z | OWNER | https://github.com/simonw/sqlite-utils/blob/ec5174ed40fa283cb06f25ee0c0136297ec313ae/sqlite_utils/cli.py#L1244-L1251 This runs against JSON as well as CSV/TSV - which isn't necessary and In fact throws errors if there is any nested data. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/283/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
925305186 | MDU6SXNzdWU5MjUzMDUxODY= | 282 | Automatic type detection for CSV data | 9599 | closed | 0 | 4 | 2021-06-19T03:33:21Z | 2021-06-19T04:42:03Z | 2021-06-19T04:38:00Z | OWNER | I've touched on this before in #179 - but now that I've added `sqlite-utils memory` this is much more important - because unlike with `sqlite-utils insert` the in-memory command doesn't give you the opportunity to fix any types you imported from CSV, so queries like `select * from stdin where age > 3` are never going to work correctly against these temporary in-memory tables. Teaching `sqlite-utils insert` to detect types for columns in a CSV file would be a backwards-compatibility breaking change. Teaching `sqlite-utils memory` that trick would not be, since it hasn't been included in a release yet. It's a little inconsistent, but I'm going to have `sqlite-utils memory` default to detecting types while `sqlite-utils insert` does not. In each case this can be controlled by a new command-line option: cat file.csv | sqlite-utils memory - --no-detect-types To opt-in for `sqlite-utils insert`: cat file.csv | sqlite-utils insert blah.db blah - --detect-types I'll have short options for these too: `-n` for `--no-detect-types` and `-d` for `--detect-types`. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/282/reactions", "total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 1, "rocket": 0, "eyes": 0 } |
completed | ||||||
709577625 | MDU6SXNzdWU3MDk1Nzc2MjU= | 179 | sqlite-utils transform/insert --detect-types | 9599 | closed | 0 | 4 | 2020-09-26T17:28:55Z | 2021-06-19T03:36:16Z | 2021-06-19T03:36:05Z | OWNER | Idea from https://github.com/simonw/datasette-edit-tables/issues/13 - provide Python utility methods and accompanying CLI options for detecting the likely types of TEXT columns. So if you have a text column that actually contained exclusively integer string values, it can let you know and let you run transform against it. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/179/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
924990677 | MDU6SXNzdWU5MjQ5OTA2Nzc= | 279 | sqlite-utils memory should handle TSV and JSON in addition to CSV | 9599 | closed | 0 | 7 | 2021-06-18T15:02:54Z | 2021-06-19T03:11:59Z | 2021-06-19T03:11:59Z | OWNER | - Use sniff to detect CSV or TSV (if `:tsv` or `:csv` was not specified) and delimiters Follow-on from #272 | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/279/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
924992318 | MDU6SXNzdWU5MjQ5OTIzMTg= | 281 | Mechanism for explicitly stating CSV or JSON or TSV for sqlite-utils memory | 9599 | closed | 0 | 1 | 2021-06-18T15:04:53Z | 2021-06-19T03:11:59Z | 2021-06-19T03:11:59Z | OWNER | - Implement `filename.json:json` and `-:nl` and suchlike options for specifying the format rather than guessing it - see https://github.com/simonw/sqlite-utils/issues/272#issuecomment-861985944 Follows #272 | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/281/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
924991194 | MDU6SXNzdWU5MjQ5OTExOTQ= | 280 | Add --encoding option to sqlite-utils memory | 9599 | closed | 0 | 0 | 2021-06-18T15:03:32Z | 2021-06-18T15:29:46Z | 2021-06-18T15:29:46Z | OWNER | Follow-on from #272 - this will work like `--encoding` on `sqlite-utils insert` and will affect all CSV files processed by `sqlite-utils memory`. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/280/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
922099793 | MDExOlB1bGxSZXF1ZXN0NjcxMDE0NzUx | 273 | sqlite-utils memory command for directly querying CSV/JSON data | 9599 | closed | 0 | 8 | 2021-06-16T05:04:58Z | 2021-06-18T15:01:17Z | 2021-06-18T15:00:52Z | OWNER | simonw/sqlite-utils/pulls/273 | Refs #272. Initial implementation only does CSV data, still needs: - [x] Implement `--save` - [x] Add `--dump` to the documentation - [x] Add `--attach` example to the documentation - [x] Replace `:memory:` in documentation | 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/273/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
922955697 | MDU6SXNzdWU5MjI5NTU2OTc= | 275 | Enable code coverage | 9599 | closed | 0 | 1 | 2021-06-16T18:33:49Z | 2021-06-17T00:12:12Z | 2021-06-17T00:12:12Z | OWNER | https://app.codecov.io/gh/simonw/sqlite-utils Same mechanism as Datasette. Need to copy across the token from that page and add an equivalent of this workflow: https://github.com/simonw/datasette/blob/main/.github/workflows/test-coverage.yml | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/275/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
922832113 | MDU6SXNzdWU5MjI4MzIxMTM= | 274 | sqlite-utils dump my.db command | 9599 | closed | 0 | 0 | 2021-06-16T16:30:14Z | 2021-06-16T23:51:54Z | 2021-06-16T23:51:54Z | OWNER | Inspired by the `--dump` mechanism I added to `sqlite-utils memory` here: https://github.com/simonw/sqlite-utils/issues/272#issuecomment-862018937 > Can use `.iterdump()` to implement this: https://docs.python.org/3/library/sqlite3.html#sqlite3.Connection.iterdump > > Maybe instead (or as-well-as) offer `--dump` which dumps out the SQL from that. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/274/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
919702451 | MDU6SXNzdWU5MTk3MDI0NTE= | 271 | table.upsert_all() fails if input has a single column that should be a primary key | 9599 | closed | 0 | 1 | 2021-06-13T02:50:27Z | 2021-06-13T02:57:29Z | 2021-06-13T02:57:29Z | OWNER | This works: ```pycon >>> db['foo'].insert_all([{"name": "hello"}], pk="name") <Table foo (name)> ``` But this fails: ``` >>> db['foo3'].upsert_all([{"name": "hello"}], pk="name") Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/Users/simon/.local/share/virtualenvs/datasette.io-TK86ygSO/lib/python3.9/site-packages/sqlite_utils/db.py", line 1837, in upsert_all return self.insert_all( File "/Users/simon/.local/share/virtualenvs/datasette.io-TK86ygSO/lib/python3.9/site-packages/sqlite_utils/db.py", line 1778, in insert_all self.insert_chunk( File "/Users/simon/.local/share/virtualenvs/datasette.io-TK86ygSO/lib/python3.9/site-packages/sqlite_utils/db.py", line 1588, in insert_chunk result = self.db.execute(query, params) File "/Users/simon/.local/share/virtualenvs/datasette.io-TK86ygSO/lib/python3.9/site-packages/sqlite_utils/db.py", line 213, in execute return self.conn.execute(sql, parameters) sqlite3.OperationalError: near "WHERE": syntax error ``` With the debugger: ``` >>> import pdb; pdb.pm() > /Users/simon/.local/share/virtualenvs/datasette.io-TK86ygSO/lib/python3.9/site-packages/sqlite_utils/db.py(213)execute() -> return self.conn.execute(sql, parameters) (Pdb) print(sql, parameters) UPDATE [foo3] SET WHERE [name] = ? ['hello'] ``` | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/271/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
919181559 | MDU6SXNzdWU5MTkxODE1NTk= | 268 | db.schema property and sqlite-utils schema command | 9599 | closed | 0 | 4 | 2021-06-11T20:25:47Z | 2021-06-11T20:51:56Z | 2021-06-11T20:51:56Z | OWNER | `table.schema` returns the schema for a table. `db.schema` should return the schema for the whole databes. Can do this using `select sql from sqlite_master where sql is not null`: https://latest.datasette.io/fixtures?sql=select+sql+from+sqlite_master+where+sql+is+not+null | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/268/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
906356331 | MDU6SXNzdWU5MDYzNTYzMzE= | 263 | `sqlite-utils indexes` command | 9599 | closed | 0 | 6 | 2021-05-29T04:52:34Z | 2021-06-03T04:34:38Z | 2021-06-03T04:34:38Z | OWNER | While working on #260 I realized there's no command to show indexes in a database, even though there is one for showing tables and one for triggers. I should implement #261 first. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/263/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
906345899 | MDU6SXNzdWU5MDYzNDU4OTk= | 261 | `table.xindexes` using `PRAGMA index_xinfo(table)` | 9599 | closed | 0 | 5 | 2021-05-29T04:23:48Z | 2021-06-03T03:54:14Z | 2021-06-03T03:51:32Z | OWNER | > `PRAGMA index_xinfo(table)` DOES return that data: > ``` > (Pdb) [c[0] for c in fresh_db.execute("PRAGMA > index_xinfo('idx_dogs_age_name')").description] > ['seqno', 'cid', 'name', 'desc', 'coll', 'key'] > (Pdb) fresh_db.execute("PRAGMA index_xinfo('idx_dogs_age_name')").fetchall() > [(0, 2, 'age', 1, 'BINARY', 1), (1, 0, 'name', 0, 'BINARY', 1), (2, -1, None, 0, 'BINARY', 0)] > ``` > See https://sqlite.org/pragma.html#pragma_index_xinfo > > Example output: https://covid-19.datasettes.com/covid?sql=select+*+from+pragma_index_xinfo%28%27idx_ny_times_us_counties_date%27%29 _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/260#issuecomment-850766552_ | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/261/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
838148087 | MDU6SXNzdWU4MzgxNDgwODc= | 250 | Handle byte order marks (BOMs) in CSV files | 9599 | closed | 0 | 3 | 2021-03-22T22:13:18Z | 2021-05-29T05:34:21Z | 2021-05-29T05:34:21Z | OWNER | I often find `sqlite-utils insert ... --csv` creates a first column with a weird character at the start of it - which it turns out is the UTF-8 BOM. Fix that. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/250/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
906355849 | MDExOlB1bGxSZXF1ZXN0NjU3MzczNzI2 | 262 | Ability to add descending order indexes | 9599 | closed | 0 | 0 | 2021-05-29T04:51:04Z | 2021-05-29T05:01:42Z | 2021-05-29T05:01:39Z | OWNER | simonw/sqlite-utils/pulls/262 | Refs #260 | 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/262/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
906330187 | MDU6SXNzdWU5MDYzMzAxODc= | 260 | Support creating descending order indexes | 9599 | closed | 0 | 12 | 2021-05-29T03:42:59Z | 2021-05-29T05:01:39Z | 2021-05-29T05:01:39Z | OWNER | SQLite lets you create indexes in reverse order, which can have a surprisingly big impact on performance, see https://github.com/simonw/covid-19-datasette/issues/27 I tried doing this using `sqlite-utils` like so, but it's didn't work: ```python db["ny_times_us_counties"].create_index(["date desc"]) ``` | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/260/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
858501079 | MDU6SXNzdWU4NTg1MDEwNzk= | 255 | transform --help should tell you the available types | 9599 | closed | 0 | 0 | 2021-04-15T05:24:48Z | 2021-05-29T03:55:52Z | 2021-05-29T03:55:52Z | OWNER | ``` Usage: sqlite-utils transform [OPTIONS] PATH TABLE Transform a table beyond the capabilities of ALTER TABLE Options: --type <TEXT TEXT>... Change column type to X ``` This should specify that the possible types are 'INTEGER', 'TEXT', 'FLOAT', 'BLOB'. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/255/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
894948100 | MDU6SXNzdWU4OTQ5NDgxMDA= | 259 | Suggest the --alter option if a new column cannot be added | 9599 | closed | 0 | 1 | 2021-05-19T03:17:38Z | 2021-05-19T03:27:33Z | 2021-05-19T03:26:26Z | OWNER | Refs #256. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/259/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
816523763 | MDU6SXNzdWU4MTY1MjM3NjM= | 238 | .add_foreign_key() corrupts database if column contains a space | 9599 | closed | 0 | 1 | 2021-02-25T15:07:20Z | 2021-02-25T16:54:02Z | 2021-02-25T16:54:02Z | OWNER | I ran this: db["Reports"].add_foreign_key("Reported by ID", "Reporters", "id") And got this: ``` ~/jupyter-venv/lib/python3.9/site-packages/sqlite_utils/db.py in add_foreign_keys(self, foreign_keys) 616 # Have to VACUUM outside the transaction to ensure .foreign_keys property 617 # can see the newly created foreign key. --> 618 self.vacuum() 619 620 def index_foreign_keys(self): ~/jupyter-venv/lib/python3.9/site-packages/sqlite_utils/db.py in vacuum(self) 629 630 def vacuum(self): --> 631 self.execute("VACUUM;") 632 633 ~/jupyter-venv/lib/python3.9/site-packages/sqlite_utils/db.py in execute(self, sql, parameters) 234 return self.conn.execute(sql, parameters) 235 else: --> 236 return self.conn.execute(sql) 237 238 def executescript(self, sql): DatabaseError: database disk image is malformed ``` | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/238/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
816560819 | MDU6SXNzdWU4MTY1NjA4MTk= | 240 | table.pks_and_rows_where() method returning primary keys along with the rows | 9599 | closed | 0 | 7 | 2021-02-25T15:49:28Z | 2021-02-25T16:39:23Z | 2021-02-25T16:28:23Z | OWNER | *Original title: Easier way to update a row returned from .rows* Here's a surprisingly hard problem I ran into while trying to implement #239 - given a row returned by `db[table].rows` how can you update that row? The problem is that the `db[table].update(...)` method requires a primary key. But if you have a row from the `db[table].rows` iterator it might not even contain the primary key - provided the table is a `rowid` table. Instead, currently, you need to introspect the table and, if `rowid` is a primary key, explicitly include that in the `select=` argument to `table.rows_where(...)` - otherwise it will not be returned. A utility mechanism to make this easier would be very welcome. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/240/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
811680502 | MDU6SXNzdWU4MTE2ODA1MDI= | 236 | --attach command line option for attaching extra databases | 9599 | closed | 0 | 1 | 2021-02-19T04:38:30Z | 2021-02-19T05:10:41Z | 2021-02-19T05:08:43Z | OWNER | This will enable cross-database joins, as seen in https://github.com/simonw/datasette/issues/283 Also refs #113 | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/236/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
621286870 | MDU6SXNzdWU2MjEyODY4NzA= | 113 | Syntactic sugar for ATTACH DATABASE | 9599 | closed | 0 | 2 | 2020-05-19T21:10:00Z | 2021-02-19T05:09:12Z | 2021-02-19T04:56:36Z | OWNER | https://www.sqlite.org/lang_attach.html Maybe something like this: ```python db.attach("other_db", "other_db.db") ``` | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/113/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
808008305 | MDU6SXNzdWU4MDgwMDgzMDU= | 230 | --sniff option for sniffing delimiters | 9599 | closed | 0 | 8 | 2021-02-14T17:43:54Z | 2021-02-14T21:15:33Z | 2021-02-14T19:24:32Z | OWNER | > I just spotted that `csv.Sniffer` in the Python standard library has a `.has_header(sample)` method which detects if the first row appears to be a header or not, which is interesting. https://docs.python.org/3/library/csv.html#csv.Sniffer _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/228#issuecomment-778812050_ | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/230/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
808046597 | MDU6SXNzdWU4MDgwNDY1OTc= | 234 | .insert_all() fails if subsequent chunks contain additional columns | 9599 | closed | 0 | 1 | 2021-02-14T21:01:51Z | 2021-02-14T21:03:40Z | 2021-02-14T21:03:40Z | OWNER | Reported by @nieuwenhoven in #225 along with a proposed fix. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/234/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
808036774 | MDU6SXNzdWU4MDgwMzY3NzQ= | 232 | Run tests against Windows in GitHub Actions | 9599 | closed | 0 | 0 | 2021-02-14T20:09:45Z | 2021-02-14T20:39:55Z | 2021-02-14T20:39:55Z | OWNER | > I'm going to try and get the test suite to run in Windows on GitHub Actions. _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/225#issuecomment-778834504_ | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/232/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
808037010 | MDExOlB1bGxSZXF1ZXN0NTczMTQ3MTY4 | 233 | Run tests against Ubuntu, macOS and Windows | 9599 | closed | 0 | 0 | 2021-02-14T20:11:02Z | 2021-02-14T20:39:54Z | 2021-02-14T20:39:54Z | OWNER | simonw/sqlite-utils/pulls/233 | Refs #232 | 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/233/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
808028757 | MDU6SXNzdWU4MDgwMjg3NTc= | 231 | limit=X, offset=Y parameters for more Python methods | 9599 | closed | 0 | 2 | 2021-02-14T19:31:23Z | 2021-02-14T20:03:08Z | 2021-02-14T20:03:08Z | OWNER | > I'm going to add a `offset=` parameter to support this case. Thanks for the suggestion! _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/224#issuecomment-778828495_ | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/231/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
802583450 | MDU6SXNzdWU4MDI1ODM0NTA= | 226 | 3.4 release is broken - includes a rogue line | 9599 | closed | 0 | 0 | 2021-02-06T02:08:01Z | 2021-02-06T02:10:26Z | 2021-02-06T02:10:26Z | OWNER | I started seeing weird errors, caused by this line: https://github.com/simonw/sqlite-utils/blob/f8010ca78fed8c5fca6cde19658ec09fdd468420/sqlite_utils/cli.py#L1-L3 That was added by accident in 1b666f9315d4ea6bb332b2e75e48480c26100199 I'm surprised the tests didn't catch this! | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/226/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
788527932 | MDU6SXNzdWU3ODg1Mjc5MzI= | 223 | --delimiter option for CSV import | 9599 | closed | 0 | 2 | 2021-01-18T20:25:03Z | 2021-02-06T01:39:47Z | 2021-02-06T01:34:54Z | OWNER | https://bruxellesdata.opendatasoft.com/explore/dataset/dog-toilets/export/?location=12,50.85802,4.38054 says: > CSV uses semicolon (;) as a separator. Would be useful to be able to do this: sqlite-utils insert places.db places places.csv --delimiter ';' `--delimiter` could imply `--csv` | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/223/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
787900412 | MDU6SXNzdWU3ODc5MDA0MTI= | 222 | .m2m() should accept alter=True parameter | 9599 | closed | 0 | 0 | 2021-01-18T04:15:43Z | 2021-01-18T04:26:10Z | 2021-01-18T04:26:10Z | OWNER | Needed by https://github.com/dogsheep/swarm-to-sqlite/issues/11 | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/222/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
783910901 | MDU6SXNzdWU3ODM5MTA5MDE= | 221 | .add_missing_columns() does not take case insensitivity into account | 9599 | closed | 0 | 0 | 2021-01-12T05:01:00Z | 2021-01-12T23:17:33Z | 2021-01-12T23:17:33Z | OWNER | SQLite columns are case insensitive - but the `.add_missing_columns()` method doesn't know that. This means that it can crash if it identifies a column that is a case-insensitive duplicate of an existing column. https://github.com/simonw/sqlite-utils/blob/4cc82fd0bccc9d2eeb3510beb4e691d7da099f84/sqlite_utils/db.py#L1974-L1980 | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/221/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
777707544 | MDU6SXNzdWU3Nzc3MDc1NDQ= | 219 | reset_counts() method and command | 9599 | closed | 0 | 4 | 2021-01-03T20:08:28Z | 2021-01-03T20:59:37Z | 2021-01-03T20:59:37Z | OWNER | > Thought: maybe there should be a `.reset_counts()` method too, for if the table gets out of date with the triggers. > > One way that could happen is if a table is dropped and recreated - the counts in the `_counts` table would likely no longer match the number of rows in that table. _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/215#issuecomment-753545757_ | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/219/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
777535402 | MDU6SXNzdWU3Nzc1MzU0MDI= | 215 | Use _counts to speed up counts | 9599 | closed | 0 | 9 | 2021-01-02T22:30:17Z | 2021-01-03T20:19:40Z | 2021-01-03T20:19:40Z | OWNER | Utility mechanism for taking advantage of the new `_counts` table from #212 would be nice. These can trigger automatically if the `_counts` table exists, but since `sqlite-utils` needs to work against any existing database there should be a way of opting out of this optimization. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/215/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
738514367 | MDU6SXNzdWU3Mzg1MTQzNjc= | 202 | sqlite-utils insert -f colname - for configuring full-text search | 9599 | closed | 0 | 2 | 2020-11-08T17:30:09Z | 2021-01-03T05:00:36Z | 2021-01-03T05:00:27Z | OWNER | A mechanism for specifying columns that should be configured for full-text search as part of the initial data import: sqlite-utils insert mydb.db articles articles.csv --csv -f title -f body | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/202/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
777543336 | MDU6SXNzdWU3Nzc1NDMzMzY= | 217 | Rename .escape() to .quote() | 9599 | closed | 0 | 2 | 2021-01-02T23:40:52Z | 2021-01-03T04:27:38Z | 2021-01-03T04:15:23Z | OWNER | `.quote()` is a better name because it reflects that the method adds quotes around the value. This method has never been documented so I'm going to rename it without a major version bump. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/217/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
777540352 | MDU6SXNzdWU3Nzc1NDAzNTI= | 216 | database.triggers_dict introspection property | 9599 | closed | 0 | 1 | 2021-01-02T23:13:00Z | 2021-01-03T04:27:14Z | 2021-01-03T04:25:36Z | OWNER | Following #211 | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/216/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
777530107 | MDU6SXNzdWU3Nzc1MzAxMDc= | 214 | sqlite-utils enable-counts command | 9599 | closed | 0 | 0 | 2021-01-02T21:45:48Z | 2021-01-03T04:26:44Z | 2021-01-03T04:26:44Z | OWNER | The CLI version of #212 and #213. # Enable counts for all tables: sqlite-utils enable-counts data.db # Enable counts for specific tables: sqlite-utils enable-counts data.db table1 table2 | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/214/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
777560474 | MDU6SXNzdWU3Nzc1NjA0NzQ= | 218 | "sqlite-utils triggers" command | 9599 | closed | 0 | 1 | 2021-01-03T02:34:50Z | 2021-01-03T03:49:51Z | 2021-01-03T03:03:35Z | OWNER | A command to list the triggers in the database. sqlite-utils triggers my.db Can optionally take one or more tables: sqlite-utils triggers my.db table1 table2 | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/218/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
777529979 | MDU6SXNzdWU3Nzc1Mjk5Nzk= | 213 | db.enable_counts() method | 9599 | closed | 0 | 2 | 2021-01-02T21:44:55Z | 2021-01-02T22:04:02Z | 2021-01-02T22:04:02Z | OWNER | Following #212 it would be useful if there was a utility method for enabling counts for ALL tables in a database: ```python db.enable_counts() ``` Open question: should this setup triggers for virtual tables such as FTS tables? Could that break things? | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/213/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
777392020 | MDU6SXNzdWU3NzczOTIwMjA= | 212 | Mechanism for maintaining cache of table counts using triggers | 9599 | closed | 0 | 1 | 2021-01-02T02:58:53Z | 2021-01-02T21:40:27Z | 2021-01-02T21:40:27Z | OWNER | Counting all of the rows in a large table is expensive - this is one of the main causes of performance problems in Datasette when running against large databases. Carefully constructed SQL triggers could be used to maintain accurate cached counts for a table, by incrementing and decrementing a counter every time a row is inserted or deleted. `sqlite-utils` already has a mechanism for creating triggers for FTS - the `table.enable_fts(..., create_triggers=True)` method. How about a similar mechanism for setting up triggers to maintain a count of table rows? | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/212/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
777386465 | MDU6SXNzdWU3NzczODY0NjU= | 211 | table.triggers_dict introspection property | 9599 | closed | 0 | 0 | 2021-01-02T02:04:00Z | 2021-01-02T02:10:10Z | 2021-01-02T02:10:10Z | OWNER | `table.triggers` currently returns a list of `Trigger` values. A `table.triggers_dict` property could behave like `columns_dict`, returning a dictionary mapping trigger names to their SQL definitions for that table. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/211/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
763283616 | MDU6SXNzdWU3NjMyODM2MTY= | 207 | sqlite-utils analyze-tables command | 9599 | closed | 0 | 4 | 2020-12-12T04:33:12Z | 2020-12-13T07:25:23Z | 2020-12-13T07:20:13Z | OWNER | A command which analyzes a table (potentially taking quite a while if the table is large) and outputs information for each column - things like: - How many unique values does this column have? - How many null rows? - How many blank rows? (defined as empty string) - What are the 10 most common values? - What are the 10 least common values? The command can output this information to the terminal, but it should also provide an option for writing the information to a database table so it can be explored later. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/207/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
763320133 | MDExOlB1bGxSZXF1ZXN0NTM3NzkxNjc1 | 208 | sqlite-utils analyze-tables command and table.analyze_column() method | 9599 | closed | 0 | 6 | 2020-12-12T05:27:49Z | 2020-12-13T07:20:16Z | 2020-12-13T07:20:12Z | OWNER | simonw/sqlite-utils/pulls/208 | Refs #207 - [x] Improve design of CLI output - [x] Truncate long values in least/most common - [x] Add a `-c` column selection option - [x] Tests - [x] Documentation | 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/208/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
706167456 | MDU6SXNzdWU3MDYxNjc0NTY= | 168 | Automate (as much as possible) updates published to Homebrew | 9599 | closed | 0 | 2 | 2020-09-22T08:08:37Z | 2020-11-09T07:43:30Z | 2020-11-09T07:43:30Z | OWNER | I'd like to get new `sqlite-utils` (and Datasette) releases submitted to Homebrew as painlessly as possible. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/168/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
735650864 | MDU6SXNzdWU3MzU2NTA4NjQ= | 194 | 3.0 release with some minor breaking changes | 9599 | closed | 0 | 6079500 | 3 | 2020-11-03T21:36:31Z | 2020-11-08T17:19:35Z | 2020-11-08T17:19:34Z | OWNER | While working on search (#192) I've spotted a few small changes I would like to make that would break backwards compatibility in minor ways, hence requiring a 3.x release. `db[table].search()` - I would like this to default to sorting by rank Also I'd like to free up the `-c` and `-f` options for other purposes from the standard output formats here: https://github.com/simonw/sqlite-utils/blob/43eae8b193d362f2b292df73e087ed6f10838144/sqlite_utils/cli.py#L48-L58 I'd like `-f` to be used to indicate a full-text search column during an insert and `-c` to indicate a column (so you can specify which columns you want to output). | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/194/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
737153927 | MDU6SXNzdWU3MzcxNTM5Mjc= | 197 | Rethink how table.search() method works | 9599 | closed | 0 | 6079500 | 5 | 2020-11-05T18:04:34Z | 2020-11-08T17:07:37Z | 2020-11-08T17:07:37Z | OWNER | I need to improve this method to help build `sqlite-utils search` in #192 (PR is #195). The challenge is deciding how it should handle sorting by relevance - especially since that is easy in FTS5 but not at all easy in FTS4. > Latest test failure: > ``` > 114 -> assert [("racoons are biting trash pandas", "USA", "bar")] == table.search( > 115 "bite", order="rowid" > 116 ) > 117 > 118 > 119 def test_optimize_fts(fresh_db): > (Pdb) table.search("bite") > [(2, 'racoons are biting trash pandas', 'USA', 'bar', -9.641434262948206e-07)] > ``` > The problem here is that the `table.search()` method now behaves differently for FTS4 v.s. FTS5 tables. > > With FTS4 you get back just the table columns. > > With FTS5 you also get back the `rowid` as the first column and the `rank` score as the last column. > > This is weird. It also makes me question whether having `.search()` return a list of tuples is the right API design. _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/pull/195#issuecomment-722542895_ | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/197/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
735532751 | MDU6SXNzdWU3MzU1MzI3NTE= | 192 | sqlite-utils search command | 9599 | closed | 0 | 6079500 | 9 | 2020-11-03T18:07:59Z | 2020-11-08T17:07:01Z | 2020-11-08T17:07:01Z | OWNER | A command that knows how to run a search against a FTS enabled table and return results ranked by relevance. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/192/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
738128913 | MDU6SXNzdWU3MzgxMjg5MTM= | 201 | .search(columns=) and sqlite-utils search -c ... bug | 9599 | closed | 0 | 6079500 | 1 | 2020-11-07T01:27:26Z | 2020-11-08T16:54:15Z | 2020-11-08T16:54:15Z | OWNER | Both `table.search(columns=)` and the `sqlite-utils search -c` option do not work as expected - they always return both the `rowid` and the `rank` columns even if those have not been requested. This should be fixed before the 3.0 non-alpha release. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/201/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
738115165 | MDU6SXNzdWU3MzgxMTUxNjU= | 200 | sqlite-utils rows -c option | 9599 | closed | 0 | 6079500 | 1 | 2020-11-07T00:22:12Z | 2020-11-07T00:28:48Z | 2020-11-07T00:28:47Z | OWNER | To let you specify the exact columns you want. Based on the `-c` option to `sqlite-utils search` in #192. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/200/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
735648209 | MDU6SXNzdWU3MzU2NDgyMDk= | 193 | --tsv output format option | 9599 | closed | 0 | 6079500 | 0 | 2020-11-03T21:31:18Z | 2020-11-07T00:09:52Z | 2020-11-07T00:09:52Z | OWNER | We already support `--csv` for output, and the `insert` command accepts `--tsv`. The output format options should accept `--tsv` too. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/193/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
735663855 | MDExOlB1bGxSZXF1ZXN0NTE1MDE0ODgz | 195 | table.search() improvements plus sqlite-utils search command | 9599 | closed | 0 | 3 | 2020-11-03T22:02:08Z | 2020-11-06T18:30:49Z | 2020-11-06T18:30:42Z | OWNER | simonw/sqlite-utils/pulls/195 | Refs #192. Still needs tests. | 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/195/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
737476423 | MDU6SXNzdWU3Mzc0NzY0MjM= | 198 | Support order by relevance against FTS4 | 9599 | closed | 0 | 6 | 2020-11-06T05:36:31Z | 2020-11-06T18:30:44Z | 2020-11-06T18:30:44Z | OWNER | For #192 and #197 I've decided I want to be able to order by relevance in FTS4 as well as FTS5. This means I need to port over my work on bm25() from https://github.com/simonw/sqlite-fts4 (since I don't want to add a full dependency). | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/198/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
737855731 | MDU6SXNzdWU3Mzc4NTU3MzE= | 199 | @db.register_function(..., replace=False) to avoid double-registering custom functions | 9599 | closed | 0 | 1 | 2020-11-06T15:39:21Z | 2020-11-06T18:30:44Z | 2020-11-06T18:30:44Z | OWNER | I'd like a mechanism to optionally avoid registering a custom function if it has already been registered. SQLite doesn't seem to offer a way to introspect registered custom functions so I'll need to track what has already been registered in `sqlite-utils` instead. > Should I register the custom `rank_bm25` SQLite function for every connection, or should I register it against the connection just the first time the user attempts an FTS4 search? I think I'd rather register it only if it is needed. _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/198#issuecomment-723145383_ | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/199/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
736520310 | MDU6SXNzdWU3MzY1MjAzMTA= | 196 | Introspect if table is FTS4 or FTS5 | 9599 | closed | 0 | 19 | 2020-11-05T00:45:50Z | 2020-11-05T03:54:07Z | 2020-11-05T03:54:07Z | OWNER | > I want `.search()` to work against both FTS5 and FTS4 tables - but sort by rank should only work for FTS5. > > This means I need to be able to introspect and tell if a table is FTS4 or FTS5. _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/192#issuecomment-722054264_ | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/196/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
731740458 | MDU6SXNzdWU3MzE3NDA0NTg= | 191 | Idea: @db.register_function(deterministic=True) | 9599 | closed | 0 | 2 | 2020-10-28T19:45:18Z | 2020-10-28T21:31:06Z | 2020-10-28T21:31:06Z | OWNER | Python 3.8 added a `deterministic` parameter to `db.create_function()`: https://docs.python.org/3/library/sqlite3.html#sqlite3.Connection.create_function `sqlite-utils` could expose this in the decorator, only actually applying it if the Python version supports it (using feature detection) - since nothing will break if it's not applied. https://sqlite-utils.readthedocs.io/en/stable/python-api.html#registering-custom-sql-functions ```python db = Database(memory=True) @db.register_function(deterministic=True) def reverse_string(s): return "".join(reversed(list(s))) ``` | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/191/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
730693696 | MDExOlB1bGxSZXF1ZXN0NTEwOTU2MTM0 | 190 | Progress bar for sqlite-utils insert command | 9599 | closed | 0 | 0 | 2020-10-27T18:08:53Z | 2020-10-27T18:16:03Z | 2020-10-27T18:16:03Z | OWNER | simonw/sqlite-utils/pulls/190 | Refs #173 | 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/190/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
709861194 | MDU6SXNzdWU3MDk4NjExOTQ= | 180 | Try running some tests using Hypothesis | 9599 | closed | 0 | 1 | 2020-09-28T01:11:30Z | 2020-10-19T04:51:55Z | 2020-10-19T04:51:55Z | OWNER | Inspired by this Twitter conversation: https://twitter.com/simonw/status/1310386009465479168 | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/180/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
683830416 | MDU6SXNzdWU2ODM4MzA0MTY= | 137 | --load-extension for other sqlite-utils commands | 9599 | closed | 0 | 1 | 2020-08-21T21:12:56Z | 2020-10-16T19:14:32Z | 2020-10-16T19:14:32Z | OWNER | e.g. for this: ``` calands-datasette % sqlite-utils tables calands.db --counts [{"table": "spatial_ref_sys", "count": 4924}, {"table": "spatialite_history", "count": 14}, {"table": "sqlite_sequence", "count": 1}, {"table": "geometry_columns", "count": 2}, {"table": "spatial_ref_sys_aux", "count": 4873}, {"table": "views_geometry_columns", "count": 0}, {"table": "virts_geometry_columns", "count": 0}, {"table": "geometry_columns_statistics", "count": 2}, {"table": "views_geometry_columns_statistics", "count": 0}, {"table": "virts_geometry_columns_statistics", "count": 0}, {"table": "geometry_columns_field_infos", "count": 0}, {"table": "views_geometry_columns_field_infos", "count": 0}, {"table": "virts_geometry_columns_field_infos", "count": 0}, {"table": "geometry_columns_time", "count": 2}, {"table": "geometry_columns_auth", "count": 2}, {"table": "views_geometry_columns_auth", "count": 0}, {"table": "virts_geometry_columns_auth", "count": 0}, Traceback (most recent call last): File "/usr/local/bin/sqlite-utils", line 8, in <module> sys.exit(cli()) File "/usr/local/Cellar/sqlite-utils/2.15.1/libexec/lib/python3.8/site-packages/click/core.py", line 829, in __call__ return self.main(*args, **kwargs) File "/usr/local/Cellar/sqlite-utils/2.15.1/libexec/lib/python3.8/site-packages/click/core.py", line 782, in main rv = self.invoke(ctx) File "/usr/local/Cellar/sqlite-utils/2.15.1/libexec/lib/python3.8/site-packages/click/core.py", line 1259, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/usr/local/Cellar/sqlite-utils/2.15.1/libexec/lib/python3.8/site-packages/click/core.py", line 1066, in invoke return ctx.invoke(self.callback, **ctx.params) File "/usr/local/Cellar/sqlite-utils/2.15.1/libexec/lib/python3.8/site-packages/click/core.py", line 610, in invoke return callback(*args, **kwargs) File "/usr/local/Cellar/sqlite-utils/2.15.1/libexec/lib/python3.8/site-packages/sqlite_utils/cli.py", line 143, in tables … | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/137/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
709920027 | MDU6SXNzdWU3MDk5MjAwMjc= | 181 | pk=["id"] should have same effect as pk="id" | 9599 | closed | 0 | 1 | 2020-09-28T04:28:07Z | 2020-10-14T21:59:47Z | 2020-10-14T21:59:47Z | OWNER | ``` In [11]: db['one'].insert({"id": 1, "name": "oentuh"}, pk="id") Out[11]: <Table one (id, name)> In [12]: db['two'].insert({"id": 1, "name": "oentuh"}, pk=["id"]) Out[12]: <Table two (id, name)> In [13]: db['one'].schema Out[13]: 'CREATE TABLE [one] (\n [id] INTEGER PRIMARY KEY,\n [name] TEXT\n)' In [14]: db['two'].schema Out[14]: 'CREATE TABLE [two] (\n [id] INTEGER,\n [name] TEXT\n)' ``` | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/181/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
718952107 | MDU6SXNzdWU3MTg5NTIxMDc= | 185 | Use db[table] consistently in documentation | 9599 | closed | 0 | 0 | 2020-10-11T23:39:12Z | 2020-10-12T00:13:41Z | 2020-10-12T00:13:41Z | OWNER | The Python docs have a bunch of examples like this: https://sqlite-utils.readthedocs.io/en/stable/python-api.html ```python dogs.enable_fts(["name", "twitter"], create_triggers=True) ``` This would be easier for people to understand if it looked like this instead: ```python db["dogs"].enable_fts(["name", "twitter"], create_triggers=True) ``` | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/185/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
716955793 | MDExOlB1bGxSZXF1ZXN0NDk5NjAzMzU5 | 184 | Test against Python 3.9 | 9599 | closed | 0 | 0 | 2020-10-08T01:37:05Z | 2020-10-08T01:44:06Z | 2020-10-08T01:44:06Z | OWNER | simonw/sqlite-utils/pulls/184 | 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/184/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||||
712316959 | MDU6SXNzdWU3MTIzMTY5NTk= | 183 | Try out GitHub code scanning | 9599 | closed | 0 | 1 | 2020-09-30T22:16:14Z | 2020-09-30T22:23:44Z | 2020-09-30T22:23:44Z | OWNER | https://github.blog/2020-09-30-code-scanning-is-now-available/ | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/183/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
684118950 | MDU6SXNzdWU2ODQxMTg5NTA= | 138 | extracts= doesn't configure foreign keys | 9599 | closed | 0 | 2 | 2020-08-23T05:21:15Z | 2020-09-24T22:47:01Z | 2020-09-24T22:46:52Z | OWNER | In using `extracts=` for `shapefiles-to-sqlite` in https://github.com/simonw/shapefile-to-sqlite/issues/9 I've run into a couple of pretty serious flaws: - The columns in the original table are still `TEXT` even when the foreign key they are supposed to reference is an `INTEGER` - which means Datasette foreign key features don't actually work - Those foreign key relationships aren't setup automatically - creating them is left as an exercise for the developer | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/138/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
652700770 | MDU6SXNzdWU2NTI3MDA3NzA= | 119 | Ability to remove a foreign key | 9599 | closed | 0 | 3 | 2020-07-07T22:31:37Z | 2020-09-24T20:36:59Z | 2020-09-24T20:36:59Z | OWNER | Useful if you add one but make a mistake and need to undo it without recreating the database from scratch. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/119/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
708261775 | MDU6SXNzdWU3MDgyNjE3NzU= | 175 | Add docs for .transform(column_order=) | 9599 | closed | 0 | 3 | 2020-09-24T15:19:04Z | 2020-09-24T20:35:48Z | 2020-09-24T16:00:56Z | OWNER | > Need to update docs for `.transform()` now that `column_order=` is available. _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/pull/174#discussion_r494403327_ Maybe also add this as an option to `sqlite-utils transform` - since reordering columns is actually a pretty nice capability. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/175/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
705190723 | MDU6SXNzdWU3MDUxOTA3MjM= | 160 | table.enable_fts(..., replace=True) | 9599 | closed | 0 | 5896742 | 1 | 2020-09-20T21:36:23Z | 2020-09-24T20:35:47Z | 2020-09-20T22:05:51Z | OWNER | I noticed that https://til.simonwillison.net/ search doesn't use porter stemming. I'd like to add that, but since [the build script](https://github.com/simonw/til/blob/9d3f0fca30e94df3970df52b0447907a077e4673/build_database.py) always operates on an existing database (to avoid re-rendering markdown and re-building image thumbnails) I'd like it to only add porter stemming if it's not there already. So I'd like to be able to say "set up FTS to look like this, and fix it if it doesn't". I think the neatest way to do that is with a `replace=True` argument to `.enable_fts()`, for consistency with `def .create_view(self, name, sql, replace=True)`. So the `replace=True` argument would check and see if the configured FTS exists already with the correct options (columns, stemming, triggers) - and if any of those are incorrect it would call `.disable_fts()` and then create a new FTS configuration with the correct options. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/160/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
706091046 | MDU6SXNzdWU3MDYwOTEwNDY= | 165 | Make .transform() a keyword arguments only function | 9599 | closed | 0 | 5897911 | 0 | 2020-09-22T05:37:29Z | 2020-09-24T20:35:47Z | 2020-09-22T06:39:12Z | OWNER | And rename the first argument from `columns=` to `types=` | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/165/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
695377804 | MDU6SXNzdWU2OTUzNzc4MDQ= | 153 | table.optimize() should delete junk rows from *_fts_docsize | 9599 | closed | 0 | 3 | 2020-09-07T20:31:09Z | 2020-09-24T20:35:46Z | 2020-09-07T21:16:33Z | OWNER | > The second challenge here is cleaning up all of those junk rows in existing `*_fts_docsize` tables. Doing that just to the demo database from https://github-to-sqlite.dogsheep.net/github.db dropped its size from 22MB to 16MB! Here's the SQL: > ```sql > DELETE FROM [licenses_fts_docsize] WHERE id NOT IN ( > SELECT rowid FROM [licenses_fts]); > ``` > I can do that as part of the existing `table.optimize()` method, which optimizes FTS tables. _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/149#issuecomment-688501064_ | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/153/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
696045581 | MDU6SXNzdWU2OTYwNDU1ODE= | 155 | rebuild-fts command and table.rebuild_fts() method | 9599 | closed | 0 | 2 | 2020-09-08T17:19:26Z | 2020-09-24T20:35:46Z | 2020-09-08T23:16:10Z | OWNER | https://sqlite.org/forum/forumpost/fa777fff86 > Easiest thing would be to run a 'rebuild' to rebuild the FTS index from scratch based on the contents of the content table. i.e. > > INSERT INTO licenses_fts(licenses_fts) VALUES('rebuild'); > > https://www.sqlite.org/fts5.html#the_rebuild_command | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/155/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
708301810 | MDU6SXNzdWU3MDgzMDE4MTA= | 177 | Simplify .transform(drop_foreign_keys=) and sqlite-transform --drop-foreign-key | 9599 | closed | 0 | 1 | 2020-09-24T16:13:50Z | 2020-09-24T20:35:03Z | 2020-09-24T16:19:13Z | OWNER | These both currently require you to provide three strings, for `column`, `other_table`, `other_column`. Just providing `column` should be enough information. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/177/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
708293114 | MDU6SXNzdWU3MDgyOTMxMTQ= | 176 | sqlite-utils transform column order option | 9599 | closed | 0 | 2 | 2020-09-24T16:01:21Z | 2020-09-24T20:34:51Z | 2020-09-24T16:11:59Z | OWNER | Split from #175 | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/176/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
697179806 | MDU6SXNzdWU2OTcxNzk4MDY= | 157 | sqlite-utils add-foreign-keys command | 9599 | closed | 0 | 5896742 | 2 | 2020-09-09T21:44:30Z | 2020-09-24T20:34:50Z | 2020-09-20T20:14:30Z | OWNER | Like `add-foreign-key` but can do multiple foreign keys at once. Inspired by https://github.com/simonw/calands-datasette/blob/99de39dd80a906f5c1f16724467b0cd55ba4ef36/build.sh which does this: ``` sqlite-utils add-foreign-key calands.db units_with_maps ACCESS_TYP sqlite-utils add-foreign-key calands.db units_with_maps AGNCY_NAME sqlite-utils add-foreign-key calands.db units_with_maps AGNCY_LEV sqlite-utils add-foreign-key calands.db units_with_maps AGNCY_TYP sqlite-utils add-foreign-key calands.db units_with_maps LAYER sqlite-utils add-foreign-key calands.db units_with_maps MNG_AGENCY sqlite-utils add-foreign-key calands.db units_with_maps MNG_AG_LEV sqlite-utils add-foreign-key calands.db units_with_maps MNG_AG_TYP sqlite-utils add-foreign-key calands.db units_with_maps COUNTY sqlite-utils add-foreign-key calands.db units_with_maps DES_TP ``` | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/157/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
706017416 | MDU6SXNzdWU3MDYwMTc0MTY= | 164 | sqlite-utils transform sub-command | 9599 | closed | 0 | 5897911 | 4 | 2020-09-22T01:32:20Z | 2020-09-24T20:34:50Z | 2020-09-22T07:48:05Z | OWNER | The `.transform()` method in #114 warrants an equivalent CLI tool. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/164/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
706757891 | MDU6SXNzdWU3MDY3NTc4OTE= | 169 | Progress bar for "sqlite-utils extract" | 9599 | closed | 0 | 5897911 | 0 | 2020-09-22T23:40:21Z | 2020-09-24T20:34:50Z | 2020-09-23T00:02:40Z | OWNER | > Since these operations could take a long time against large tables, it would be neat if there was a progress bar option for the CLI command. > > The operations are full table scans so calculating progress shouldn't be too difficult. _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/42#issuecomment-513246831_ | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/169/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
707944044 | MDExOlB1bGxSZXF1ZXN0NDkyMjU3NDA1 | 174 | Much, much faster extract() implementation | 9599 | closed | 0 | 7 | 2020-09-24T07:52:31Z | 2020-09-24T15:44:00Z | 2020-09-24T15:43:56Z | OWNER | simonw/sqlite-utils/pulls/174 | Takes my test down from ten minutes to four seconds. Refs #172. | 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/174/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
707427200 | MDU6SXNzdWU3MDc0MjcyMDA= | 172 | Improve performance of extract operations | 9599 | closed | 0 | 9 | 2020-09-23T14:40:50Z | 2020-09-24T15:43:57Z | 2020-09-24T15:43:57Z | OWNER | This command took about 12 minutes (against a 150MB file with 680,000 rows in it): ``` sqlite-utils extract salaries.db salaries \ 'Organization Group Code' 'Organization Group' \ --table 'organization_groups' \ --fk-column 'organization_group_id' \ --rename 'Organization Group Code' code \ --rename 'Organization Group' name ``` I'm pretty confident we can do better than that. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/172/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
706768798 | MDU6SXNzdWU3MDY3Njg3OTg= | 170 | Release notes for 2.20 | 9599 | closed | 0 | 5897911 | 1 | 2020-09-23T00:13:22Z | 2020-09-23T00:31:25Z | 2020-09-23T00:31:25Z | OWNER | https://github.com/simonw/sqlite-utils/compare/2.19...b8e004 | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/170/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
706098005 | MDU6SXNzdWU3MDYwOTgwMDU= | 167 | Review the foreign key pragma stuff | 9599 | closed | 0 | 5897911 | 1 | 2020-09-22T05:55:20Z | 2020-09-23T00:13:02Z | 2020-09-23T00:13:02Z | OWNER | > It is not possible to enable or disable foreign key constraints in the middle of a multi-statement transaction (when SQLite is not in autocommit mode). Attempting to do so does not return an error; it simply has no effect. https://sqlite.org/foreignkeys.html | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/167/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed |