issues
604 rows where repo = 140912432 sorted by draft
This data as json, CSV (advanced)
updated_at (date) >30 ✖
- 2023-05-08 19
- 2020-09-24 14
- 2023-07-22 12
- 2022-08-27 11
- 2022-01-11 10
- 2023-05-21 10
- 2021-08-18 9
- 2021-02-14 8
- 2021-05-19 8
- 2021-06-19 8
- 2021-08-02 8
- 2022-01-10 8
- 2023-11-04 8
- 2021-01-03 7
- 2022-06-14 7
- 2022-10-25 7
- 2020-09-07 6
- 2021-11-15 6
- 2022-01-26 6
- 2022-07-15 6
- 2023-08-18 6
- 2021-12-11 5
- 2022-02-03 5
- 2022-02-05 5
- 2020-09-22 4
- 2020-11-08 4
- 2021-01-02 4
- 2021-02-25 4
- 2021-05-29 4
- 2021-06-18 4
- …
repo 1
- sqlite-utils · 504 ✖
id | node_id | number | title | user | state | locked | assignee | milestone | comments | created_at | updated_at | closed_at | author_association | pull_request | body | repo | type | active_lock_reason | performed_via_github_app | reactions | draft ▼ | state_reason |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
666639051 | MDU6SXNzdWU2NjY2MzkwNTE= | 128 | Support UUID and memoryview types | simonw 9599 | closed | 0 | 1 | 2020-07-27T23:08:34Z | 2020-07-30T01:10:43Z | 2020-07-30T01:10:43Z | OWNER |
|
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/128/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
668308777 | MDU6SXNzdWU2NjgzMDg3Nzc= | 129 | "insert-files --sqlar" for creating SQLite archives | simonw 9599 | closed | 0 | 2 | 2020-07-30T02:28:29Z | 2020-07-30T22:41:01Z | 2020-07-30T22:40:55Z | OWNER | A https://www.sqlite.org/sqlar.html and https://sqlite.org/sqlar/doc/trunk/README.md |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/129/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
671130371 | MDU6SXNzdWU2NzExMzAzNzE= | 130 | Support tokenize option for FTS | simonw 9599 | closed | 0 | 3 | 2020-08-01T19:27:22Z | 2020-08-01T20:51:28Z | 2020-08-01T20:51:14Z | OWNER | FTS5 supports things like porter stemming using a https://www.sqlite.org/fts5.html#tokenizers Something like this in code:
So... |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/130/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
675753042 | MDU6SXNzdWU2NzU3NTMwNDI= | 131 | sqlite-utils insert: options for column types | simonw 9599 | open | 0 | 5 | 2020-08-09T18:59:11Z | 2022-03-15T13:21:42Z | OWNER | The It would be useful if you could do the following:
For specific columns maybe this:
|
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/131/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
675839512 | MDU6SXNzdWU2NzU4Mzk1MTI= | 132 | Features for enabling and disabling WAL mode | simonw 9599 | closed | 0 | 5 | 2020-08-10T03:25:44Z | 2020-08-10T18:59:35Z | 2020-08-10T18:59:35Z | OWNER | I finally figured out how to enable WAL - turns out it's a property of the database file itself: https://github.com/simonw/til/blob/master/sqlite/enabling-wal-mode.md |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/132/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
677839979 | MDU6SXNzdWU2Nzc4Mzk5Nzk= | 133 | Release a sdist to PyPI | simonw 9599 | closed | 0 | 1 | 2020-08-12T16:55:09Z | 2020-08-12T17:05:06Z | 2020-08-12T17:05:06Z | OWNER | https://pypi.org/project/sqlite-utils/#files currently just has a wheel. I need this to package for homebrew: https://github.com/simonw/homebrew-datasette/issues/10 |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/133/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
683804172 | MDU6SXNzdWU2ODM4MDQxNzI= | 134 | --load-extension option for sqlite-utils query | simonw 9599 | closed | 0 | 4 | 2020-08-21T20:12:42Z | 2020-08-21T21:06:26Z | 2020-08-21T20:54:19Z | OWNER | I got this error:
|
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/134/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
683805434 | MDU6SXNzdWU2ODM4MDU0MzQ= | 135 | Code for finding SpatiaLite in the usual locations | simonw 9599 | closed | 0 | 3 | 2020-08-21T20:15:34Z | 2022-02-05T00:04:26Z | 2020-08-21T20:30:13Z | OWNER | I built this for
|
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/135/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
683812642 | MDU6SXNzdWU2ODM4MTI2NDI= | 136 | --load-extension=spatialite shortcut option | simonw 9599 | closed | 0 | 3 | 2020-08-21T20:31:25Z | 2022-02-05T00:04:26Z | 2020-10-16T19:14:32Z | OWNER | In conjunction with #135 - this would do the same thing as |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/136/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
683830416 | MDU6SXNzdWU2ODM4MzA0MTY= | 137 | --load-extension for other sqlite-utils commands | simonw 9599 | closed | 0 | 1 | 2020-08-21T21:12:56Z | 2020-10-16T19:14:32Z | 2020-10-16T19:14:32Z | OWNER | e.g. for this:
Follow-on from #134 |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/137/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
684118950 | MDU6SXNzdWU2ODQxMTg5NTA= | 138 | extracts= doesn't configure foreign keys | simonw 9599 | closed | 0 | 2 | 2020-08-23T05:21:15Z | 2020-09-24T22:47:01Z | 2020-09-24T22:46:52Z | OWNER | In using
|
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/138/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
686978131 | MDU6SXNzdWU2ODY5NzgxMzE= | 139 | insert_all(..., alter=True) should work for new columns introduced after the first 100 records | simonwiles 96218 | closed | 0 | 7 | 2020-08-27T06:25:25Z | 2020-08-28T22:48:51Z | 2020-08-28T22:30:14Z | CONTRIBUTOR | Is there a way to make I'm using It took me a while to find this little snippet in the documentation for
I tried changing the Is there a way around this that you would suggest? It seems like it should raise an exception at least. |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/139/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
688351054 | MDU6SXNzdWU2ODgzNTEwNTQ= | 140 | Idea: insert-files mechanism for adding extra columns with fixed values | simonw 9599 | open | 0 | 1 | 2020-08-28T20:57:36Z | 2022-03-20T19:45:45Z | OWNER | Say for example you want to populate a
|
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/140/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
688352145 | MDU6SXNzdWU2ODgzNTIxNDU= | 141 | insert-files support for compressed values | simonw 9599 | open | 0 | 0 | 2020-08-28T20:59:46Z | 2020-09-24T20:36:08Z | OWNER | The |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/141/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
688389933 | MDU6SXNzdWU2ODgzODk5MzM= | 143 | Move to GitHub Actions CI | simonw 9599 | closed | 0 | 1 | 2020-08-28T22:34:11Z | 2020-08-28T22:41:35Z | 2020-08-28T22:41:35Z | OWNER | sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/143/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||||
688395275 | MDU6SXNzdWU2ODgzOTUyNzU= | 144 | Run some tests against numpy | simonw 9599 | closed | 0 | 2 | 2020-08-28T22:53:00Z | 2020-08-28T22:57:05Z | 2020-08-28T22:57:04Z | OWNER | Accidentally removed in #143: |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/144/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
688659182 | MDU6SXNzdWU2ODg2NTkxODI= | 145 | Bug when first record contains fewer columns than subsequent records | simonwiles 96218 | closed | 0 | 2 | 2020-08-30T05:44:44Z | 2020-09-08T23:21:23Z | 2020-09-08T23:21:23Z | CONTRIBUTOR |
I suspect that this bug is masked somewhat by the fact that while:
it is common that it is increased at compile time. Debian-based systems, for example, seem to ship with a version of sqlite compiled with A test for this issue might look like this:
The best solution, I think, is simply to process all the records when determining columns, column types, and the batch size. In my tests this doesn't seem to be particularly costly at all, and cuts out a lot of complications (including obviating my implementation of #139 at #142). I'll raise a PR for your consideration. |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/145/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
688670158 | MDU6SXNzdWU2ODg2NzAxNTg= | 147 | SQLITE_MAX_VARS maybe hard-coded too low | simonwiles 96218 | open | 0 | 7 | 2020-08-30T07:26:45Z | 2021-02-15T21:27:55Z | CONTRIBUTOR | I came across this while about to open an issue and PR against the documentation for As mentioned in #145, while:
it is common that it is increased at compile time. Debian-based systems, for example, seem to ship with a version of sqlite compiled with SQLITE_MAX_VARIABLE_NUMBER set to 250,000, and I believe this is the case for homebrew installations too. In working to understand what Unfortunately, it seems that Obviously this couldn't be relied upon in |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/147/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
695276328 | MDU6SXNzdWU2OTUyNzYzMjg= | 148 | More attractive indentation of created FTS table schema | simonw 9599 | closed | 0 | 1 | 2020-09-07T16:49:30Z | 2020-09-07T18:12:50Z | 2020-09-07T18:12:50Z | OWNER | On https://github-to-sqlite.dogsheep.net/github/licenses_fts the create table SQL is displayed as:
|
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/148/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
695319258 | MDU6SXNzdWU2OTUzMTkyNTg= | 149 | FTS table with 7 rows has _fts_docsize table with 9,141 rows | simonw 9599 | closed | 0 | 10 | 2020-09-07T18:06:16Z | 2020-09-07T21:16:34Z | 2020-09-07T21:16:34Z | OWNER | I'm seeing a weird issue with some of the SQLite databases that I am using with the FTS5 module. I have a database with a The FTS table also has 7 rows: https://github-to-sqlite.dogsheep.net/github/licenses_fts Somehow the accompanying And I have a hunch that it might be a problem with the triggers. These are the triggers that are updating that FTS table: https://github-to-sqlite.dogsheep.net/github?sql=select+*+from+sqlite_master+where+type+%3D+%27trigger%27+and+tbl_name+%3D+%27licenses%27 | type | name | tbl_name | rootpage | sql |
| --- | --- | --- | --- | --- |
| trigger | licenses_ai | licenses | 0 | |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/149/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
695359607 | MDU6SXNzdWU2OTUzNTk2MDc= | 150 | Feature for tracing SQL queries | simonw 9599 | closed | 0 | 0 | 2020-09-07T19:43:08Z | 2020-09-07T21:57:01Z | 2020-09-07T21:57:01Z | OWNER | Debugging An optional "tracing" mechanism for seeing what SQL is being executed would be useful. |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/150/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
695376054 | MDU6SXNzdWU2OTUzNzYwNTQ= | 152 | Turn on recursive_triggers by default | simonw 9599 | closed | 0 | 2 | 2020-09-07T20:26:36Z | 2020-09-07T21:17:48Z | 2020-09-07T20:45:14Z | OWNER | https://www.sqlite.org/pragma.html#pragma_recursive_triggers says:
So I think the fix for the complex issue in #149 is to turn on Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/149#issuecomment-688499924 |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/152/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
695377804 | MDU6SXNzdWU2OTUzNzc4MDQ= | 153 | table.optimize() should delete junk rows from *_fts_docsize | simonw 9599 | closed | 0 | 3 | 2020-09-07T20:31:09Z | 2020-09-24T20:35:46Z | 2020-09-07T21:16:33Z | OWNER |
|
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/153/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
695441530 | MDU6SXNzdWU2OTU0NDE1MzA= | 154 | OperationalError: cannot change into wal mode from within a transaction | simonw 9599 | open | 0 | 2 | 2020-09-07T23:42:44Z | 2020-09-07T23:47:10Z | OWNER | I'm getting this error when running:
I'm worried that maybe that's because of this new code from #152: |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/154/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
696045581 | MDU6SXNzdWU2OTYwNDU1ODE= | 155 | rebuild-fts command and table.rebuild_fts() method | simonw 9599 | closed | 0 | 2 | 2020-09-08T17:19:26Z | 2020-09-24T20:35:46Z | 2020-09-08T23:16:10Z | OWNER | https://sqlite.org/forum/forumpost/fa777fff86
|
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/155/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
697179806 | MDU6SXNzdWU2OTcxNzk4MDY= | 157 | sqlite-utils add-foreign-keys command | simonw 9599 | closed | 0 | 2.19 5896742 | 2 | 2020-09-09T21:44:30Z | 2020-09-24T20:34:50Z | 2020-09-20T20:14:30Z | OWNER | Like
|
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/157/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
702386948 | MDU6SXNzdWU3MDIzODY5NDg= | 159 | .delete_where() does not auto-commit (unlike .insert() or .upsert()) | spdkils 11712349 | open | 0 | 9 | 2020-09-16T01:55:52Z | 2023-04-01T17:21:05Z | NONE | When you use the delete_where() function on a table, it never commits.... Is that intentional? |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/159/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
705190723 | MDU6SXNzdWU3MDUxOTA3MjM= | 160 | table.enable_fts(..., replace=True) | simonw 9599 | closed | 0 | 2.19 5896742 | 1 | 2020-09-20T21:36:23Z | 2020-09-24T20:35:47Z | 2020-09-20T22:05:51Z | OWNER | I noticed that https://til.simonwillison.net/ search doesn't use porter stemming. I'd like to add that, but since the build script always operates on an existing database (to avoid re-rendering markdown and re-building image thumbnails) I'd like it to only add porter stemming if it's not there already. So I'd like to be able to say "set up FTS to look like this, and fix it if it doesn't". I think the neatest way to do that is with a So the |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/160/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
705995722 | MDU6SXNzdWU3MDU5OTU3MjI= | 162 | A decorator for registering custom SQL functions | simonw 9599 | closed | 0 | 2 | 2020-09-22T00:18:32Z | 2020-09-22T00:40:44Z | 2020-09-22T00:32:17Z | OWNER | Syntactic sugar for ```python db = sqlite_utils.Database("mydb.db") @db.register_function
def scramble(text):
chars = list(text)
random.shuffle(chars)
return "".join(chars)
|
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/162/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
706001517 | MDU6SXNzdWU3MDYwMDE1MTc= | 163 | Idea: conversions= could take Python functions | simonw 9599 | open | 0 | 4 | 2020-09-22T00:37:12Z | 2021-12-20T00:56:52Z | OWNER | Right now you use
|
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/163/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
706017416 | MDU6SXNzdWU3MDYwMTc0MTY= | 164 | sqlite-utils transform sub-command | simonw 9599 | closed | 0 | 2.20 5897911 | 4 | 2020-09-22T01:32:20Z | 2020-09-24T20:34:50Z | 2020-09-22T07:48:05Z | OWNER | The |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/164/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
706091046 | MDU6SXNzdWU3MDYwOTEwNDY= | 165 | Make .transform() a keyword arguments only function | simonw 9599 | closed | 0 | 2.20 5897911 | 0 | 2020-09-22T05:37:29Z | 2020-09-24T20:35:47Z | 2020-09-22T06:39:12Z | OWNER | And rename the first argument from |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/165/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
706098005 | MDU6SXNzdWU3MDYwOTgwMDU= | 167 | Review the foreign key pragma stuff | simonw 9599 | closed | 0 | 2.20 5897911 | 1 | 2020-09-22T05:55:20Z | 2020-09-23T00:13:02Z | 2020-09-23T00:13:02Z | OWNER |
|
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/167/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
706167456 | MDU6SXNzdWU3MDYxNjc0NTY= | 168 | Automate (as much as possible) updates published to Homebrew | simonw 9599 | closed | 0 | 2 | 2020-09-22T08:08:37Z | 2020-11-09T07:43:30Z | 2020-11-09T07:43:30Z | OWNER | I'd like to get new |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/168/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
706757891 | MDU6SXNzdWU3MDY3NTc4OTE= | 169 | Progress bar for "sqlite-utils extract" | simonw 9599 | closed | 0 | 2.20 5897911 | 0 | 2020-09-22T23:40:21Z | 2020-09-24T20:34:50Z | 2020-09-23T00:02:40Z | OWNER |
|
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/169/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
706768798 | MDU6SXNzdWU3MDY3Njg3OTg= | 170 | Release notes for 2.20 | simonw 9599 | closed | 0 | 2.20 5897911 | 1 | 2020-09-23T00:13:22Z | 2020-09-23T00:31:25Z | 2020-09-23T00:31:25Z | OWNER | sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/170/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
707407567 | MDU6SXNzdWU3MDc0MDc1Njc= | 171 | Idea: transitive closure tables for tree structures | mhalle 649467 | closed | 0 | 2 | 2020-09-23T14:17:33Z | 2020-10-22T04:38:35Z | 2020-10-22T04:07:14Z | NONE | I just read that sqlite has a transitive closure table extension using a virtual table in order to represent trees: Even without this extension, though, a util to build a transitive closure table would allow trees to be queried easily. Since it relies on self-referential foreign keys, the relationships might even be able to be automatically detected. |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/171/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
707427200 | MDU6SXNzdWU3MDc0MjcyMDA= | 172 | Improve performance of extract operations | simonw 9599 | closed | 0 | 9 | 2020-09-23T14:40:50Z | 2020-09-24T15:43:57Z | 2020-09-24T15:43:57Z | OWNER | This command took about 12 minutes (against a 150MB file with 680,000 rows in it):
|
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/172/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
707478649 | MDU6SXNzdWU3MDc0Nzg2NDk= | 173 | Progress bar for sqlite-utils insert | simonw 9599 | closed | 0 | 6 | 2020-09-23T15:43:56Z | 2021-11-01T08:42:24Z | 2020-10-27T18:16:04Z | OWNER | It would be nice if |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/173/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
708261775 | MDU6SXNzdWU3MDgyNjE3NzU= | 175 | Add docs for .transform(column_order=) | simonw 9599 | closed | 0 | 3 | 2020-09-24T15:19:04Z | 2020-09-24T20:35:48Z | 2020-09-24T16:00:56Z | OWNER |
Maybe also add this as an option to |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/175/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
708293114 | MDU6SXNzdWU3MDgyOTMxMTQ= | 176 | sqlite-utils transform column order option | simonw 9599 | closed | 0 | 2 | 2020-09-24T16:01:21Z | 2020-09-24T20:34:51Z | 2020-09-24T16:11:59Z | OWNER | Split from #175 |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/176/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
708301810 | MDU6SXNzdWU3MDgzMDE4MTA= | 177 | Simplify .transform(drop_foreign_keys=) and sqlite-transform --drop-foreign-key | simonw 9599 | closed | 0 | 1 | 2020-09-24T16:13:50Z | 2020-09-24T20:35:03Z | 2020-09-24T16:19:13Z | OWNER | These both currently require you to provide three strings, for Just providing |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/177/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
709577625 | MDU6SXNzdWU3MDk1Nzc2MjU= | 179 | sqlite-utils transform/insert --detect-types | simonw 9599 | closed | 0 | 4 | 2020-09-26T17:28:55Z | 2021-06-19T03:36:16Z | 2021-06-19T03:36:05Z | OWNER | Idea from https://github.com/simonw/datasette-edit-tables/issues/13 - provide Python utility methods and accompanying CLI options for detecting the likely types of TEXT columns. So if you have a text column that actually contained exclusively integer string values, it can let you know and let you run transform against it. |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/179/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
709861194 | MDU6SXNzdWU3MDk4NjExOTQ= | 180 | Try running some tests using Hypothesis | simonw 9599 | closed | 0 | 1 | 2020-09-28T01:11:30Z | 2020-10-19T04:51:55Z | 2020-10-19T04:51:55Z | OWNER | Inspired by this Twitter conversation: https://twitter.com/simonw/status/1310386009465479168 |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/180/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
709920027 | MDU6SXNzdWU3MDk5MjAwMjc= | 181 | pk=["id"] should have same effect as pk="id" | simonw 9599 | closed | 0 | 1 | 2020-09-28T04:28:07Z | 2020-10-14T21:59:47Z | 2020-10-14T21:59:47Z | OWNER | ``` In [11]: db['one'].insert({"id": 1, "name": "oentuh"}, pk="id") Out[11]: <Table one (id, name)> In [12]: db['two'].insert({"id": 1, "name": "oentuh"}, pk=["id"]) Out[12]: <Table two (id, name)> In [13]: db['one'].schema Out[13]: 'CREATE TABLE [one] (\n [id] INTEGER PRIMARY KEY,\n [name] TEXT\n)' In [14]: db['two'].schema Out[14]: 'CREATE TABLE [two] (\n [id] INTEGER,\n [name] TEXT\n)' ``` |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/181/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
711649325 | MDU6SXNzdWU3MTE2NDkzMjU= | 182 | Better handling of encodings other than utf-8 for "sqlite-utils insert" | kaihendry 765871 | closed | 0 | 5 | 2020-09-30T05:43:48Z | 2020-10-16T17:20:41Z | 2020-10-16T17:18:52Z | NONE | Makefile: ``` data.db: curl -O http://maps.natalian.org/data.txt go run csv-write.go > data.csv sqlite-utils insert data.db travels data.csv --csv clean: rm data* ``` csv-write.go Error message is:
Little bit surprised if Go is spewing out bad Unicode, but I'm not sure how to grok |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/182/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
712316959 | MDU6SXNzdWU3MTIzMTY5NTk= | 183 | Try out GitHub code scanning | simonw 9599 | closed | 0 | 1 | 2020-09-30T22:16:14Z | 2020-09-30T22:23:44Z | 2020-09-30T22:23:44Z | OWNER | https://github.blog/2020-09-30-code-scanning-is-now-available/ |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/183/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
718952107 | MDU6SXNzdWU3MTg5NTIxMDc= | 185 | Use db[table] consistently in documentation | simonw 9599 | closed | 0 | 0 | 2020-10-11T23:39:12Z | 2020-10-12T00:13:41Z | 2020-10-12T00:13:41Z | OWNER | The Python docs have a bunch of examples like this: https://sqlite-utils.readthedocs.io/en/stable/python-api.html
|
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/185/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
722816436 | MDU6SXNzdWU3MjI4MTY0MzY= | 186 | .extract() shouldn't extract null values | simonw 9599 | open | 0 | 7 | 2020-10-16T02:41:08Z | 2021-08-12T12:32:14Z | OWNER | This almost works, but it creates a rogue |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/186/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
723460107 | MDU6SXNzdWU3MjM0NjAxMDc= | 187 | Maybe: Utility method / CLI tool for initializing SpatiaLite | simonw 9599 | closed | 0 | 2 | 2020-10-16T19:04:03Z | 2022-02-05T00:04:26Z | 2020-10-16T19:15:13Z | OWNER |
|
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/187/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
723708310 | MDU6SXNzdWU3MjM3MDgzMTA= | 188 | About loading spatialite | aborruso 30607 | closed | 0 | 1 | 2020-10-17T08:47:02Z | 2022-02-05T00:04:26Z | 2020-10-17T08:52:58Z | NONE | Hi @simonw , If I run
I have If I run
I have
How to load properly spatialite extension in sqlite-utils? Thank you very muc |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/188/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
731740458 | MDU6SXNzdWU3MzE3NDA0NTg= | 191 | Idea: @db.register_function(deterministic=True) | simonw 9599 | closed | 0 | 2 | 2020-10-28T19:45:18Z | 2020-10-28T21:31:06Z | 2020-10-28T21:31:06Z | OWNER | Python 3.8 added a
https://sqlite-utils.readthedocs.io/en/stable/python-api.html#registering-custom-sql-functions ```python db = Database(memory=True) @db.register_function(deterministic=True) def reverse_string(s): return "".join(reversed(list(s))) ``` |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/191/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
735532751 | MDU6SXNzdWU3MzU1MzI3NTE= | 192 | sqlite-utils search command | simonw 9599 | closed | 0 | 3.0 6079500 | 9 | 2020-11-03T18:07:59Z | 2020-11-08T17:07:01Z | 2020-11-08T17:07:01Z | OWNER | A command that knows how to run a search against a FTS enabled table and return results ranked by relevance. |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/192/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
735648209 | MDU6SXNzdWU3MzU2NDgyMDk= | 193 | --tsv output format option | simonw 9599 | closed | 0 | 3.0 6079500 | 0 | 2020-11-03T21:31:18Z | 2020-11-07T00:09:52Z | 2020-11-07T00:09:52Z | OWNER | We already support |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/193/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
735650864 | MDU6SXNzdWU3MzU2NTA4NjQ= | 194 | 3.0 release with some minor breaking changes | simonw 9599 | closed | 0 | 3.0 6079500 | 3 | 2020-11-03T21:36:31Z | 2020-11-08T17:19:35Z | 2020-11-08T17:19:34Z | OWNER | While working on search (#192) I've spotted a few small changes I would like to make that would break backwards compatibility in minor ways, hence requiring a 3.x release.
Also I'd like to free up the I'd like |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/194/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
736520310 | MDU6SXNzdWU3MzY1MjAzMTA= | 196 | Introspect if table is FTS4 or FTS5 | simonw 9599 | closed | 0 | 19 | 2020-11-05T00:45:50Z | 2020-11-05T03:54:07Z | 2020-11-05T03:54:07Z | OWNER |
Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/192#issuecomment-722054264 |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/196/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
737153927 | MDU6SXNzdWU3MzcxNTM5Mjc= | 197 | Rethink how table.search() method works | simonw 9599 | closed | 0 | 3.0 6079500 | 5 | 2020-11-05T18:04:34Z | 2020-11-08T17:07:37Z | 2020-11-08T17:07:37Z | OWNER | I need to improve this method to help build The challenge is deciding how it should handle sorting by relevance - especially since that is easy in FTS5 but not at all easy in FTS4.
Originally posted by @simonw in https://github.com/simonw/sqlite-utils/pull/195#issuecomment-722542895 |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/197/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
737476423 | MDU6SXNzdWU3Mzc0NzY0MjM= | 198 | Support order by relevance against FTS4 | simonw 9599 | closed | 0 | 6 | 2020-11-06T05:36:31Z | 2020-11-06T18:30:44Z | 2020-11-06T18:30:44Z | OWNER | For #192 and #197 I've decided I want to be able to order by relevance in FTS4 as well as FTS5. This means I need to port over my work on bm25() from https://github.com/simonw/sqlite-fts4 (since I don't want to add a full dependency). |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/198/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
737855731 | MDU6SXNzdWU3Mzc4NTU3MzE= | 199 | @db.register_function(..., replace=False) to avoid double-registering custom functions | simonw 9599 | closed | 0 | 1 | 2020-11-06T15:39:21Z | 2020-11-06T18:30:44Z | 2020-11-06T18:30:44Z | OWNER | I'd like a mechanism to optionally avoid registering a custom function if it has already been registered. SQLite doesn't seem to offer a way to introspect registered custom functions so I'll need to track what has already been registered in
Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/198#issuecomment-723145383 |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/199/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
738115165 | MDU6SXNzdWU3MzgxMTUxNjU= | 200 | sqlite-utils rows -c option | simonw 9599 | closed | 0 | 3.0 6079500 | 1 | 2020-11-07T00:22:12Z | 2020-11-07T00:28:48Z | 2020-11-07T00:28:47Z | OWNER | To let you specify the exact columns you want. Based on the |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/200/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
738128913 | MDU6SXNzdWU3MzgxMjg5MTM= | 201 | .search(columns=) and sqlite-utils search -c ... bug | simonw 9599 | closed | 0 | 3.0 6079500 | 1 | 2020-11-07T01:27:26Z | 2020-11-08T16:54:15Z | 2020-11-08T16:54:15Z | OWNER | Both This should be fixed before the 3.0 non-alpha release. |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/201/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
738514367 | MDU6SXNzdWU3Mzg1MTQzNjc= | 202 | sqlite-utils insert -f colname - for configuring full-text search | simonw 9599 | closed | 0 | 2 | 2020-11-08T17:30:09Z | 2021-01-03T05:00:36Z | 2021-01-03T05:00:27Z | OWNER | A mechanism for specifying columns that should be configured for full-text search as part of the initial data import:
|
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/202/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
760960559 | MDU6SXNzdWU3NjA5NjA1NTk= | 205 | sqlite3.OperationalError: near "(": syntax error | kaihendry 765871 | closed | 0 | 2 | 2020-12-10T06:44:40Z | 2020-12-10T19:18:22Z | 2020-12-10T07:24:23Z | NONE | The sqlite version is 3.22.0 2018-01-22 18:45:57 0c55d179733b46d8d0ba4d88e01a25e10677046ee3da1d5b1581e86726f2alt1 sqlite-utils, version 3.0 It fails here: https://github.com/kaihendry/aws-partners-datasette/runs/1528432635?check_suite_focus=true I'm not sure where the problem is, since it works fine locally on Archlinux system running 3.34.0 2020-12-01 16:14:00 a26b6597e3ae272231b96f9982c3bcc17ddec2f2b6eb4df06a224b91089fed5b https://github.com/kaihendry/aws-partners-datasette/blob/main/create-summary-view.sh Maybe I need to bump up from ubuntu-latest to ? |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/205/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
761915790 | MDU6SXNzdWU3NjE5MTU3OTA= | 206 | sqlite-utils should suggest --csv if JSON parsing fails | simonw 9599 | closed | 0 | 4 | 2020-12-11T05:17:56Z | 2021-10-30T15:52:17Z | 2021-01-03T18:42:22Z | OWNER |
|
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/206/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
763283616 | MDU6SXNzdWU3NjMyODM2MTY= | 207 | sqlite-utils analyze-tables command | simonw 9599 | closed | 0 | 4 | 2020-12-12T04:33:12Z | 2020-12-13T07:25:23Z | 2020-12-13T07:20:13Z | OWNER | A command which analyzes a table (potentially taking quite a while if the table is large) and outputs information for each column - things like:
The command can output this information to the terminal, but it should also provide an option for writing the information to a database table so it can be explored later. |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/207/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
766156875 | MDU6SXNzdWU3NjYxNTY4NzU= | 209 | Test failure with sqlite 3.34 in test_cli.py::test_optimize | meatcar 191622 | closed | 0 | 1 | 2020-12-14T08:58:18Z | 2021-01-01T23:52:46Z | 2021-01-01T23:52:46Z | CONTRIBUTOR | pytest output:
Came across this while packaging ``` docker run --rm -it alpine:edge /bin/sh apk update && apk add git sqlite python3 gcc python3-dev musl-dev && python3 -m ensurepipgit clone https://github.com/simonw/sqlite-utils.gitcd sqlite-utils/pip3 install -e .[test]pytest``` This definitely works on sqlite v3.33. |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/209/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
767685961 | MDU6SXNzdWU3Njc2ODU5NjE= | 210 | Support of RData files | PeterBailey 23739126 | closed | 0 | 1 | 2020-12-15T15:04:14Z | 2021-01-02T00:02:40Z | 2021-01-02T00:02:40Z | NONE | Hi Simon, Would be great if you could ingest RData files! I could do this in a few lines of code but I am too lazy - sorry! Peter |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/210/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
777386465 | MDU6SXNzdWU3NzczODY0NjU= | 211 | table.triggers_dict introspection property | simonw 9599 | closed | 0 | 0 | 2021-01-02T02:04:00Z | 2021-01-02T02:10:10Z | 2021-01-02T02:10:10Z | OWNER |
|
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/211/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
777392020 | MDU6SXNzdWU3NzczOTIwMjA= | 212 | Mechanism for maintaining cache of table counts using triggers | simonw 9599 | closed | 0 | 1 | 2021-01-02T02:58:53Z | 2021-01-02T21:40:27Z | 2021-01-02T21:40:27Z | OWNER | Counting all of the rows in a large table is expensive - this is one of the main causes of performance problems in Datasette when running against large databases. Carefully constructed SQL triggers could be used to maintain accurate cached counts for a table, by incrementing and decrementing a counter every time a row is inserted or deleted.
|
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/212/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
777529979 | MDU6SXNzdWU3Nzc1Mjk5Nzk= | 213 | db.enable_counts() method | simonw 9599 | closed | 0 | 2 | 2021-01-02T21:44:55Z | 2021-01-02T22:04:02Z | 2021-01-02T22:04:02Z | OWNER | Following #212 it would be useful if there was a utility method for enabling counts for ALL tables in a database:
Open question: should this setup triggers for virtual tables such as FTS tables? Could that break things? |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/213/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
777530107 | MDU6SXNzdWU3Nzc1MzAxMDc= | 214 | sqlite-utils enable-counts command | simonw 9599 | closed | 0 | 0 | 2021-01-02T21:45:48Z | 2021-01-03T04:26:44Z | 2021-01-03T04:26:44Z | OWNER | The CLI version of #212 and #213.
|
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/214/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
777535402 | MDU6SXNzdWU3Nzc1MzU0MDI= | 215 | Use _counts to speed up counts | simonw 9599 | closed | 0 | 9 | 2021-01-02T22:30:17Z | 2021-01-03T20:19:40Z | 2021-01-03T20:19:40Z | OWNER | Utility mechanism for taking advantage of the new These can trigger automatically if the |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/215/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
777540352 | MDU6SXNzdWU3Nzc1NDAzNTI= | 216 | database.triggers_dict introspection property | simonw 9599 | closed | 0 | 1 | 2021-01-02T23:13:00Z | 2021-01-03T04:27:14Z | 2021-01-03T04:25:36Z | OWNER | Following #211 |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/216/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
777543336 | MDU6SXNzdWU3Nzc1NDMzMzY= | 217 | Rename .escape() to .quote() | simonw 9599 | closed | 0 | 2 | 2021-01-02T23:40:52Z | 2021-01-03T04:27:38Z | 2021-01-03T04:15:23Z | OWNER |
This method has never been documented so I'm going to rename it without a major version bump. |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/217/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
777560474 | MDU6SXNzdWU3Nzc1NjA0NzQ= | 218 | "sqlite-utils triggers" command | simonw 9599 | closed | 0 | 1 | 2021-01-03T02:34:50Z | 2021-01-03T03:49:51Z | 2021-01-03T03:03:35Z | OWNER | A command to list the triggers in the database.
Can optionally take one or more tables:
|
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/218/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
777707544 | MDU6SXNzdWU3Nzc3MDc1NDQ= | 219 | reset_counts() method and command | simonw 9599 | closed | 0 | 4 | 2021-01-03T20:08:28Z | 2021-01-03T20:59:37Z | 2021-01-03T20:59:37Z | OWNER |
Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/215#issuecomment-753545757 |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/219/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
783778672 | MDU6SXNzdWU3ODM3Nzg2NzI= | 220 | Better error message for *_fts methods against views | mhalle 649467 | closed | 0 | 3 | 2021-01-11T23:24:00Z | 2021-02-22T20:44:51Z | 2021-02-14T22:34:26Z | NONE | enable_fts and its related methods only work on tables, not views. Could those methods and possibly others move up to the Queryable superclass? |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/220/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
783910901 | MDU6SXNzdWU3ODM5MTA5MDE= | 221 | .add_missing_columns() does not take case insensitivity into account | simonw 9599 | closed | 0 | 0 | 2021-01-12T05:01:00Z | 2021-01-12T23:17:33Z | 2021-01-12T23:17:33Z | OWNER | SQLite columns are case insensitive - but the |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/221/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
787900412 | MDU6SXNzdWU3ODc5MDA0MTI= | 222 | .m2m() should accept alter=True parameter | simonw 9599 | closed | 0 | 0 | 2021-01-18T04:15:43Z | 2021-01-18T04:26:10Z | 2021-01-18T04:26:10Z | OWNER | sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/222/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||||
788527932 | MDU6SXNzdWU3ODg1Mjc5MzI= | 223 | --delimiter option for CSV import | simonw 9599 | closed | 0 | 2 | 2021-01-18T20:25:03Z | 2021-02-06T01:39:47Z | 2021-02-06T01:34:54Z | OWNER |
Would be useful to be able to do this:
|
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/223/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
802583450 | MDU6SXNzdWU4MDI1ODM0NTA= | 226 | 3.4 release is broken - includes a rogue line | simonw 9599 | closed | 0 | 0 | 2021-02-06T02:08:01Z | 2021-02-06T02:10:26Z | 2021-02-06T02:10:26Z | OWNER | I started seeing weird errors, caused by this line: https://github.com/simonw/sqlite-utils/blob/f8010ca78fed8c5fca6cde19658ec09fdd468420/sqlite_utils/cli.py#L1-L3 That was added by accident in 1b666f9315d4ea6bb332b2e75e48480c26100199 I'm surprised the tests didn't catch this! |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/226/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
807174161 | MDU6SXNzdWU4MDcxNzQxNjE= | 227 | Error reading csv files with large column data | camallen 295329 | closed | 0 | 4 | 2021-02-12T11:51:47Z | 2021-02-16T11:48:03Z | 2021-02-14T21:17:19Z | NONE | Feel free to close this issue - I mostly added it for reference for future folks that run into this :) I have a CSV file with one column that has very long strings. When i try to import this file via the Traceback (most recent call last):
File "/usr/local/bin/sqlite-utils", line 10, in <module>
sys.exit(cli())
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 829, in call
return self.main(args, kwargs)
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, ctx.params)
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 610, in invoke
return callback(args, kwargs)
File "/usr/local/lib/python3.7/site-packages/sqlite_utils/cli.py", line 774, in insert
default=default,
File "/usr/local/lib/python3.7/site-packages/sqlite_utils/cli.py", line 705, in insert_upsert_implementation
docs, pk=pk, batch_size=batch_size, alter=alter, extra_kwargs
File "/usr/local/lib/python3.7/site-packages/sqlite_utils/db.py", line 1852, in insert_all
first_record = next(records)
File "/usr/local/lib/python3.7/site-packages/sqlite_utils/cli.py", line 703, in <genexpr>
docs = (decode_base64_values(doc) for doc in docs)
File "/usr/local/lib/python3.7/site-packages/sqlite_utils/cli.py", line 681, in <genexpr>
docs = (dict(zip(headers, row)) for row in reader)
_csv.Error: field larger than field limit (131072)
sqlite-utils --versionsqlite-utils, version 3.4.1 datasette --versiondatasette, version 0.54 ``` It appears this is a known issue reading in csv files in python and doesn't look to be modifiable through system / env vars (i may be very wrong on this). Noting that using sqlite3 Finally, I'm loving https://datasette.io/ thank you very much for an amazing tool and data ecosytem 🙇♀️ |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/227/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
807437089 | MDU6SXNzdWU4MDc0MzcwODk= | 228 | --no-headers option for CSV and TSV | simonw 9599 | closed | 0 | 10 | 2021-02-12T17:56:51Z | 2021-12-26T07:01:31Z | 2021-02-14T22:25:17Z | OWNER | https://bl.iro.bl.uk/work/ns/3037474a-761c-456d-a00c-9ef3c6773f4c has a fascinating CSV file that doesn't have a header row - it starts like this:
It would be useful if |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/228/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
807817197 | MDU6SXNzdWU4MDc4MTcxOTc= | 229 | Hitting `_csv.Error: field larger than field limit (131072)` | frosencrantz 631242 | closed | 0 | 3 | 2021-02-13T19:52:44Z | 2021-02-14T21:33:33Z | 2021-02-14T21:33:33Z | NONE | I have a csv file where one of the fields is so large it is throwing an exception with this error and stops loading:
The stack trace occurs here: https://github.com/simonw/sqlite-utils/blob/3.1/sqlite_utils/cli.py#L633 There is a way to handle this that helps: https://stackoverflow.com/questions/15063936/csv-error-field-larger-than-field-limit-131072 One issue I had with this problem was sqlite-utils only provides limited context as to where the problem line is. There is the progress bar, but that is by percent rather than by line number. It would have been helpful if it could have provided a line number. Also, it would have been useful if it had allowed the loading to continue with later lines. |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/229/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
808008305 | MDU6SXNzdWU4MDgwMDgzMDU= | 230 | --sniff option for sniffing delimiters | simonw 9599 | closed | 0 | 8 | 2021-02-14T17:43:54Z | 2021-02-14T21:15:33Z | 2021-02-14T19:24:32Z | OWNER |
Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/228#issuecomment-778812050 |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/230/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
808028757 | MDU6SXNzdWU4MDgwMjg3NTc= | 231 | limit=X, offset=Y parameters for more Python methods | simonw 9599 | closed | 0 | 2 | 2021-02-14T19:31:23Z | 2021-02-14T20:03:08Z | 2021-02-14T20:03:08Z | OWNER |
Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/224#issuecomment-778828495 |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/231/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
808036774 | MDU6SXNzdWU4MDgwMzY3NzQ= | 232 | Run tests against Windows in GitHub Actions | simonw 9599 | closed | 0 | 0 | 2021-02-14T20:09:45Z | 2021-02-14T20:39:55Z | 2021-02-14T20:39:55Z | OWNER |
Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/225#issuecomment-778834504 |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/232/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
808046597 | MDU6SXNzdWU4MDgwNDY1OTc= | 234 | .insert_all() fails if subsequent chunks contain additional columns | simonw 9599 | closed | 0 | 1 | 2021-02-14T21:01:51Z | 2021-02-14T21:03:40Z | 2021-02-14T21:03:40Z | OWNER | Reported by @nieuwenhoven in #225 along with a proposed fix. |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/234/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
810618495 | MDU6SXNzdWU4MTA2MTg0OTU= | 235 | Extract columns cannot create foreign key relation: sqlite3.OperationalError: table sqlite_master may not be modified | kristomi 6913891 | closed | 0 | 18 | 2021-02-17T23:33:23Z | 2023-06-26T01:47:01Z | 2023-06-25T23:25:53Z | NONE | Thanks for what seems like a truly great suite of libraries. I wanted to try out Datasette, but never got more than half way through your YouTube video with the SF tree dataset. Whenever I try to extract a column, I get a I have tried googling the problem, but all I've found is that this might be a problem with the sqlite3 database running in defensive mode, but I definitely can't know for sure. Does the problem seem familiar to you? |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/235/reactions", "total_count": 3, "+1": 3, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
811680502 | MDU6SXNzdWU4MTE2ODA1MDI= | 236 | --attach command line option for attaching extra databases | simonw 9599 | closed | 0 | 1 | 2021-02-19T04:38:30Z | 2021-02-19T05:10:41Z | 2021-02-19T05:08:43Z | OWNER | This will enable cross-database joins, as seen in https://github.com/simonw/datasette/issues/283 Also refs #113 |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/236/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
815554385 | MDU6SXNzdWU4MTU1NTQzODU= | 237 | db["my_table"].drop(ignore=True) parameter, plus sqlite-utils drop-table --ignore and drop-view --ignore | mhalle 649467 | closed | 0 | 3 | 2021-02-24T14:55:06Z | 2021-02-25T17:11:41Z | 2021-02-25T17:11:41Z | NONE | When I'm generating a derived table in python, I often drop the table and create it from scratch. However, the first time I generate the table, it doesn't exist, so the drop raises an exception. That means more boilerplate. I was going to submit a pull request that adds an "if_exists" option to the However, for a utility like sqlite_utils, perhaps the "IF EXISTS" SQL semantics is what you want most of the time, and thus should be the default. What do you think? |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/237/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
816523763 | MDU6SXNzdWU4MTY1MjM3NjM= | 238 | .add_foreign_key() corrupts database if column contains a space | simonw 9599 | closed | 0 | 1 | 2021-02-25T15:07:20Z | 2021-02-25T16:54:02Z | 2021-02-25T16:54:02Z | OWNER | I ran this:
And got this: ``` ~/jupyter-venv/lib/python3.9/site-packages/sqlite_utils/db.py in add_foreign_keys(self, foreign_keys) 616 # Have to VACUUM outside the transaction to ensure .foreign_keys property 617 # can see the newly created foreign key. --> 618 self.vacuum() 619 620 def index_foreign_keys(self): ~/jupyter-venv/lib/python3.9/site-packages/sqlite_utils/db.py in vacuum(self) 629 630 def vacuum(self): --> 631 self.execute("VACUUM;") 632 633 ~/jupyter-venv/lib/python3.9/site-packages/sqlite_utils/db.py in execute(self, sql, parameters) 234 return self.conn.execute(sql, parameters) 235 else: --> 236 return self.conn.execute(sql) 237 238 def executescript(self, sql): DatabaseError: database disk image is malformed ``` |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/238/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
816526538 | MDU6SXNzdWU4MTY1MjY1Mzg= | 239 | sqlite-utils extract could handle nested objects | simonw 9599 | open | 0 | 16 | 2021-02-25T15:10:28Z | 2022-09-03T23:46:02Z | OWNER | Imagine a table (imported from a nested JSON file) where one of the columns contains values that look like this:
The |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/239/reactions", "total_count": 6, "+1": 5, "-1": 0, "laugh": 0, "hooray": 1, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
816560819 | MDU6SXNzdWU4MTY1NjA4MTk= | 240 | table.pks_and_rows_where() method returning primary keys along with the rows | simonw 9599 | closed | 0 | 7 | 2021-02-25T15:49:28Z | 2021-02-25T16:39:23Z | 2021-02-25T16:28:23Z | OWNER | Original title: Easier way to update a row returned from .rows Here's a surprisingly hard problem I ran into while trying to implement #239 - given a row returned by The problem is that the Instead, currently, you need to introspect the table and, if A utility mechanism to make this easier would be very welcome. |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/240/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
817989436 | MDU6SXNzdWU4MTc5ODk0MzY= | 242 | Async support | eyeseast 25778 | open | 0 | 13 | 2021-02-27T18:29:38Z | 2021-10-28T14:37:56Z | CONTRIBUTOR | Following our conversation last week, want to note this here before I forget. I've had a couple situations where I'd like to do a bunch of updates in an async event loop, but I run into SQLite's issues with concurrent writes. This feels like something sqlite-utils could help with. PeeWee ORM has a SQLite write queue that might be a good model. It's using threads or gevent, but I think that approach would translate well enough to asyncio. Happy to help with this, too. |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/242/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
818684978 | MDU6SXNzdWU4MTg2ODQ5Nzg= | 243 | How can i use this utils to deal with fts on column meta of tables ? | svjack 27874014 | open | 0 | 0 | 2021-03-01T09:45:05Z | 2021-03-01T09:45:05Z | NONE | Thank you to release this bravo project. When i use this project on multi table db, I want to implement convenient search on column name from different tables. I want to develop a meta table to save the meta data of different columns of different tables and search on this meta table to get rows from the data table (which the meta table describes) does this project provide some simple function on it ? You can think a have a knowledge graph about the table in the db, and i save this knowledge graph into the db with fts enabled. |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/243/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
831751367 | MDU6SXNzdWU4MzE3NTEzNjc= | 246 | Escaping FTS search strings | DeNeutoy 16001974 | closed | 0 | 4 | 2021-03-15T12:15:09Z | 2021-08-18T18:57:13Z | 2021-08-18T18:43:12Z | CONTRIBUTOR | Thanks for the excellent library, it's very nice to use! I've been building some in memory search functionality for a data annotation tool i'm making, and I got tripped up a little bit with escaping the full text search queries. First I tried using http://search-24ways.herokuapp.com/24ways-f8f455f/articles?_search=acces%2A I got around this by aggressively escaping quotes inside the query string like this: ```python quoted = q.replace('"', '""') quoted = f'"{quoted}"' print(quoted) results = db["data"].search(quoted, columns=["id"]) return [x["id"] for x in results] ``` This works in the sense it doesn't crash, but it also removes access to the search query syntax. Given the well specified definition, it might be possible for sqlite-utils to provide a |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/246/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
836829560 | MDU6SXNzdWU4MzY4Mjk1NjA= | 248 | support for Apache Arrow / parquet files I/O | mhalle 649467 | open | 0 | 1 | 2021-03-20T14:59:30Z | 2021-10-28T23:46:48Z | NONE | I just started looking at Apache Arrow using pyarrow for import and export of tabular datasets, and it looks quite compelling. It might be worth looking at for sqlite-utils and/or datasette. As a test, I took a random jsonl data dump of a dataset I have with floats, strings, and ints and converted it to arrow's parquet format using the naive The only hangup is the automatic type inference of the naive reader. It's great for general laziness and for parsing JSON columns (it correctly interpreted a table of mine with a JSON array). However, I did get an exception for a string column where most entries looked integer-like but had a couple values that weren't -- the reader tried to coerce all of them for some reason, even though the JSON type is string. Since the writer optionally takes a schema, it shouldn't be too hard to grab the sqlite header types. With some additional hinting, you might get datetime columns and JSON, which are native Arrow types. Somewhat tangentially, someone even wrote an sqlite vfs extension for Parquet: https://cldellow.com/2018/06/22/sqlite-parquet-vtable.html |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/248/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
836963850 | MDU6SXNzdWU4MzY5NjM4NTA= | 249 | Full text search possibly broken? | prabhur 36287 | closed | 0 | 2 | 2021-03-21T02:03:44Z | 2021-03-21T02:43:32Z | 2021-03-21T02:43:32Z | NONE | I'm not quite sure if this is an issue with sqlite-utils or datasette. Background I was previously using sqlite-utils version < 3.6. I have a bunch of csv files that have some data scraped from a website.
Data is loaded to the table via
Note that the data contains text in my language Tamil. Loading happens just fine. datasette serves the db file just fine. It recognizes FTS and shows the "search" box. However, none of the queries work. Whatever text I supply, it always returns 0 rows. I literally copy paste words from the row listing on the screen and paste it on the search box. Interestingly, only thing I can remember is switching to sqlite-utils 3.6. I had to do this because the prior version had an issue with column size. I have attached one of the csv files that can be loaded to the table. Substitute "${filename}" with that file for the sqlite-utils insert command. posts_20200417-20201231.csv.zip Interestingly, the FTS based search from datasette worked just fine before this version upgrade. That is, the queries returned results. I will try to downgrade just to see if the theory is correct. I appreciate any help here. Thanks. |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/249/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
838148087 | MDU6SXNzdWU4MzgxNDgwODc= | 250 | Handle byte order marks (BOMs) in CSV files | simonw 9599 | closed | 0 | 3 | 2021-03-22T22:13:18Z | 2021-05-29T05:34:21Z | 2021-05-29T05:34:21Z | OWNER | I often find |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/250/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed |
Advanced export
JSON shape: default, array, newline-delimited, object
CREATE TABLE [issues] ( [id] INTEGER PRIMARY KEY, [node_id] TEXT, [number] INTEGER, [title] TEXT, [user] INTEGER REFERENCES [users]([id]), [state] TEXT, [locked] INTEGER, [assignee] INTEGER REFERENCES [users]([id]), [milestone] INTEGER REFERENCES [milestones]([id]), [comments] INTEGER, [created_at] TEXT, [updated_at] TEXT, [closed_at] TEXT, [author_association] TEXT, [pull_request] TEXT, [body] TEXT, [repo] INTEGER REFERENCES [repos]([id]), [type] TEXT , [active_lock_reason] TEXT, [performed_via_github_app] TEXT, [reactions] TEXT, [draft] INTEGER, [state_reason] TEXT); CREATE INDEX [idx_issues_repo] ON [issues] ([repo]); CREATE INDEX [idx_issues_milestone] ON [issues] ([milestone]); CREATE INDEX [idx_issues_assignee] ON [issues] ([assignee]); CREATE INDEX [idx_issues_user] ON [issues] ([user]);