id,node_id,number,state,locked,title,user,body,created_at,updated_at,closed_at,merged_at,merge_commit_sha,assignee,milestone,draft,head,base,author_association,repo,url,merged_by,auto_merge 249680944,MDExOlB1bGxSZXF1ZXN0MjQ5NjgwOTQ0,9,closed,0,:pencil: Updates my_database.py to my_database.db,50527,I noticed that both `.py` and `.db` were used in the docs and assumed you'd prefer `.db`. ,2019-02-01T17:35:43Z,2019-02-24T03:55:04Z,2019-02-24T03:55:04Z,2019-02-24T03:55:04Z,c5068a0972651b3e359ebc2d6c1486b8b7d2c242,,,0,1ad604fbbd3311f041357190796a3613c0c729d1,441c131db5cc68e197db19f0623ff8a96c90c3ff,CONTRIBUTOR,140912432,https://github.com/simonw/sqlite-utils/pull/9,, 313007483,MDExOlB1bGxSZXF1ZXN0MzEzMDA3NDgz,56,closed,0,Escape the table name in populate_fts and search.,49260,"The table names weren't escaped using double quotes in the populate_fts method. Reproducible case: ``` >>> import sqlite_utils >>> db = sqlite_utils.Database(""abc.db"") >>> db[""http://example.com""].insert_all([ ... {""id"": 1, ""age"": 4, ""name"": ""Cleo""}, ... {""id"": 2, ""age"": 2, ""name"": ""Pancakes""} ... ], pk=""id"") >>> db[""http://example.com""].enable_fts([""name""]) Traceback (most recent call last): File """", line 1, in db[""http://example.com""].enable_fts([""name""]) File ""/home/amjith/.virtualenvs/itsysearch/lib/python3.7/site-packages/sqlite_utils/db.py"", l ine 705, in enable_fts self.populate_fts(columns) File ""/home/amjith/.virtualenvs/itsysearch/lib/python3.7/site-packages/sqlite_utils/db.py"", l ine 715, in populate_fts self.db.conn.executescript(sql) sqlite3.OperationalError: unrecognized token: "":"" >>> ```",2019-09-01T06:29:05Z,2019-09-02T17:23:21Z,2019-09-02T17:23:21Z,,79852e97ecb69b88da87da0cba2a55887cf83bda,,,0,83ca4c802f5d5102e73ff366e61514ded81dc7a1,cb70f7d10996b844154bf3da88779dd1f65590bc,CONTRIBUTOR,140912432,https://github.com/simonw/sqlite-utils/pull/56,, 313105634,MDExOlB1bGxSZXF1ZXN0MzEzMTA1NjM0,57,closed,0,Add triggers while enabling FTS,49260,"This adds the option for a user to set up triggers in the database to keep their FTS table in sync with the parent table. Ref: https://sqlite.org/fts5.html#external_content_and_contentless_tables I would prefer to make the creation of triggers the default behavior, but that will break existing usage where people have been calling `populate_fts` after inserting new rows. I am happy to make changes to the PR as you see fit. ",2019-09-02T04:23:40Z,2019-09-03T01:03:59Z,2019-09-02T23:42:29Z,2019-09-02T23:42:29Z,405e092d5916e70df10f82d15e9c052aa9ee8d80,,,0,e01943271b17115fbe0e81d523126d2fb1c7c24b,cb70f7d10996b844154bf3da88779dd1f65590bc,CONTRIBUTOR,140912432,https://github.com/simonw/sqlite-utils/pull/57,, 359822773,MDExOlB1bGxSZXF1ZXN0MzU5ODIyNzcz,75,closed,0,Explicitly include tests and docs in sdist,15092,Also exclude 'tests' from runtime installation.,2020-01-07T04:53:20Z,2020-01-31T00:21:27Z,2020-01-31T00:21:27Z,2020-01-31T00:21:27Z,0988f2eccc2dfa26b1a55243582222f540a72838,,,0,39a836450b23bf2fe1ba9a071a5a9a00a0949cd6,59a2e8ebdcbde7e6fb091b0556713ca5a20ea4e7,CONTRIBUTOR,140912432,https://github.com/simonw/sqlite-utils/pull/75,, 395258687,MDExOlB1bGxSZXF1ZXN0Mzk1MjU4Njg3,96,closed,0,Add type conversion for Panda's Timestamp,32605365,"Add type conversion for Panda's Timestamp, if Panda library is present in system (thanks for this project, I was about to do the same thing from scratch)",2020-03-29T14:13:09Z,2020-03-31T04:40:49Z,2020-03-31T04:40:48Z,2020-03-31T04:40:48Z,8ea626e5fcdc4c9e52f615c6347e68173805f8b4,,,0,16ebbd2d494caabd0eeb502f8a944614b464bb12,22250a9c735077d6f365b73bf824e6c67b122c83,CONTRIBUTOR,140912432,https://github.com/simonw/sqlite-utils/pull/96,, 445023326,MDExOlB1bGxSZXF1ZXN0NDQ1MDIzMzI2,118,closed,0,Add insert --truncate option,79913," Deletes all rows in the table (if it exists) before inserting new rows. SQLite doesn't implement a TRUNCATE TABLE statement but does optimize an unqualified DELETE FROM. This can be handy if you want to refresh the entire contents of a table but a) don't have a PK (so can't use --replace), b) don't want the table to disappear (even briefly) for other connections, and c) have to handle records that used to exist being deleted. Ideally the replacement of rows would appear instantaneous to other connections by putting the DELETE + INSERT in a transaction, but this is very difficult without breaking other code as the current transaction handling is inconsistent and non-systematic. There exists the possibility for the DELETE to succeed but the INSERT to fail, leaving an empty table. This is not much worse, however, than the current possibility of one chunked INSERT succeeding and being committed while the next chunked INSERT fails, leaving a partially complete operation.",2020-07-06T21:58:40Z,2020-07-08T17:26:21Z,2020-07-08T17:26:21Z,2020-07-08T17:26:21Z,ae4593316ccf5e42ad26f27033193834a7e696c8,,,0,332f7d770b84734dbed4842ab3ed24ee5b687889,f8277d0fb9c05a88a9ff01d996e31d55f0f0a645,CONTRIBUTOR,140912432,https://github.com/simonw/sqlite-utils/pull/118,, 445833908,MDExOlB1bGxSZXF1ZXN0NDQ1ODMzOTA4,120,closed,0,Fix query command's support for DML,79913,See commit messages for details. I ran into this while investigating another feature/issue.,2020-07-08T01:36:34Z,2020-07-08T05:14:04Z,2020-07-08T05:14:04Z,2020-07-08T05:14:04Z,f8277d0fb9c05a88a9ff01d996e31d55f0f0a645,,,0,6a660d12a27864d6ab552e11eef9fd13bc281198,d0cdaaaf00249230e847be3a3b393ee2689fbfe4,CONTRIBUTOR,140912432,https://github.com/simonw/sqlite-utils/pull/120,, 475665984,MDExOlB1bGxSZXF1ZXN0NDc1NjY1OTg0,142,closed,0,"insert_all(..., alter=True) should work for new columns introduced after the first 100 records",96218,Closes #139.,2020-08-28T22:22:57Z,2020-08-30T07:28:23Z,2020-08-28T22:30:14Z,2020-08-28T22:30:14Z,947bb7626fd1763608a470adf9cf5f156ef003e9,,,0,1b992c7c1e5855caa87c7939efad287052f446f3,ea87c2b943fdd162c42a900ac0aea5ecc2f4b9d9,CONTRIBUTOR,140912432,https://github.com/simonw/sqlite-utils/pull/142,, 475874493,MDExOlB1bGxSZXF1ZXN0NDc1ODc0NDkz,146,closed,0,Handle case where subsequent records (after first batch) include extra columns,96218,"Addresses #145. I think this should do the job. If it meets with your approval I'll update this PR to include an update to the documentation -- I came across this bug while preparing a PR to update the documentation around `batch_size` in any event.",2020-08-30T07:13:58Z,2020-09-08T23:20:37Z,2020-09-08T23:20:37Z,2020-09-08T23:20:36Z,e6d202b742a7b531fffa593703d34f8337632d68,,,0,40fcea41525682fdc2f601da787b29d5a8517dcd,deb2eb013ff85bbc828ebc244a9654f0d9c3139e,CONTRIBUTOR,140912432,https://github.com/simonw/sqlite-utils/pull/146,, 483027587,MDExOlB1bGxSZXF1ZXN0NDgzMDI3NTg3,156,closed,0,Typos in tests,96218,"One of these is my fault, and the other is one I just happened to come across. They're harmless, but might as well be fixed.",2020-09-09T18:00:58Z,2020-09-09T18:24:50Z,2020-09-09T18:21:23Z,2020-09-09T18:21:23Z,367082e787101fb90901ef3214804ab23a92ce46,,,0,b777776753ef08f35dfaa7d86b2d5082e8904f27,6be61263642d8e46ec54cf5f51af74e0df2f2393,CONTRIBUTOR,140912432,https://github.com/simonw/sqlite-utils/pull/156,, 483175509,MDExOlB1bGxSZXF1ZXN0NDgzMTc1NTA5,158,closed,0,Fix accidental mega long line in docs,167319,,2020-09-09T22:31:23Z,2020-09-16T06:21:43Z,2020-09-16T06:21:43Z,2020-09-16T06:21:43Z,7805d53bcf11199bd1f2b07e05ae90151f9d0eb0,,,0,5c1d8a5f3ac8684657eefe4f57ff5ff605f6b633,367082e787101fb90901ef3214804ab23a92ce46,CONTRIBUTOR,140912432,https://github.com/simonw/sqlite-utils/pull/158,, 493162767,MDExOlB1bGxSZXF1ZXN0NDkzMTYyNzY3,178,closed,0,Update README.md,19921,"The `sqlite-utils insert releases.db releases - --pk` is missing the pk field name, added ` ""id""` to fix it.",2020-09-25T15:52:11Z,2020-10-01T14:18:30Z,2020-09-30T20:29:28Z,2020-09-30T20:29:28Z,cada1017edcfa691c2314d7ad1b7c7576495317f,,,0,0c6a81dae802a55d5469c18cd32402d11b6e8a59,94fc62857ee2655a21d85f6dae84b67bbfa5956d,CONTRIBUTOR,140912432,https://github.com/simonw/sqlite-utils/pull/178,, 510235909,MDExOlB1bGxSZXF1ZXN0NTEwMjM1OTA5,189,closed,0,Allow iterables other than Lists in m2m records,35681,"I was playing around with sqlite-utils, creating a Roam Research dogsheep-style importer for Datasette, and ran into a slight snag. I wanted to use a generator to add an order column in an importer. It looked something like: ``` def order_generator(iterable, attr=None): if attr is None: attr = ""order"" order: int = 0 for i in iterable: i[attr] = order order += 1 yield i ``` When I used this with `insert_all` and other things, it worked fine--but it didn't work as the `records` argument to `m2m`. I dug into it, and sqlite-utils is explicitly checking if the records argument is a list or a tuple. I flipped the check upside down, and now it checks if the argument is a mapping. If it's a mapping, it wraps it in a list, otherwise it leaves it alone. (I get that it might not really make sense to put the order column on the second table. I changed my import schema a bit, and no longer have a real example, but maybe this change still makes sense.) The automated tests still pass, but I did not add any new ones. Let me know what you think! I'm really loving Datasette and its ecosystem; thanks for everything!",2020-10-26T18:47:44Z,2020-10-27T16:28:37Z,2020-10-27T16:24:21Z,2020-10-27T16:24:21Z,f045d8559a6d2cb922a2de30fbcc896a4486b82f,,,0,93230b2acb61635b6d5070ad9c65e7221c63b75a,e4f1c7b936981de29823730c5dbef4f4ba7a4286,CONTRIBUTOR,140912432,https://github.com/simonw/sqlite-utils/pull/189,, 529090560,MDExOlB1bGxSZXF1ZXN0NTI5MDkwNTYw,204,closed,0,use jsonify_if_need for sql updates,78035,,2020-11-29T10:49:00Z,2020-12-08T17:49:42Z,2020-12-08T17:49:42Z,2020-12-08T17:49:42Z,c5f4f0f70ce394dfec6054c3c5aaedf330887093,,,0,8ea9ad1913aa63395391818ebb4e43bfce936dda,60d3c4821be4cf25c41097c1e8b79b2e60c5ead5,CONTRIBUTOR,140912432,https://github.com/simonw/sqlite-utils/pull/204,, 583407989,MDExOlB1bGxSZXF1ZXN0NTgzNDA3OTg5,244,closed,0,Typo in upsert example,387669,Remove extra `[`,2021-03-02T23:14:14Z,2021-05-19T02:58:21Z,2021-05-19T02:58:21Z,2021-05-19T02:58:21Z,328211eaca1247cd6b33a2c0a54642f87866d85b,,,0,1899200fa5a43f638573ea2c483df62819ba53b1,09c3386f55f766b135b6a1c00295646c4ae29bec,CONTRIBUTOR,140912432,https://github.com/simonw/sqlite-utils/pull/244,, 592289324,MDExOlB1bGxSZXF1ZXN0NTkyMjg5MzI0,245,closed,0,Correct some typos,1076745,Noticed a typo in the docs and followed that up with a spellcheck. Had to bite my tongue at some of the British spellings.,2021-03-13T04:26:56Z,2021-05-19T02:58:04Z,2021-05-19T02:58:04Z,2021-05-19T02:58:04Z,3e62ab62a88992d4bbb0fe83debec3bacd93ebf3,,,0,79de6b639f9d14818a5006af4f63e4e20c961a10,22f1d9e1999f70af4c5b0f880a820cd9eead3942,CONTRIBUTOR,140912432,https://github.com/simonw/sqlite-utils/pull/245,, 593805804,MDExOlB1bGxSZXF1ZXN0NTkzODA1ODA0,247,closed,0,FTS quote functionality from datasette,16001974,"Addresses #246 - this is a bit of a kludge because it doesn't actually *validate* the FTS string, just makes sure that it will not crash when executed, but I figured that building a query parser is a bit out of the scope of sqlite-utils and if you actually want to use the query language, you probably need to parse that yourself. ",2021-03-16T11:17:34Z,2021-08-18T18:43:12Z,2021-08-18T18:43:12Z,2021-08-18T18:43:11Z,53fec0d8639d2a66e322d05e1fcc8f34caa57815,,,0,af989af658e23be0077f226292594226103b063d,1fe73c898b44695052f1a9ca832818d50cecf662,CONTRIBUTOR,140912432,https://github.com/simonw/sqlite-utils/pull/247,, 614727036,MDExOlB1bGxSZXF1ZXN0NjE0NzI3MDM2,254,closed,0,Fix incorrect create-table cli description,1935268,The description for `create-table` was duplicated from `create-index`.,2021-04-13T20:03:15Z,2021-05-19T04:43:46Z,2021-05-19T02:57:26Z,2021-05-19T02:57:26Z,e7b2626291040b78b9a2dbc2982ba72691fb1a0f,,,0,cdeb7d7da77c6d1cd3554d08cb5659243cb3d55b,22f1d9e1999f70af4c5b0f880a820cd9eead3942,CONTRIBUTOR,140912432,https://github.com/simonw/sqlite-utils/pull/254,, 623755723,MDExOlB1bGxSZXF1ZXN0NjIzNzU1NzIz,258,closed,0,Fixing insert from JSON containing strings with non-ascii characters …,6586811,"…are escaped aps unicode for lists, tuples, dicts Fix of #257 ",2021-04-26T20:50:00Z,2021-05-19T02:47:44Z,2021-05-19T02:47:44Z,2021-05-19T02:47:44Z,6f4f9a3effeb16de0348d3cf136664f7531f498d,,,0,27e7c1ae1913e2127361af4fc664c2061fb76627,22f1d9e1999f70af4c5b0f880a820cd9eead3942,CONTRIBUTOR,140912432,https://github.com/simonw/sqlite-utils/pull/258,, 672359609,MDExOlB1bGxSZXF1ZXN0NjcyMzU5NjA5,277,closed,0,add -h support closes #276,601708,This appears to be the [canonical solution](https://click.palletsprojects.com/en/7.x/documentation/#help-parameter-customization).,2021-06-17T08:08:26Z,2021-06-18T14:56:59Z,2021-06-18T14:56:59Z,2021-06-18T14:56:59Z,aa652b6afe43d2b40fabc7a513c3e68866e030a5,,,0,abbd324a2178f09b19966ec36ab2f584d1824c59,a19ce1a4d0048d389411cfe11a5dbe4c503720e1,CONTRIBUTOR,140912432,https://github.com/simonw/sqlite-utils/pull/277,, 719998225,MDExOlB1bGxSZXF1ZXN0NzE5OTk4MjI1,322,closed,0,Add dict type to be mapped as TEXT in sqllite,2496189,"the library deal with Postgres type jsonb as dictionary, add dict type as a TEXT for mapping to sqlite ",2021-08-25T20:54:26Z,2021-11-15T00:27:40Z,2021-11-15T00:27:40Z,2021-11-15T00:27:40Z,271b894af52eb6437ae6cd84eba9867ad8dd43f6,,,0,69619f68c26478fdee479110e084fd22711013a3,77c240df56068341561e95e4a412cbfa24dc5bc7,CONTRIBUTOR,140912432,https://github.com/simonw/sqlite-utils/pull/322,, 727012996,MDExOlB1bGxSZXF1ZXN0NzI3MDEyOTk2,324,closed,0,Use python-dateutil package instead of dateutils,191622,"While working on updating `sqlite-utils` for NixOS/Nixpkgs, I came a cross the following: In 5ec6686153e29ae10d4921a1ad4c841f192f20e2, a new dependency was added on `dateutils` (https://pypi.org/project/dateutils/). I believe this is unintentional, and instead `python-dateutil` (https://pypi.org/project/python-dateutil/) was intended. My reasoning is: - `python-dateutil` is imported here in [recipes.py](https://github.com/simonw/sqlite-utils/blob/5ec6686153e29ae10d4921a1ad4c841f192f20e2/sqlite_utils/recipes.py#L1) - The `mypy` `type-python-dateutil` dependency in [setup.py](https://github.com/simonw/sqlite-utils/blob/5ec6686153e29ae10d4921a1ad4c841f192f20e2/setup.py#L36) - `python-dateutil` is a dependency of `dateutils` as seen in the output in [docs/tutorial.ipynb](https://github.com/simonw/sqlite-utils/blob/77c240df56068341561e95e4a412cbfa24dc5bc7/docs/tutorial.ipynb#L43) Seems like the trailing ""s"" seems to be the source of confusion 😅 I've swapped the dependencies out, hope this helps.",2021-09-03T18:31:19Z,2021-11-14T23:25:40Z,2021-11-14T23:25:40Z,2021-11-14T23:25:40Z,bc4c42d68879c710c851dba3c98deda96ca6caa8,,,0,275ac3b282947fa5df5d18de835d5d958a778e72,77c240df56068341561e95e4a412cbfa24dc5bc7,CONTRIBUTOR,140912432,https://github.com/simonw/sqlite-utils/pull/324,, 821992886,PR_kwDOCGYnMM4w_p22,385,closed,0,Add new spatialite helper methods,25778,"Refs #79 This PR adds three new Spatialite-related methods to Database and Table: - `Database.init_spatialite` loads the Spatialite extension and initializes it - `Table.add_geometry_column` adds a geometry column - `Table.create_spatial_index` creates a spatial index Has tests and documentation. Feedback very welcome.",2022-01-14T03:57:30Z,2022-02-05T00:04:26Z,2022-02-04T05:55:10Z,2022-02-04T05:55:10Z,ee11274fcb1c00f32c95f2ef2924d5349538eb4d,,,0,af86b17acf2fa50048e38b96497636d49db89766,74586d3cb26fa3cc3412721985ecdc1864c2a31d,CONTRIBUTOR,140912432,https://github.com/simonw/sqlite-utils/pull/385,, 853484980,PR_kwDOCGYnMM4y3yW0,407,closed,0,Add SpatiaLite helpers to CLI,25778,"Closes #398 This adds SpatiaLite helpers to the CLI. ```sh # init spatialite when creating a database sqlite-utils create database.db --enable-wal --init-spatialite # add geometry columns # needs a database, table, geometry column name, type, with optional SRID and not-null # this will throw an error if the table doesn't already exist sqlite-utils add-geometry-column database.db table-name geometry --srid 4326 --not-null # spatial index an existing table/column # this will throw an error it the table and column don't exist sqlite-utils create-spatial-index database.db table-name geometry ``` Docs and tests are included. ",2022-02-15T16:50:17Z,2022-02-16T01:49:40Z,2022-02-16T00:58:08Z,2022-02-16T00:58:07Z,a692c56659c3563b26dcdc9e3534d63ecc26e180,,,0,a974da591915e0548182bbbf01da34ecb9e537e6,e7f040106b5f5a892ebd984f19b21c605e87c142,CONTRIBUTOR,140912432,https://github.com/simonw/sqlite-utils/pull/407,, 862823026,PR_kwDOCGYnMM4zbaJy,410,closed,0,Correct spelling mistakes (found with codespell),3818,,2022-02-24T20:44:18Z,2022-03-06T08:48:29Z,2022-03-01T21:05:29Z,2022-03-01T21:05:29Z,b6c9dfce0ba27eb5fb6bc2221044798420f861c4,,,0,a174069da6669b8914d4a08dded7fd750d52a316,7a098aa0c5e8beef6ccc55c866cf7792af5fcf43,CONTRIBUTOR,140912432,https://github.com/simonw/sqlite-utils/pull/410,, 887534558,PR_kwDOCGYnMM405rPe,419,closed,0,Ignore common generated files,25778,"Closes #418 This adds four files to `.gitignore`: .hypothesis/ Pipfile Pipfile.lock pyproject.toml Those are all generated in the course of development and testing.",2022-03-23T18:06:22Z,2022-03-24T21:01:44Z,2022-03-24T21:01:44Z,2022-03-24T21:01:44Z,396f80fcc60da8dd844577114f7920830a2e5403,,,0,228f73615cf993dd3caf3c240682c00392832bb8,93fa79d30b1531bea281d0eb6b925c4e61bc1aa6,CONTRIBUTOR,140912432,https://github.com/simonw/sqlite-utils/pull/419,, 943686022,PR_kwDOCGYnMM44P4GG,437,closed,0,docs to dogs,114388,Fixes a typo.,2022-05-22T15:50:33Z,2022-05-30T21:32:41Z,2022-05-30T21:32:41Z,2022-05-30T21:32:41Z,9fedfc69d7239ac49900051e1c48ee9cdd470d9e,,,0,9e540417821385939db0fdd2d0a859f85be377e6,59be60c471fd7a2c4be7f75e8911163e618ff5ca,CONTRIBUTOR,140912432,https://github.com/simonw/sqlite-utils/pull/437,, 992299943,PR_kwDOCGYnMM47JUun,452,closed,0,Add duplicate table feature,1690072,"This PR addresses a feature request raised in issue #449. Specifically this PR adds a functionality that lets users duplicate a table via: ```python table_new = db[""my_table""].duplicate(""new_table"") ``` Test added in file `tests/test_duplicate.py`. Happy to make changes to meet maintainers' feedback, if any. ",2022-07-09T20:24:31Z,2022-07-15T21:21:37Z,2022-07-15T21:21:36Z,2022-07-15T21:21:36Z,b366e68deb0780048a23610c279552f8529d4726,,,0,eef350fe543c6301c61b257c5f708e0e16ed5a34,42440d6345c242ee39778045e29143fb550bd2c2,CONTRIBUTOR,140912432,https://github.com/simonw/sqlite-utils/pull/452,, 1000800283,PR_kwDOCGYnMM47pwAb,455,closed,0,"in extract code, check equality with IS instead of = for nulls",536941,"sqlite ""IS"" is equivalent to SQL ""IS NOT DISTINCT FROM"" closes #423",2022-07-19T13:40:25Z,2022-08-27T14:45:03Z,2022-08-27T14:45:03Z,2022-08-27T14:45:03Z,c5f8a2eb1a81a18b52825cc649112f71fe419b12,,,0,1b35a92e3ede76f0f29f6f8dcd899f44b2abbb02,855bce8c3823718def13e0b8928c58bf857e41b2,CONTRIBUTOR,140912432,https://github.com/simonw/sqlite-utils/pull/455,, 1022525059,PR_kwDOCGYnMM488n6D,463,closed,0,Use Read the Docs action v1,244656,"Read the Docs repository was renamed from `readthedocs/readthedocs-preview` to `readthedocs/actions/`. Now, the `preview` action is under `readthedocs/actions/preview` and is tagged as `v1` ---- :books: Documentation preview :books:: https://sqlite-utils--463.org.readthedocs.build/en/463/ ",2022-08-10T10:31:47Z,2022-08-18T08:30:14Z,2022-08-17T23:11:16Z,2022-08-17T23:11:16Z,83e7339255e811c62e6db8498c483c44a84d0f28,,,0,364110b97bd1c6e0d2bc841730e5dc1a595e537d,271433fdd18e436b0a527ab899cb6f6fa67f23d0,CONTRIBUTOR,140912432,https://github.com/simonw/sqlite-utils/pull/463,, 1040691996,PR_kwDOCGYnMM4-B7Mc,480,closed,0,search_sql add include_rank option,7908073,"I haven't tested this yet but wanted to get a heads-up whether this kind of change would be useful or if I should just duplicate the function and tweak it within my code ---- :books: Documentation preview :books:: https://sqlite-utils--480.org.readthedocs.build/en/480/ ",2022-08-30T09:10:29Z,2022-08-31T03:40:35Z,2022-08-31T03:40:35Z,2022-08-31T03:40:35Z,ecf1d40112e52a8f4e509c39b98caae996b7bc36,,,0,3dbfca6fdd878ba381f83157fdae2042eeaed371,087753cd42c406f1e060c1822dcd9b5fda3d60f4,CONTRIBUTOR,140912432,https://github.com/simonw/sqlite-utils/pull/480,, 1050417981,PR_kwDOCGYnMM4-nBs9,486,closed,0,"progressbar for inserts/upserts of all fileformats, closes #485",99098079," ---- :books: Documentation preview :books:: https://sqlite-utils--486.org.readthedocs.build/en/486/ ",2022-09-08T14:58:02Z,2022-09-15T20:40:03Z,2022-09-15T20:37:51Z,2022-09-15T20:37:51Z,0b315d3fa83c1584eaeec32f24912898621e437a,,,0,d5db749480aaf8518e611ff55da186cadf6c63bc,d9b9e075f07a20f1137cd2e34ed5d3f1a3db4ad8,CONTRIBUTOR,140912432,https://github.com/simonw/sqlite-utils/pull/486,, 1082769953,PR_kwDOCGYnMM5AicIh,498,closed,0,fix: enable-fts permanently save triggers,7908073,"I was wondering why my all my databases were giving wild search results. Turns out create_trigger was not sticking! Running `sqlite-utils triggers x.db` shows `[]` after running `enable-fts` using the python api. Looking at the counts trigger it seems that is the right way to save triggers. triggers show up now ---- :books: Documentation preview :books:: https://sqlite-utils--498.org.readthedocs.build/en/498/ ",2022-10-11T05:10:51Z,2022-10-15T04:33:08Z,2022-10-11T06:34:31Z,,ab24d21e9c2aa9c514c3b65f05cfc0bafa7621c1,,,0,75577e595d9177177a48030a1c39573af010df51,d792dad1cf5f16525da81b1e162fb71d469995f3,CONTRIBUTOR,140912432,https://github.com/simonw/sqlite-utils/pull/498,, 1083804914,PR_kwDOCGYnMM5AmYzy,499,open,0,feat: recreate fts triggers after table transform,7908073,"https://github.com/simonw/sqlite-utils/pull/498 ---- :books: Documentation preview :books:: https://sqlite-utils--499.org.readthedocs.build/en/499/ alternatively, `self.disable_fts()`",2022-10-11T20:35:39Z,2022-10-26T17:54:51Z,,,6c71562a7a21ab93541a14453961ce3d6abca3cc,,,0,a25e83841f2703afb08aff4589cb559ae9060776,d792dad1cf5f16525da81b1e162fb71d469995f3,CONTRIBUTOR,140912432,https://github.com/simonw/sqlite-utils/pull/499,, 1105985162,PR_kwDOCGYnMM5B6_6K,508,closed,0,Allow surrogates in parameters,7908073,"closes #507 https://dwheeler.com/essays/fixing-unix-linux-filenames.html ---- :books: Documentation preview :books:: https://sqlite-utils--508.org.readthedocs.build/en/508/ ",2022-10-31T22:11:49Z,2022-11-17T15:11:16Z,2022-10-31T22:55:36Z,,3b551597240d9a6058b1c3c720073120db213678,,,0,43a8c4c91fc22fb6bea07846f144072b0d047f4e,529110e7d8c4a6b1bbf5fb61f2e29d72aa95a611,CONTRIBUTOR,140912432,https://github.com/simonw/sqlite-utils/pull/508,, 1173049178,PR_kwDOCGYnMM5F609a,519,closed,0,Fixes breaking DEFAULT values,13819005,"Fixes #509, Fixes #336 Thanks for the great library! I fixed a bug that `sqlite-utils transform` breaks DEFAULT values. All tests already present passed with no changes, and I added some tests for this PR. In #509 case, fixed here. ```shell $ sqlite3 test.db << EOF CREATE TABLE mytable ( col1 TEXT DEFAULT 'foo', col2 TEXT DEFAULT (STRFTIME('%Y-%m-%d %H:%M:%f', 'NOW')) ) EOF $ sqlite3 test.db ""SELECT sql FROM sqlite_master WHERE name = 'mytable';"" CREATE TABLE mytable ( col1 TEXT DEFAULT 'foo', col2 TEXT DEFAULT (STRFTIME('%Y-%m-%d %H:%M:%f', 'NOW')) ) $ sqlite3 test.db ""INSERT INTO mytable DEFAULT VALUES; SELECT * FROM mytable;"" foo|2022-12-21 01:15:39.669 $ sqlite-utils transform test.db mytable --rename col1 renamedcol1 $ sqlite3 test.db ""SELECT sql FROM sqlite_master WHERE name = 'mytable';"" CREATE TABLE ""mytable"" ( [renamedcol1] TEXT DEFAULT 'foo', [col2] TEXT DEFAULT (STRFTIME('%Y-%m-%d %H:%M:%f', 'NOW')) # ← Non-String Value ) $ sqlite3 test.db ""INSERT INTO mytable DEFAULT VALUES; SELECT * FROM mytable;"" foo|2022-12-21 01:15:39.669 foo|2022-12-21 01:15:56.432 ``` And #336 case also fixed. Special values are described [here](https://www.sqlite.org/lang_createtable.html). > 3.2. The DEFAULT clause > ... A default value may also be one of the special case-independent keywords CURRENT_TIME, CURRENT_DATE or CURRENT_TIMESTAMP. ```shell $ echo 'create table bar (baz text, created_at timestamp default CURRENT_TIMESTAMP)' | sqlite3 foo.db $ sqlite3 foo.db SQLite version 3.39.5 2022-10-14 20:58:05 Enter "".help"" for usage hints. sqlite> .schema bar CREATE TABLE bar (baz text, created_at timestamp default CURRENT_TIMESTAMP); sqlite> .exit $ sqlite-utils transform foo.db bar --column-order baz $ sqlite3 foo.db SQLite version 3.39.5 2022-10-14 20:58:05 Enter "".help"" for usage hints. sqlite> .schema bar CREATE TABLE IF NOT EXISTS ""bar"" ( [baz] TEXT, [created_at] FLOAT DEFAULT CURRENT_TIMESTAMP ); sqlite> .exit $ sqlite-utils transform foo.db bar --column-order baz $ sqlite3 foo.db SQLite version 3.39.5 2022-10-14 20:58:05 Enter "".help"" for usage hints. sqlite> .schema bar CREATE TABLE IF NOT EXISTS ""bar"" ( [baz] TEXT, [created_at] FLOAT DEFAULT CURRENT_TIMESTAMP # ← Non-String Value ); ``` ---- :books: Documentation preview :books:: https://sqlite-utils--519.org.readthedocs.build/en/519/ ",2022-12-21T01:27:52Z,2023-05-08T21:13:37Z,2023-05-08T21:13:37Z,2023-05-08T21:13:37Z,6500fed8b2085869b9714ce3a08c30f61dc829ad,,,0,5e5c262fab69eb3c470afa57fc52d3d7d8beee7a,fc221f9b62ed8624b1d2098e564f525c84497969,CONTRIBUTOR,140912432,https://github.com/simonw/sqlite-utils/pull/519,, 1234321667,PR_kwDOCGYnMM5JkkED,526,closed,0,Fix repeated calls to `Table.convert()`,167893,"Fixes #525. All tests pass. There's perhaps a better way to name lambdas? There could be a collision if a caller passes a function with name like `lambda_123456`. SQLite [documentation](https://www.sqlite.org/appfunc.html) is a little, ah, lite on function name specs. If there is a character that can be used in place of underscore in a SQLite function name that is not permitted in a Python function identifier then that could be a good way to prevent accidental collisions. (I tried dash, colon, dot, no joy). Otherwise, there is little chance of this happening and if it should happen the risk is mitigated by now throwing an exception in the case of a (name, arity) collision without `replace=True`. ---- :books: Documentation preview :books:: https://sqlite-utils--526.org.readthedocs.build/en/526/ ",2023-02-09T00:14:49Z,2023-05-08T21:56:05Z,2023-05-08T21:53:58Z,2023-05-08T21:53:58Z,02f5c4d69d7b4baebde015c56e5bc62923f33314,,,0,486e0cc1cd67e98c22d1f125403144cdedfa3d58,fc221f9b62ed8624b1d2098e564f525c84497969,CONTRIBUTOR,140912432,https://github.com/simonw/sqlite-utils/pull/526,, 1235909998,PR_kwDOCGYnMM5Jqn1u,528,closed,0,Enable `Table.convert()` on falsey values,167893,"Fixes #527 ---- :books: Documentation preview :books:: https://sqlite-utils--528.org.readthedocs.build/en/528/ ",2023-02-10T00:04:09Z,2023-05-08T21:08:23Z,2023-05-08T21:08:23Z,,279fb2c85c6b3fc7b593aaf4245bd3a038bff5f0,,,0,a09df3844ab91b70cebb7b888fc07cb21dffd01c,fc221f9b62ed8624b1d2098e564f525c84497969,CONTRIBUTOR,140912432,https://github.com/simonw/sqlite-utils/pull/528,, 1272169404,PR_kwDOCGYnMM5L08O8,531,closed,0,Add paths for homebrew on Apple silicon,25778,"This also passes in the extension path when specified in GIS methods. Wherever we know an extension path, we use `db.init_spatialite(find_spatialite() or load_extension)`. ---- :books: Documentation preview :books:: https://sqlite-utils--531.org.readthedocs.build/en/531/ ",2023-03-11T22:27:52Z,2023-04-09T01:49:44Z,2023-04-09T01:49:43Z,,24f3eb082b98b8d676bab2eab4f763cd9b50fe96,,,0,afdf6187716b19fce8692f6887a1d45c85477fee,c0251cc9271260de73b4227859a51fab9b4cb745,CONTRIBUTOR,140912432,https://github.com/simonw/sqlite-utils/pull/531,, 1306498393,PR_kwDOCGYnMM5N35VZ,536,closed,0,Add paths for homebrew on Apple silicon,25778,"Does what it says and nothing else. This is the same set of paths as Datasette uses. ---- :books: Documentation preview :books:: https://sqlite-utils--536.org.readthedocs.build/en/536/ ",2023-04-08T13:34:21Z,2023-04-13T01:44:43Z,2023-04-13T01:44:43Z,2023-04-13T01:44:43Z,8f9a729e8aff972cb18de25b40f4113e26bbc758,,,0,cea05dc5eab8d10fbd8943e615d2ab0dceff863c,c0251cc9271260de73b4227859a51fab9b4cb745,CONTRIBUTOR,140912432,https://github.com/simonw/sqlite-utils/pull/536,, 1311438738,PR_kwDOCGYnMM5OKveS,537,closed,0,Support self-referencing FKs in `Table.create`,544011," ---- :books: Documentation preview :books:: https://sqlite-utils--537.org.readthedocs.build/en/537/ ",2023-04-12T20:26:58Z,2023-05-08T22:45:33Z,2023-05-08T21:10:01Z,2023-05-08T21:10:01Z,39ef137e6760d385dc48d03eccf9b89943636fc7,,,0,a75abeb61b91a28650d3b9933e7ec80ad0d92529,c0251cc9271260de73b4227859a51fab9b4cb745,CONTRIBUTOR,140912432,https://github.com/simonw/sqlite-utils/pull/537,, 1445438054,PR_kwDOCGYnMM5WJ6Jm,573,closed,0,feat: Implement a prepare_connection plugin hook,15178711,"Just like the [Datasette prepare_connection hook](https://docs.datasette.io/en/stable/plugin_hooks.html#prepare-connection-conn-database-datasette), this PR adds a similar hook for the `sqlite-utils` plugin system. The sole argument is `conn`, since I don't believe a `database` or `datasette` argument would be relevant here. I want to do this so I can release `sqlite-utils` plugins for my [SQLite extensions](https://github.com/asg017/sqlite-ecosystem), similar to the Datasette plugins I've release for them. An example plugin: https://gist.github.com/asg017/d7cdf0d56e2be87efda28cebee27fa3c ```bash $ sqlite-utils install https://gist.github.com/asg017/d7cdf0d56e2be87efda28cebee27fa3c/archive/5f5ad549a40860787629c69ca120a08c32519e99.zip $ sqlite-utils memory 'select hello(""alex"") as response' [{""response"": ""Hello, alex!""}] ``` Refs: - #574 ---- :books: Documentation preview :books:: https://sqlite-utils--573.org.readthedocs.build/en/573/ ",2023-07-22T22:48:44Z,2023-07-22T22:59:09Z,2023-07-22T22:59:09Z,2023-07-22T22:59:09Z,3f80a026983d3e634f05a46f2a6da162b5139dd9,,,0,faf398fe075f60929337d3cd0f12309fc4229a3c,091c63cfbf7b40e99e2017a3c37619c7689cc447,CONTRIBUTOR,140912432,https://github.com/simonw/sqlite-utils/pull/573,, 1535767490,PR_kwDOCGYnMM5bifPC,596,closed,0,"Fixes mapping for time fields related to mysql, closes #522",4420927,"Adds `COLUMN_TYPE_MAPPING` for `TIME` fields that are mapped as `datetime.timedelta` for MySQL and json represantation for `datetime.timedelta` in order to fix #522 ---- :books: Documentation preview :books:: https://sqlite-utils--596.org.readthedocs.build/en/596/ ",2023-09-29T13:41:48Z,2023-11-04T00:49:50Z,2023-11-04T00:49:50Z,2023-11-04T00:49:50Z,347fdc865e91b8d3410f49a5c9d5b499fbb594c1,,,0,ab12125b3f7d0ef327b08d95f27c6c4cec6fdfb3,622c3a5a7dd53a09c029e2af40c2643fe7579340,CONTRIBUTOR,140912432,https://github.com/simonw/sqlite-utils/pull/596,, 1542299634,PR_kwDOCGYnMM5b7Z_y,598,closed,0,Fixed issue #433 - CLI eats cursor,62745,"The issue is that underlying iterator is not fully consumed within the body of the `with file_progress()` block. Instead, that block creates generator expressions like `docs = (dict(zip(headers, row)) for row in reader)` These iterables are consumed later, outside the `with file_progress()` block, which consumes the underlying iterator, and in turn updates the progress bar. This means that the `ProgressBar.__exit__` method gets called before the last time the `ProgressBar.update` method gets called. The result is that the code to make the cursor invisible (inside the `update()` method) is called after the cleanup code to make it visible (in the `__exit__` method). The fix is to move consumption of the `docs` iterators within the progress bar block. ( (An additional fix, to make ProgressBar more robust against this kind of misuse, would to make it refusing to update after its `__exit__` method had been called, just like files cannot be `read()` after they are closed. That requires a in the click library). Note that Github diff obscures the simplicity of this diff, it's just indenting a block of code. ---- :books: Documentation preview :books:: https://sqlite-utils--598.org.readthedocs.build/en/598/ ",2023-10-04T18:06:58Z,2023-11-04T00:46:55Z,2023-11-04T00:40:30Z,2023-11-04T00:40:30Z,37273d7f63f08872aa1c90c4233a0580e384ac19,,,0,76113d1cb1387811d556fb7a32e0628492c34aea,622c3a5a7dd53a09c029e2af40c2643fe7579340,CONTRIBUTOR,140912432,https://github.com/simonw/sqlite-utils/pull/598,, 1586779743,PR_kwDOCGYnMM5elFZf,600,closed,0,Add spatialite arm64 linux path,37802088,"According to both [Debian](https://packages.debian.org/bookworm/arm64/libsqlite3-mod-spatialite/filelist) and [Ubuntu](https://packages.ubuntu.com/mantic/arm64/libsqlite3-mod-spatialite/filelist), the correct “target triple” for arm64 is `aarch64-linux-gnu`, so we should be looking in `/usr/lib/aarch64-linux-gnu` for `mod_spatialite.so`. I can confirm that on both of my Debian arm64 SBCs, `libsqlite3-mod-spatialite` installs to that path. ``` $ ls -l /usr/lib/*/*spatial* lrwxrwxrwx 1 root root 23 Dec 1 2022 /usr/lib/aarch64-linux-gnu/mod_spatialite.so -> mod_spatialite.so.7.1.0 lrwxrwxrwx 1 root root 23 Dec 1 2022 /usr/lib/aarch64-linux-gnu/mod_spatialite.so.7 -> mod_spatialite.so.7.1.0 -rw-r--r-- 1 root root 7348584 Dec 1 2022 /usr/lib/aarch64-linux-gnu/mod_spatialite.so.7.1.0 ``` This is a set of before and after snippets of pytest’s output for this PR. ### Before ``` $ pytest tests/test_get.py ...... [ 73%] tests/test_gis.py ssssssssssss [ 75%] tests/test_hypothesis.py .... [ 75%] ``` ### After ``` $ pytest tests/test_get.py ...... [ 73%] tests/test_gis.py ............ [ 75%] tests/test_hypothesis.py .... [ 75%] ``` Issue: #599 ---- :books: Documentation preview :books:: https://sqlite-utils--600.org.readthedocs.build/en/600/ ",2023-11-03T22:23:26Z,2023-11-04T00:34:33Z,2023-11-04T00:31:49Z,2023-11-04T00:31:49Z,b92ea4793ce4dcb73cf762aae634ab72f65ec50f,,,0,b1a60766a4150268557c4445297087a3f076be01,622c3a5a7dd53a09c029e2af40c2643fe7579340,CONTRIBUTOR,140912432,https://github.com/simonw/sqlite-utils/pull/600,, 1608050242,PR_kwDOCGYnMM5f2OZC,604,closed,0,Add more STRICT table support,16437338,"- https://github.com/simonw/sqlite-utils/issues/344#issuecomment-982014776 Make `table.transform()` preserve STRICT mode. ---- :books: Documentation preview :books:: https://sqlite-utils--604.org.readthedocs.build/en/604/ ",2023-11-19T19:38:53Z,2023-12-08T05:17:20Z,2023-12-08T05:05:27Z,2023-12-08T05:05:27Z,1500c19bd0f31b2e7f28a5ec2d7bfa133a2e4d4c,,,0,61c6e26cf922c70b65b161473723ff9d869a04a5,9286c1ba432e890b1bb4b2a1f847b15364c1fa18,CONTRIBUTOR,140912432,https://github.com/simonw/sqlite-utils/pull/604,,