github
html_url | issue_url | id | node_id | user | created_at | updated_at | author_association | body | reactions | issue | performed_via_github_app |
---|---|---|---|---|---|---|---|---|---|---|---|
https://github.com/simonw/sqlite-utils/pull/312#issuecomment-896162082 | https://api.github.com/repos/simonw/sqlite-utils/issues/312 | 896162082 | IC_kwDOCGYnMM41alki | 22429695 | 2021-08-10T17:10:39Z | 2021-08-10T23:07:35Z | NONE | # [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/312?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report > Merging [#312](https://codecov.io/gh/simonw/sqlite-utils/pull/312?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (43bc064) into [main](https://codecov.io/gh/simonw/sqlite-utils/commit/ee469e3122d6f5973ec2584c1580d930daca2e7c?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (ee469e3) will **decrease** coverage by `0.02%`. > The diff coverage is `96.84%`. [![Impacted file tree graph](https://codecov.io/gh/simonw/sqlite-utils/pull/312/graphs/tree.svg?width=650&height=150&src=pr&token=O0X3703L9P&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)](https://codecov.io/gh/simonw/sqlite-utils/pull/312?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) ```diff @@ Coverage Diff @@ ## main #312 +/- ## ========================================== - Coverage 96.30% 96.28% -0.03% ========================================== Files 5 5 Lines 2168 2179 +11 ========================================== + Hits 2088 2098 +10 - Misses 80 81 +1 ``` | [Impacted Files](https://codecov.io/gh/simonw/sqlite-utils/pull/312?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage Δ | | |---|---|---| | [sqlite\_utils/db.py](https://codecov.io/gh/simonw/sqlite-utils/pull/312/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-c3FsaXRlX3V0aWxzL2RiLnB5) | `97.91% <96.84%> (-0.08%)` … | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
965143346 | |
https://github.com/simonw/sqlite-utils/pull/312#issuecomment-896156971 | https://api.github.com/repos/simonw/sqlite-utils/issues/312 | 896156971 | IC_kwDOCGYnMM41akUr | 9599 | 2021-08-10T17:04:22Z | 2021-08-10T17:05:59Z | OWNER | I'm going to get Read The Docs to build the docs for this branch too - on https://readthedocs.org/projects/sqlite-utils/versions/ I am clicking this button: <img width="881" alt="Versions___Read_the_Docs" src="https://user-images.githubusercontent.com/9599/128903015-0f2649e7-34a8-4dfd-984c-b1b5bd8a98be.png"> I then set it to "active" (so pushes to the branch will build it) and "hidden" (so it wouldn't show up in search or in the navigation menu). https://docs.readthedocs.io/en/stable/versions.html#version-states <img width="846" alt="autodoc_-_sqlite-utils___Read_the_Docs" src="https://user-images.githubusercontent.com/9599/128903217-efe827c6-4f54-42dd-943e-ddf9ab3a05d8.png"> | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
965143346 | |
https://github.com/simonw/sqlite-utils/pull/312#issuecomment-896154028 | https://api.github.com/repos/simonw/sqlite-utils/issues/312 | 896154028 | IC_kwDOCGYnMM41ajms | 9599 | 2021-08-10T17:01:06Z | 2021-08-10T17:01:06Z | OWNER | On Python 3.6: ``` sqlite_utils/db.py:366: in Database def tables(self) -> List[Table]: E NameError: name 'Table' is not defined ``` Python 3.7 can fix this with `from __future__ import annotations` but since we still support 3.6 I'll have to use a string instead. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
965143346 | |
https://github.com/simonw/sqlite-utils/pull/303#issuecomment-890553014 | https://api.github.com/repos/simonw/sqlite-utils/issues/303 | 890553014 | IC_kwDOCGYnMM41FMK2 | 22429695 | 2021-08-01T16:53:35Z | 2021-08-02T04:45:19Z | NONE | # [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/303?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report > Merging [#303](https://codecov.io/gh/simonw/sqlite-utils/pull/303?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (4c3bf97) into [main](https://codecov.io/gh/simonw/sqlite-utils/commit/c7e8d72be9fe8fe0811f685a18eebc637662d41b?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (c7e8d72) will **increase** coverage by `0.24%`. > The diff coverage is `100.00%`. [![Impacted file tree graph](https://codecov.io/gh/simonw/sqlite-utils/pull/303/graphs/tree.svg?width=650&height=150&src=pr&token=O0X3703L9P&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)](https://codecov.io/gh/simonw/sqlite-utils/pull/303?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) ```diff @@ Coverage Diff @@ ## main #303 +/- ## ========================================== + Coverage 96.04% 96.28% +0.24% ========================================== Files 4 5 +1 Lines 1998 2129 +131 ========================================== + Hits 1919 2050 +131 Misses 79 79 ``` | [Impacted Files](https://codecov.io/gh/simonw/sqlite-utils/pull/303?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage Δ | | |---|---|---| | [sqlite\_utils/cli.py](https://codecov.io/gh/simonw/sqlite-utils/pull/303/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-c3FsaXRlX3V0aWxzL2NsaS5weQ==) | `94.91% <100.00%> (+0… | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
957536983 | |
https://github.com/simonw/sqlite-utils/pull/293#issuecomment-868728092 | https://api.github.com/repos/simonw/sqlite-utils/issues/293 | 868728092 | MDEyOklzc3VlQ29tbWVudDg2ODcyODA5Mg== | 9599 | 2021-06-25T17:39:35Z | 2021-06-25T17:39:35Z | OWNER | Here's more about this problem: https://github.com/numpy/numpy/issues/15947 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
929748885 | |
https://github.com/simonw/sqlite-utils/pull/293#issuecomment-868134040 | https://api.github.com/repos/simonw/sqlite-utils/issues/293 | 868134040 | MDEyOklzc3VlQ29tbWVudDg2ODEzNDA0MA== | 9599 | 2021-06-25T01:49:44Z | 2021-06-25T01:50:33Z | OWNER | Test failed on 3.10 with `numpy` on macOS: ``` sqlite_utils/__init__.py:1: in <module> 11 from .db import Database 12 sqlite_utils/db.py:48: in <module> 13 import numpy as np # type: ignore 14 ../../../hostedtoolcache/Python/3.10.0-beta.3/x64/lib/python3.10/site-packages/numpy/__init__.py:391: in <module> 15 raise RuntimeError(msg) 16 E RuntimeError: Polyfit sanity test emitted a warning, most likely due to using a buggy Accelerate backend. If you compiled yourself, more information is available at https://numpy.org/doc/stable/user/building.html#accelerated-blas-lapack-libraries Otherwise report this to the vendor that provided NumPy. 17 E RankWarning: Polyfit may be poorly conditioned 18 Error: Process completed with exit code 4. ``` | { "total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 1 } |
929748885 | |
https://github.com/simonw/sqlite-utils/pull/293#issuecomment-868125750 | https://api.github.com/repos/simonw/sqlite-utils/issues/293 | 868125750 | MDEyOklzc3VlQ29tbWVudDg2ODEyNTc1MA== | 22429695 | 2021-06-25T01:42:43Z | 2021-06-25T01:42:43Z | NONE | # [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/293?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report > Merging [#293](https://codecov.io/gh/simonw/sqlite-utils/pull/293?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (ae0f46a) into [main](https://codecov.io/gh/simonw/sqlite-utils/commit/747be6057d09a4e5d9d726e29d5cf99b10c59dea?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (747be60) will **not change** coverage. > The diff coverage is `n/a`. [![Impacted file tree graph](https://codecov.io/gh/simonw/sqlite-utils/pull/293/graphs/tree.svg?width=650&height=150&src=pr&token=O0X3703L9P&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)](https://codecov.io/gh/simonw/sqlite-utils/pull/293?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) ```diff @@ Coverage Diff @@ ## main #293 +/- ## ======================================= Coverage 96.03% 96.03% ======================================= Files 4 4 Lines 1994 1994 ======================================= Hits 1915 1915 Misses 79 79 ``` ------ [Continue to review full report at Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/293?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](htt… | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
929748885 | |
https://github.com/simonw/sqlite-utils/pull/277#issuecomment-864092515 | https://api.github.com/repos/simonw/sqlite-utils/issues/277 | 864092515 | MDEyOklzc3VlQ29tbWVudDg2NDA5MjUxNQ== | 9599 | 2021-06-18T14:47:57Z | 2021-06-18T14:47:57Z | OWNER | This is a neat improvement. | { "total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 1, "rocket": 0, "eyes": 0 } |
923612361 | |
https://github.com/simonw/sqlite-utils/pull/277#issuecomment-863205049 | https://api.github.com/repos/simonw/sqlite-utils/issues/277 | 863205049 | MDEyOklzc3VlQ29tbWVudDg2MzIwNTA0OQ== | 22429695 | 2021-06-17T12:40:49Z | 2021-06-17T12:40:49Z | NONE | # [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/277?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report > Merging [#277](https://codecov.io/gh/simonw/sqlite-utils/pull/277?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (abbd324) into [main](https://codecov.io/gh/simonw/sqlite-utils/commit/a19ce1a4d0048d389411cfe11a5dbe4c503720e1?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (a19ce1a) will **increase** coverage by `0.00%`. > The diff coverage is `100.00%`. [![Impacted file tree graph](https://codecov.io/gh/simonw/sqlite-utils/pull/277/graphs/tree.svg?width=650&height=150&src=pr&token=O0X3703L9P&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)](https://codecov.io/gh/simonw/sqlite-utils/pull/277?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) ```diff @@ Coverage Diff @@ ## main #277 +/- ## ======================================= Coverage 96.06% 96.06% ======================================= Files 4 4 Lines 1828 1829 +1 ======================================= + Hits 1756 1757 +1 Misses 72 72 ``` | [Impacted Files](https://codecov.io/gh/simonw/sqlite-utils/pull/277?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage Δ | | |---|---|---| | [sqlite\_utils/cli.py](https://codecov.io/gh/simonw/sqlite-utils/pull/277/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-c3FsaXRlX3V0aWxzL2NsaS5weQ==) | `94.03% <100.00%> (+<0.01%)` | :arrow_up: | ------… | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
923612361 | |
https://github.com/simonw/sqlite-utils/pull/273#issuecomment-864099764 | https://api.github.com/repos/simonw/sqlite-utils/issues/273 | 864099764 | MDEyOklzc3VlQ29tbWVudDg2NDA5OTc2NA== | 9599 | 2021-06-18T14:59:27Z | 2021-06-18T14:59:27Z | OWNER | I'm going to merge this as-is and work on the JSON/TSV support in a separate issue. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
922099793 | |
https://github.com/simonw/sqlite-utils/pull/273#issuecomment-862817185 | https://api.github.com/repos/simonw/sqlite-utils/issues/273 | 862817185 | MDEyOklzc3VlQ29tbWVudDg2MjgxNzE4NQ== | 22429695 | 2021-06-17T00:15:34Z | 2021-06-17T00:15:34Z | NONE | # [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/273?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report > :exclamation: No coverage uploaded for pull request base (`main@78aebb6`). [Click here to learn what that means](https://docs.codecov.io/docs/error-reference?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#section-missing-base-commit). > The diff coverage is `n/a`. [![Impacted file tree graph](https://codecov.io/gh/simonw/sqlite-utils/pull/273/graphs/tree.svg?width=650&height=150&src=pr&token=O0X3703L9P&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)](https://codecov.io/gh/simonw/sqlite-utils/pull/273?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) ```diff @@ Coverage Diff @@ ## main #273 +/- ## ======================================= Coverage ? 96.10% ======================================= Files ? 4 Lines ? 1873 Branches ? 0 ======================================= Hits ? 1800 Misses ? 73 Partials ? 0 ``` ------ [Continue to review full report at Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/273?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/273?src=pr&el=footer&utm_medium=referr… | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
922099793 | |
https://github.com/simonw/sqlite-utils/pull/273#issuecomment-862605436 | https://api.github.com/repos/simonw/sqlite-utils/issues/273 | 862605436 | MDEyOklzc3VlQ29tbWVudDg2MjYwNTQzNg== | 9599 | 2021-06-16T18:19:05Z | 2021-06-16T18:19:05Z | OWNER | `--attach` documentation: https://github.com/simonw/sqlite-utils/blob/192dc2c5b73bd836ab8e2e5fed4b36c6ea02f250/docs/cli.rst#joining-in-memory-data-against-existing-databases-using-attach | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
922099793 | |
https://github.com/simonw/sqlite-utils/pull/273#issuecomment-862046009 | https://api.github.com/repos/simonw/sqlite-utils/issues/273 | 862046009 | MDEyOklzc3VlQ29tbWVudDg2MjA0NjAwOQ== | 9599 | 2021-06-16T05:15:38Z | 2021-06-16T05:15:38Z | OWNER | I'm going to add a `--encoding` option - it will affect ALL CSV input files, so if you have CSV files with different encodings you'll need to sort that mess out yourself (likely by importing each CSV file separately into a database using `sqlite-utils insert` with different `--encoding` values). | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
922099793 | |
https://github.com/simonw/sqlite-utils/pull/273#issuecomment-862045639 | https://api.github.com/repos/simonw/sqlite-utils/issues/273 | 862045639 | MDEyOklzc3VlQ29tbWVudDg2MjA0NTYzOQ== | 9599 | 2021-06-16T05:14:38Z | 2021-06-16T05:14:38Z | OWNER | Can't share much code though since a bunch of that `insert` stuff is specific to that command - showing progress bars, returning errors on illegal option combinations etc. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
922099793 | |
https://github.com/simonw/sqlite-utils/pull/273#issuecomment-862045438 | https://api.github.com/repos/simonw/sqlite-utils/issues/273 | 862045438 | MDEyOklzc3VlQ29tbWVudDg2MjA0NTQzOA== | 9599 | 2021-06-16T05:14:00Z | 2021-06-16T05:14:00Z | OWNER | I should probably refactor the CSV/JSON/loading stuff into a function in `utils.py` in order to share some of the implementation with the existing `sqlite-utils insert` code: https://github.com/simonw/sqlite-utils/blob/287cdcae8908916687f2ecccc87c38549d004ac6/sqlite_utils/cli.py#L691-L734 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
922099793 | |
https://github.com/simonw/sqlite-utils/pull/273#issuecomment-862043974 | https://api.github.com/repos/simonw/sqlite-utils/issues/273 | 862043974 | MDEyOklzc3VlQ29tbWVudDg2MjA0Mzk3NA== | 9599 | 2021-06-16T05:10:12Z | 2021-06-16T05:10:12Z | OWNER | I can stop promoting `:memory:` here and promote `memory` instead: https://github.com/simonw/sqlite-utils/blob/c7234cae8336b8525034e8f917d82dd0699abd42/docs/cli.rst#L83-L86 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
922099793 | |
https://github.com/simonw/sqlite-utils/pull/273#issuecomment-862042110 | https://api.github.com/repos/simonw/sqlite-utils/issues/273 | 862042110 | MDEyOklzc3VlQ29tbWVudDg2MjA0MjExMA== | 9599 | 2021-06-16T05:05:51Z | 2021-06-16T05:06:11Z | OWNER | Initial documentation is here: https://github.com/simonw/sqlite-utils/blob/c7234cae8336b8525034e8f917d82dd0699abd42/docs/cli.rst#running-queries-directly-against-csv-data It only talks about CSV at the moment - needs to be updated to mention JSON too once that is implemented. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
922099793 | |
https://github.com/simonw/sqlite-utils/pull/258#issuecomment-843702392 | https://api.github.com/repos/simonw/sqlite-utils/issues/258 | 843702392 | MDEyOklzc3VlQ29tbWVudDg0MzcwMjM5Mg== | 9599 | 2021-05-19T02:47:37Z | 2021-05-19T02:47:37Z | OWNER | I'm going to merge this and add a test - thanks! | { "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
868191959 | |
https://github.com/simonw/sqlite-utils/pull/254#issuecomment-843705533 | https://api.github.com/repos/simonw/sqlite-utils/issues/254 | 843705533 | MDEyOklzc3VlQ29tbWVudDg0MzcwNTUzMw== | 9599 | 2021-05-19T02:57:22Z | 2021-05-19T02:57:22Z | OWNER | Thanks! | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
857280617 | |
https://github.com/simonw/sqlite-utils/pull/247#issuecomment-901344634 | https://api.github.com/repos/simonw/sqlite-utils/issues/247 | 901344634 | IC_kwDOCGYnMM41uW16 | 22429695 | 2021-08-18T18:42:54Z | 2021-08-18T18:42:54Z | NONE | # [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/247?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report > Merging [#247](https://codecov.io/gh/simonw/sqlite-utils/pull/247?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (af989af) into [main](https://codecov.io/gh/simonw/sqlite-utils/commit/1fe73c898b44695052f1a9ca832818d50cecf662?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (1fe73c8) will **decrease** coverage by `0.03%`. > The diff coverage is `85.71%`. [![Impacted file tree graph](https://codecov.io/gh/simonw/sqlite-utils/pull/247/graphs/tree.svg?width=650&height=150&src=pr&token=O0X3703L9P&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)](https://codecov.io/gh/simonw/sqlite-utils/pull/247?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) ```diff @@ Coverage Diff @@ ## main #247 +/- ## ========================================== - Coverage 96.28% 96.24% -0.04% ========================================== Files 5 5 Lines 2179 2186 +7 ========================================== + Hits 2098 2104 +6 - Misses 81 82 +1 ``` | [Impacted Files](https://codecov.io/gh/simonw/sqlite-utils/pull/247?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage Δ | | |---|---|---| | [sqlite\_utils/db.py](https://codecov.io/gh/simonw/sqlite-utils/pull/247/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-c3FsaXRlX3V0aWxzL2RiLnB5) | `97.84% <85.71%> (-0.08%)` … | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
832687563 | |
https://github.com/simonw/sqlite-utils/pull/247#issuecomment-901338988 | https://api.github.com/repos/simonw/sqlite-utils/issues/247 | 901338988 | IC_kwDOCGYnMM41uVds | 9599 | 2021-08-18T18:33:39Z | 2021-08-18T18:33:39Z | OWNER | This was also requested in #296. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
832687563 | |
https://github.com/simonw/sqlite-utils/pull/245#issuecomment-843705721 | https://api.github.com/repos/simonw/sqlite-utils/issues/245 | 843705721 | MDEyOklzc3VlQ29tbWVudDg0MzcwNTcyMQ== | 9599 | 2021-05-19T02:58:01Z | 2021-05-19T02:58:01Z | OWNER | Thanks very much. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
830803173 | |
https://github.com/simonw/sqlite-utils/pull/244#issuecomment-843705806 | https://api.github.com/repos/simonw/sqlite-utils/issues/244 | 843705806 | MDEyOklzc3VlQ29tbWVudDg0MzcwNTgwNg== | 9599 | 2021-05-19T02:58:18Z | 2021-05-19T02:58:18Z | OWNER | Thanks! | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
820468864 | |
https://github.com/simonw/sqlite-utils/pull/225#issuecomment-778841547 | https://api.github.com/repos/simonw/sqlite-utils/issues/225 | 778841547 | MDEyOklzc3VlQ29tbWVudDc3ODg0MTU0Nw== | 9599 | 2021-02-14T21:04:13Z | 2021-02-14T21:04:13Z | OWNER | I added a test and fixed this in #234 - thanks for the fix. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
797159961 | |
https://github.com/simonw/sqlite-utils/pull/225#issuecomment-778834504 | https://api.github.com/repos/simonw/sqlite-utils/issues/225 | 778834504 | MDEyOklzc3VlQ29tbWVudDc3ODgzNDUwNA== | 9599 | 2021-02-14T20:09:30Z | 2021-02-14T20:09:30Z | OWNER | Thanks for this. I'm going to try and get the test suite to run in Windows on GitHub Actions. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
797159961 | |
https://github.com/simonw/sqlite-utils/pull/224#issuecomment-778828495 | https://api.github.com/repos/simonw/sqlite-utils/issues/224 | 778828495 | MDEyOklzc3VlQ29tbWVudDc3ODgyODQ5NQ== | 9599 | 2021-02-14T19:31:06Z | 2021-02-14T19:31:06Z | OWNER | I'm going to add a `offset=` parameter to support this case. Thanks for the suggestion! | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
792297010 | |
https://github.com/simonw/sqlite-utils/pull/224#issuecomment-765678057 | https://api.github.com/repos/simonw/sqlite-utils/issues/224 | 765678057 | MDEyOklzc3VlQ29tbWVudDc2NTY3ODA1Nw== | 37962604 | 2021-01-22T20:53:06Z | 2021-01-23T20:13:27Z | NONE | I'm using the FTS methods in sqlite-utils for this website: [drwn.io](https://drwn.io/). I wanted to get pagination to have some kind of infinite scrolling in the landing page, and I ended up using that. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
792297010 | |
https://github.com/simonw/sqlite-utils/pull/208#issuecomment-743956666 | https://api.github.com/repos/simonw/sqlite-utils/issues/208 | 743956666 | MDEyOklzc3VlQ29tbWVudDc0Mzk1NjY2Ng== | 9599 | 2020-12-13T05:44:49Z | 2020-12-13T05:44:49Z | OWNER | Example output: ``` % sqlite-utils analyze-tables github.db tags tags.repo: (1/3) Total rows: 261 Null rows: 0 Blank rows: 0 Distinct values: 14 Most common: 88: 107914493 75: 140912432 27: 206156866 21: 207052882 17: 197431109 8: 197882382 5: 256834907 5: 205429375 4: 248903544 3: 206202864 Least common: 1: 209590345 2: 206649770 2: 303218369 3: 206202864 3: 213286752 4: 248903544 5: 205429375 5: 256834907 8: 197882382 17: 197431109 tags.name: (2/3) Total rows: 261 Null rows: 0 Blank rows: 0 Distinct values: 175 Most common: 10: 0.2 9: 0.1 7: 0.3 6: 0.4 5: 0.7 5: 0.5 5: 0.1a 4: 0.9 4: 0.8 4: 0.6 Least common: 1: 0.1.1 1: 0.11.1 1: 0.1a2 1: 0.20.1 1: 0.21.1 1: 0.21.2 1: 0.21.3 1: 0.22 1: 0.22.1 1: 0.23 tags.sha: (3/3) Total rows: 261 Null rows: 0 Blank rows: 0 Distinct values: 261 ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
763320133 | |
https://github.com/simonw/sqlite-utils/pull/208#issuecomment-743708524 | https://api.github.com/repos/simonw/sqlite-utils/issues/208 | 743708524 | MDEyOklzc3VlQ29tbWVudDc0MzcwODUyNA== | 9599 | 2020-12-12T05:48:20Z | 2020-12-12T05:48:32Z | OWNER | ``` % sqlite-utils analyze-tables ../datasette/fixtures.db facetable --column pk 1/1: ColumnDetails(table='facetable', column='pk', total_rows=15, num_null=0, num_blank=0, num_distinct=15, most_common=None, least_common=None) ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
763320133 | |
https://github.com/simonw/sqlite-utils/pull/208#issuecomment-743708325 | https://api.github.com/repos/simonw/sqlite-utils/issues/208 | 743708325 | MDEyOklzc3VlQ29tbWVudDc0MzcwODMyNQ== | 9599 | 2020-12-12T05:46:27Z | 2020-12-12T05:46:27Z | OWNER | It would be neat if you could optionally specify a subset of columns to analyze, using `-c` or `--column`. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
763320133 | |
https://github.com/simonw/sqlite-utils/pull/208#issuecomment-743708169 | https://api.github.com/repos/simonw/sqlite-utils/issues/208 | 743708169 | MDEyOklzc3VlQ29tbWVudDc0MzcwODE2OQ== | 9599 | 2020-12-12T05:44:46Z | 2020-12-12T05:44:46Z | OWNER | If there are less than ten values is it worth outputting them twice, once in `most_common` and then in reverse in `least_common`? Feels redundant - I think I should leave `least_common` empty in that case. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
763320133 | |
https://github.com/simonw/sqlite-utils/pull/208#issuecomment-743708080 | https://api.github.com/repos/simonw/sqlite-utils/issues/208 | 743708080 | MDEyOklzc3VlQ29tbWVudDc0MzcwODA4MA== | 9599 | 2020-12-12T05:43:45Z | 2020-12-12T05:43:45Z | OWNER | CLI output looks like this at the moment, which is bad: ``` % sqlite-utils analyze-tables ../datasette/fixtures.db facetable 1/10: ColumnDetails(table='facetable', column='pk', total_rows=15, num_null=0, num_blank=0, num_distinct=15, most_common=None, least_common=None) 2/10: ColumnDetails(table='facetable', column='created', total_rows=15, num_null=0, num_blank=0, num_distinct=4, most_common=[('2019-01-17 08:00:00', 4), ('2019-01-15 08:00:00', 4), ('2019-01-14 08:00:00', 4), ('2019-01-16 08:00:00', 3)], least_common=[('2019-01-16 08:00:00', 3), ('2019-01-14 08:00:00', 4), ('2019-01-15 08:00:00', 4), ('2019-01-17 08:00:00', 4)]) 3/10: ColumnDetails(table='facetable', column='planet_int', total_rows=15, num_null=0, num_blank=0, num_distinct=2, most_common=[(1, 14), (2, 1)], least_common=[(2, 1), (1, 14)]) 4/10: ColumnDetails(table='facetable', column='on_earth', total_rows=15, num_null=0, num_blank=0, num_distinct=2, most_common=[(1, 14), (0, 1)], least_common=[(0, 1), (1, 14)]) 5/10: ColumnDetails(table='facetable', column='state', total_rows=15, num_null=0, num_blank=0, num_distinct=3, most_common=[('CA', 10), ('MI', 4), ('MC', 1)], least_common=[('MC', 1), ('MI', 4), ('CA', 10)]) 6/10: ColumnDetails(table='facetable', column='city_id', total_rows=15, num_null=0, num_blank=0, num_distinct=4, most_common=[(1, 6), (3, 4), (2, 4), (4, 1)], least_common=[(4, 1), (2, 4), (3, 4), (1, 6)]) 7/10: ColumnDetails(table='facetable', column='neighborhood', total_rows=15, num_null=0, num_blank=0, num_distinct=14, most_common=[('Downtown', 2), ('Tenderloin', 1), ('SOMA', 1), ('Mission', 1), ('Mexicantown', 1), ('Los Feliz', 1), ('Koreatown', 1), ('Hollywood', 1), ('Hayes Valley', 1), ('Greektown', 1)], least_common=[('Arcadia Planitia', 1), ('Bernal Heights', 1), ('Corktown', 1), ('Dogpatch', 1), ('Greektown', 1), ('Hayes Valley', 1), ('Hollywood', 1), ('Koreatown', 1), ('Los Feliz', 1), ('Mexicantown', 1)]) 8/10: ColumnDetails(table='facetable', column='tags', total_rows=15, num_null=0, num_blank=0, num_distinct=3,… | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
763320133 | |
https://github.com/simonw/sqlite-utils/pull/208#issuecomment-743707969 | https://api.github.com/repos/simonw/sqlite-utils/issues/208 | 743707969 | MDEyOklzc3VlQ29tbWVudDc0MzcwNzk2OQ== | 9599 | 2020-12-12T05:42:26Z | 2020-12-12T05:43:06Z | OWNER | Should truncate values in the least/most common JSON array to a sensible length, otherwise you end up with stuff like this: ```json [ [ "b'\\x00\\x05barry\\x03\\x01\\x02\\x00\\x00\\x03cat\\x03\\x01\\x03\\x00\\x00\\x03dog\\x08\\x01\\x01\\x01\\x03\\x00\\x01\\x03\\x00\\x00\\x07panther\\x05\\x01\\x01\\x02\\x02\\x00\\x01\\x03uma\\x05\\x02\\x01\\x02\\x02\\x00\\x00\\x04sara\\x05\\x02\\x01\\x01\\x02\\x00\\x00\\x05terry\\x08\\x01\\x01\\x01\\x02\\x00\\x01\\x02\\x00\\x00\\x06weasel\\x05\\x02\\x01\\x01\\x03\\x00'", 1 ] ] ``` This example also shows that binary values (like those in `_fts` tables) look a bit weird, but I think I'm OK with that since binary data can't be represented neatly in JSON anyway. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
763320133 | |
https://github.com/simonw/sqlite-utils/pull/204#issuecomment-740796067 | https://api.github.com/repos/simonw/sqlite-utils/issues/204 | 740796067 | MDEyOklzc3VlQ29tbWVudDc0MDc5NjA2Nw== | 9599 | 2020-12-08T17:49:22Z | 2020-12-08T17:49:22Z | OWNER | Great catch, thank you. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
752888228 | |
https://github.com/simonw/sqlite-utils/pull/203#issuecomment-774217792 | https://api.github.com/repos/simonw/sqlite-utils/issues/203 | 774217792 | MDEyOklzc3VlQ29tbWVudDc3NDIxNzc5Mg== | 1049910 | 2021-02-05T18:44:13Z | 2021-02-05T18:44:13Z | NONE | Thanks for looking at this - home schooling kids has prevented me from replying. I'd struggled with how to adapt the API for the foreign keys too - I definitely tried the String/Tuple approach. I hadn't considered the breaking changes that would introduce though. I can take a look at this and try and make the change - see which of your options works best. I've got a workaround for the use-case I was looking at this for, so it wouldn't be a problem for me if it was put on the back burner until a hypothetical v4.0 anyway. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
743384829 | |
https://github.com/simonw/sqlite-utils/pull/203#issuecomment-753567932 | https://api.github.com/repos/simonw/sqlite-utils/issues/203 | 753567932 | MDEyOklzc3VlQ29tbWVudDc1MzU2NzkzMg== | 9599 | 2021-01-03T04:54:43Z | 2021-01-03T04:54:43Z | OWNER | Another option: expand the `ForeignKey` object to have `.columns` and `.other_columns` properties in addition to the existing `.column` and `.other_column` properties. These new plural properties would always return a tuple, which would be a one-item tuple for a non-compound-foreign-key. The question then is what should `.column` and `.other_column` return for compound foreign keys? I'd be inclined to say they should return `None` - which would trigger errors in code that encounters a compound foreign key for the first time, but those errors would at least be a strong indicator as to what had gone wrong. We can label `.column` and `.other_column` as deprecated and then remove them in `sqlite-utils 4.0`. Since this would still be a breaking change in some minor edge-cases I'm thinking maybe 4.0 needs to happen in order to land this feature. I'm not opposed to doing that, I was just hoping it might be avoidable. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
743384829 | |
https://github.com/simonw/sqlite-utils/pull/203#issuecomment-753567744 | https://api.github.com/repos/simonw/sqlite-utils/issues/203 | 753567744 | MDEyOklzc3VlQ29tbWVudDc1MzU2Nzc0NA== | 9599 | 2021-01-03T04:51:44Z | 2021-01-03T04:51:44Z | OWNER | One way that this could avoid a breaking change would be to have `fk.column` and `fk.other_column` remain as strings for non-compound-foreign-keys, but turn into tuples for a compound foreign key. This is a bit of an ugly API design, and it could still break existing code that encounters a compound foreign key for the first time - but it would leave code working for the more common case of a non-compound-foreign-key. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
743384829 | |
https://github.com/simonw/sqlite-utils/pull/203#issuecomment-753567508 | https://api.github.com/repos/simonw/sqlite-utils/issues/203 | 753567508 | MDEyOklzc3VlQ29tbWVudDc1MzU2NzUwOA== | 9599 | 2021-01-03T04:48:17Z | 2021-01-03T04:48:17Z | OWNER | Sorry for taking so long to review this! This approach looks great to me - being able to optionally pass a tuple anywhere the API currently expects a column is smart, and it's consistent with how the `pk=` parameter works elsewhere. There's just one problem I can see with this: the way it changes the `ForeignKey(...)` interface to always return a tuple for `.column` and `.other_column`, even if that tuple only contains a single item. This represents a breaking change to the existing API - any code that expects `ForeignKey.column` to be a single string (which is any code that has been written against that) will break. As such, I'd have to bump the major version of `sqlite-utils` to `4.0` in order to ship this. Ideally I'd like to make this change in a way that doesn't represent an API compatibility break. I need to think a bit harder about how that might be achieved. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
743384829 | |
https://github.com/simonw/sqlite-utils/pull/203#issuecomment-743966289 | https://api.github.com/repos/simonw/sqlite-utils/issues/203 | 743966289 | MDEyOklzc3VlQ29tbWVudDc0Mzk2NjI4OQ== | 9599 | 2020-12-13T07:20:51Z | 2020-12-13T07:20:51Z | OWNER | Sorry for not reviewing this yet! I'll try to carve out time to look at it in the next few days. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
743384829 | |
https://github.com/simonw/sqlite-utils/pull/203#issuecomment-1404070841 | https://api.github.com/repos/simonw/sqlite-utils/issues/203 | 1404070841 | IC_kwDOCGYnMM5TsGu5 | 536941 | 2023-01-25T18:47:18Z | 2023-01-25T18:47:18Z | CONTRIBUTOR | i'll adopt this PR to make the changes @simonw suggested https://github.com/simonw/sqlite-utils/pull/203#issuecomment-753567932 | { "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
743384829 | |
https://github.com/simonw/sqlite-utils/pull/203#issuecomment-1033641009 | https://api.github.com/repos/simonw/sqlite-utils/issues/203 | 1033641009 | IC_kwDOCGYnMM49nBwx | 82988 | 2022-02-09T11:06:18Z | 2022-02-09T11:06:18Z | NONE | Is there any progress elsewhere on the handling of compound / composite foreign keys, or is this PR still effectively open? | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
743384829 | |
https://github.com/simonw/sqlite-utils/pull/195#issuecomment-723148906 | https://api.github.com/repos/simonw/sqlite-utils/issues/195 | 723148906 | MDEyOklzc3VlQ29tbWVudDcyMzE0ODkwNg== | 9599 | 2020-11-06T15:43:51Z | 2020-11-06T15:43:51Z | OWNER | Thanks to #198 (introducing a `rank_bm25()` custom function for FTS4) this feature will be able to offer relevance search for both FTS5 AND FTS4 tables. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
735663855 | |
https://github.com/simonw/sqlite-utils/pull/195#issuecomment-722542895 | https://api.github.com/repos/simonw/sqlite-utils/issues/195 | 722542895 | MDEyOklzc3VlQ29tbWVudDcyMjU0Mjg5NQ== | 9599 | 2020-11-05T18:01:33Z | 2020-11-05T18:01:33Z | OWNER | Latest test failure: ``` 114 -> assert [("racoons are biting trash pandas", "USA", "bar")] == table.search( 115 "bite", order="rowid" 116 ) 117 118 119 def test_optimize_fts(fresh_db): (Pdb) table.search("bite") [(2, 'racoons are biting trash pandas', 'USA', 'bar', -9.641434262948206e-07)] ``` The problem here is that the `table.search()` method now behaves differently for FTS4 v.s. FTS5 tables. With FTS4 you get back just the table columns. With FTS5 you also get back the `rowid` as the first column and the `rank` score as the last column. This is weird. It also makes me question whether having `.search()` return a list of tuples is the right API design. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
735663855 | |
https://github.com/simonw/sqlite-utils/pull/195#issuecomment-721397665 | https://api.github.com/repos/simonw/sqlite-utils/issues/195 | 721397665 | MDEyOklzc3VlQ29tbWVudDcyMTM5NzY2NQ== | 9599 | 2020-11-03T22:02:57Z | 2020-11-03T22:02:57Z | OWNER | Documentation so far: https://github.com/simonw/sqlite-utils/blob/6cadc6103ff1ba58c6409ce7fba74259e72965d9/docs/cli.rst#executing-searches | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
735663855 | |
https://github.com/simonw/sqlite-utils/pull/189#issuecomment-717361487 | https://api.github.com/repos/simonw/sqlite-utils/issues/189 | 717361487 | MDEyOklzc3VlQ29tbWVudDcxNzM2MTQ4Nw== | 9599 | 2020-10-27T16:24:04Z | 2020-10-27T16:24:04Z | OWNER | This is great, thank you very much. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
729818242 | |
https://github.com/simonw/sqlite-utils/pull/189#issuecomment-717359145 | https://api.github.com/repos/simonw/sqlite-utils/issues/189 | 717359145 | MDEyOklzc3VlQ29tbWVudDcxNzM1OTE0NQ== | 35681 | 2020-10-27T16:20:32Z | 2020-10-27T16:20:32Z | CONTRIBUTOR | No problem. I added a test. Let me know if it looks sufficient or if you want me to to tweak something! If you don't mind, would you tag this PR as "hacktoberfest-accepted"? If you do mind, no problem and I'm sorry for asking :) My kiddos like the shirts. | { "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
729818242 | |
https://github.com/simonw/sqlite-utils/pull/189#issuecomment-716756103 | https://api.github.com/repos/simonw/sqlite-utils/issues/189 | 716756103 | MDEyOklzc3VlQ29tbWVudDcxNjc1NjEwMw== | 9599 | 2020-10-26T18:56:19Z | 2020-10-26T18:56:19Z | OWNER | This is a great fix, thanks! If you add a unit test somewhere in here I'll merge the PR: https://github.com/simonw/sqlite-utils/blob/main/tests/test_m2m.py | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
729818242 | |
https://github.com/simonw/sqlite-utils/pull/178#issuecomment-701627158 | https://api.github.com/repos/simonw/sqlite-utils/issues/178 | 701627158 | MDEyOklzc3VlQ29tbWVudDcwMTYyNzE1OA== | 9599 | 2020-09-30T20:29:11Z | 2020-09-30T20:29:11Z | OWNER | Thanks for the fix! | { "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
709043182 | |
https://github.com/simonw/sqlite-utils/pull/174#issuecomment-698400790 | https://api.github.com/repos/simonw/sqlite-utils/issues/174 | 698400790 | MDEyOklzc3VlQ29tbWVudDY5ODQwMDc5MA== | 9599 | 2020-09-24T14:59:50Z | 2020-09-24T14:59:50Z | OWNER | For reusing the lookup table: I'm going to raise an error if a lookup table exists but without the correct columns. The caller can then add those columns and try again. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
707944044 | |
https://github.com/simonw/sqlite-utils/pull/174#issuecomment-698184166 | https://api.github.com/repos/simonw/sqlite-utils/issues/174 | 698184166 | MDEyOklzc3VlQ29tbWVudDY5ODE4NDE2Ng== | 9599 | 2020-09-24T08:01:07Z | 2020-09-24T08:01:07Z | OWNER | I may revert the now unnecessary undocumented tweaks to the `.update()` method made in 66d506587eba9f0715267d6560b97c1fa44cc781 as well. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
707944044 | |
https://github.com/simonw/sqlite-utils/pull/174#issuecomment-698182656 | https://api.github.com/repos/simonw/sqlite-utils/issues/174 | 698182656 | MDEyOklzc3VlQ29tbWVudDY5ODE4MjY1Ng== | 9599 | 2020-09-24T07:58:08Z | 2020-09-24T07:58:08Z | OWNER | The way the lookup table works here differs from the previous implementation. In the previous implementation the usage of `.lookup()` meant that an existing table would be modified to fit the new purpose. That no longer happens in this version. Need to make a design decision about how this should work. It should definitely be possible to use an existing lookup table - imagine a database where several tables have a "Departments" column and we want to extract all of those values out to a single shared "Departments" table. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
707944044 | |
https://github.com/simonw/sqlite-utils/pull/174#issuecomment-698182037 | https://api.github.com/repos/simonw/sqlite-utils/issues/174 | 698182037 | MDEyOklzc3VlQ29tbWVudDY5ODE4MjAzNw== | 9599 | 2020-09-24T07:56:50Z | 2020-09-24T07:56:50Z | OWNER | I could also be a bit smarter about transaction handling. I think it may be possible to run this entire operation in a single transaction now. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
707944044 | |
https://github.com/simonw/sqlite-utils/pull/174#issuecomment-698181478 | https://api.github.com/repos/simonw/sqlite-utils/issues/174 | 698181478 | MDEyOklzc3VlQ29tbWVudDY5ODE4MTQ3OA== | 9599 | 2020-09-24T07:55:45Z | 2020-09-24T07:55:45Z | OWNER | `import functools` is no longer needed. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
707944044 | |
https://github.com/simonw/sqlite-utils/pull/174#issuecomment-698180705 | https://api.github.com/repos/simonw/sqlite-utils/issues/174 | 698180705 | MDEyOklzc3VlQ29tbWVudDY5ODE4MDcwNQ== | 9599 | 2020-09-24T07:54:10Z | 2020-09-24T07:54:10Z | OWNER | After running through the steps in https://simonwillison.net/2020/Sep/23/sqlite-utils-extract/ I get a table that looks like this: <img width="1699" alt="salaries__salaries__683_558_rows" src="https://user-images.githubusercontent.com/9599/94116875-666b8900-fe00-11ea-9e97-2b9ccbfeae29.png"> The foreign key columns are all at the end of the table. It would be nicer if they were arranged in the same order as the columns they replaced. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
707944044 | |
https://github.com/simonw/sqlite-utils/pull/174#issuecomment-698180113 | https://api.github.com/repos/simonw/sqlite-utils/issues/174 | 698180113 | MDEyOklzc3VlQ29tbWVudDY5ODE4MDExMw== | 9599 | 2020-09-24T07:53:03Z | 2020-09-24T07:53:03Z | OWNER | This could do with a little bit more testing - I'm worried there may be column or table name edge cases that are not covered yet. I also need to remove the progress bar code since that no longer makes sense for this implementation. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
707944044 | |
https://github.com/simonw/sqlite-utils/pull/161#issuecomment-696494070 | https://api.github.com/repos/simonw/sqlite-utils/issues/161 | 696494070 | MDEyOklzc3VlQ29tbWVudDY5NjQ5NDA3MA== | 9599 | 2020-09-22T03:48:58Z | 2020-09-22T03:48:58Z | OWNER | One last thing. https://www.sqlite.org/lang_altertable.html#making_other_kinds_of_table_schema_change says that the first step should be: > If foreign key constraints are enabled, disable them using PRAGMA foreign_keys=OFF. And the last steps should be: > If foreign key constraints were originally enabled then run PRAGMA foreign_key_check to verify that the schema change did not break any foreign key constraints. > > Commit the transaction started in step 2. > > If foreign keys constraints were originally enabled, reenable them now. I need to implement that. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
705975133 | |
https://github.com/simonw/sqlite-utils/pull/161#issuecomment-696490851 | https://api.github.com/repos/simonw/sqlite-utils/issues/161 | 696490851 | MDEyOklzc3VlQ29tbWVudDY5NjQ5MDg1MQ== | 9599 | 2020-09-22T03:33:54Z | 2020-09-22T03:33:54Z | OWNER | It would be neat if `.transform(pk=None)` converted a primary key table to a rowid table. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
705975133 | |
https://github.com/simonw/sqlite-utils/pull/161#issuecomment-696488201 | https://api.github.com/repos/simonw/sqlite-utils/issues/161 | 696488201 | MDEyOklzc3VlQ29tbWVudDY5NjQ4ODIwMQ== | 9599 | 2020-09-22T03:21:16Z | 2020-09-22T03:21:16Z | OWNER | Just needs documentation now. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
705975133 | |
https://github.com/simonw/sqlite-utils/pull/161#issuecomment-696485791 | https://api.github.com/repos/simonw/sqlite-utils/issues/161 | 696485791 | MDEyOklzc3VlQ29tbWVudDY5NjQ4NTc5MQ== | 9599 | 2020-09-22T03:10:15Z | 2020-09-22T03:10:15Z | OWNER | Design decision needed on foreign keys: what does the syntax look like for removing an existing foreign key? Since I already have a good implementation of `add_foreign_key()` I'm tempted to only support dropping them. Maybe like this: ```python table.transform(drop_foreign_keys=[("author_id", "author", "id")]) ``` It's a bit crufty but it's such a rare use-case that I think this will be good enough. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
705975133 | |
https://github.com/simonw/sqlite-utils/pull/161#issuecomment-696480925 | https://api.github.com/repos/simonw/sqlite-utils/issues/161 | 696480925 | MDEyOklzc3VlQ29tbWVudDY5NjQ4MDkyNQ== | 9599 | 2020-09-22T02:45:47Z | 2020-09-22T02:45:47Z | OWNER | I'm not going to do `conversions=` because it would be inconsistent with how they work elsewhere. The SQL generated by this function looks like this: INSERT INTO dogs_new_tmp VALUES (a, b) SELECT a, b from dogs; So passing `conversions={"name": "upper(?)"})` wouldn't make sense, since we're not using arguments hence there is no-where for that `?` to go. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
705975133 | |
https://github.com/simonw/sqlite-utils/pull/161#issuecomment-696446658 | https://api.github.com/repos/simonw/sqlite-utils/issues/161 | 696446658 | MDEyOklzc3VlQ29tbWVudDY5NjQ0NjY1OA== | 9599 | 2020-09-22T00:13:55Z | 2020-09-22T00:14:21Z | OWNER | Idea: allow a `conversions=` parameter, as seen on `.insert_all()` and friends, which lets you apply a SQL transformation function as part of the operation. E.g.: ```python table.transform({"age": int}, conversions={"name": "upper(?)"}) ``` https://sqlite-utils.readthedocs.io/en/stable/python-api.html#converting-column-values-using-sql-functions | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
705975133 | |
https://github.com/simonw/sqlite-utils/pull/161#issuecomment-696445766 | https://api.github.com/repos/simonw/sqlite-utils/issues/161 | 696445766 | MDEyOklzc3VlQ29tbWVudDY5NjQ0NTc2Ng== | 9599 | 2020-09-22T00:10:50Z | 2020-09-22T00:11:12Z | OWNER | A less horrible interface might be the following: ```python # Ensure the 'age' column is not null: table.transform(not_null={"age"}) # The 'age' column is not null but I don't want it to be: table.transform(not_null={"age": False}) ``` So if the argument is a set it means "make sure these are all not null" - if the argument is a dictionary it means "set these to be null or not null depending on if their dictionary value is true or false". | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
705975133 | |
https://github.com/simonw/sqlite-utils/pull/161#issuecomment-696444842 | https://api.github.com/repos/simonw/sqlite-utils/issues/161 | 696444842 | MDEyOklzc3VlQ29tbWVudDY5NjQ0NDg0Mg== | 9599 | 2020-09-22T00:07:43Z | 2020-09-22T00:09:05Z | OWNER | Syntax challenge: I could use `.transform(defaults={"age": None})` to indicate that the `age` column should have its default removed, but how would I tell `.transform()` that the `age` column, currently `not null`, should have the `not null` removed from it? I could do this: `.transform(not_not_null={"age"})` - it's a bit gross but it's also kind of funny. I actually like it! | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
705975133 | |
https://github.com/simonw/sqlite-utils/pull/161#issuecomment-696444353 | https://api.github.com/repos/simonw/sqlite-utils/issues/161 | 696444353 | MDEyOklzc3VlQ29tbWVudDY5NjQ0NDM1Mw== | 9599 | 2020-09-22T00:06:12Z | 2020-09-22T00:06:12Z | OWNER | I should support `not_null=` and `default=` arguments to the `.transform()` method because it looks like you can't use `ALTER TABLE` to change those. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
705975133 | |
https://github.com/simonw/sqlite-utils/pull/161#issuecomment-696443845 | https://api.github.com/repos/simonw/sqlite-utils/issues/161 | 696443845 | MDEyOklzc3VlQ29tbWVudDY5NjQ0Mzg0NQ== | 9599 | 2020-09-22T00:04:31Z | 2020-09-22T00:04:44Z | OWNER | Good news: the `.columns` introspection does tell me those things: ``` >>> import sqlite_utils >>> db = sqlite_utils.Database(memory=True) >>> db.create_table("foo", {"id": int, "name": str, "age": int}, defaults={"age": 1}, not_null={"name", "age"}) <Table foo (id, name, age)> >>> db["foo"] <Table foo (id, name, age)> >>> print(db["foo"].schema) CREATE TABLE [foo] ( [id] INTEGER, [name] TEXT NOT NULL, [age] INTEGER NOT NULL DEFAULT 1 ) >>> db["foo"].columns [Column(cid=0, name='id', type='INTEGER', notnull=0, default_value=None, is_pk=0), Column(cid=1, name='name', type='TEXT', notnull=1, default_value=None, is_pk=0), Column(cid=2, name='age', type='INTEGER', notnull=1, default_value='1', is_pk=0)] ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
705975133 | |
https://github.com/simonw/sqlite-utils/pull/161#issuecomment-696443190 | https://api.github.com/repos/simonw/sqlite-utils/issues/161 | 696443190 | MDEyOklzc3VlQ29tbWVudDY5NjQ0MzE5MA== | 9599 | 2020-09-22T00:02:22Z | 2020-09-22T00:02:22Z | OWNER | How would I detect which columns are `not_null` and what their defaults are? I don`t think my introspection logic handles that yet. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
705975133 | |
https://github.com/simonw/sqlite-utils/pull/161#issuecomment-696443042 | https://api.github.com/repos/simonw/sqlite-utils/issues/161 | 696443042 | MDEyOklzc3VlQ29tbWVudDY5NjQ0MzA0Mg== | 9599 | 2020-09-22T00:01:50Z | 2020-09-22T00:01:50Z | OWNER | When you transform a table, it should keep its primary key, foreign keys, not_null and defaults. I don't think it needs to care about `hash_id` or `extracts=` since those don't affect the structure of the table as it is being created - well, `hash_id` does but if we are transforming an existing table we will get the `hash_id` column for free. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
705975133 | |
https://github.com/simonw/sqlite-utils/pull/161#issuecomment-696442621 | https://api.github.com/repos/simonw/sqlite-utils/issues/161 | 696442621 | MDEyOklzc3VlQ29tbWVudDY5NjQ0MjYyMQ== | 9599 | 2020-09-22T00:00:23Z | 2020-09-22T00:00:23Z | OWNER | I still need to figure out what to do about these various other table properties: https://github.com/simonw/sqlite-utils/blob/b34c9b40c206d7a9d7ee57a8c1f198ff1f522735/sqlite_utils/db.py#L775-L787 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
705975133 | |
https://github.com/simonw/sqlite-utils/pull/158#issuecomment-693199392 | https://api.github.com/repos/simonw/sqlite-utils/issues/158 | 693199392 | MDEyOklzc3VlQ29tbWVudDY5MzE5OTM5Mg== | 9599 | 2020-09-16T06:21:29Z | 2020-09-16T06:21:29Z | OWNER | Thanks! | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
697203800 | |
https://github.com/simonw/sqlite-utils/pull/156#issuecomment-689735140 | https://api.github.com/repos/simonw/sqlite-utils/issues/156 | 689735140 | MDEyOklzc3VlQ29tbWVudDY4OTczNTE0MA== | 9599 | 2020-09-09T18:21:06Z | 2020-09-09T18:21:06Z | OWNER | Good spot, thanks. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
697030843 | |
https://github.com/simonw/sqlite-utils/pull/146#issuecomment-689185393 | https://api.github.com/repos/simonw/sqlite-utils/issues/146 | 689185393 | MDEyOklzc3VlQ29tbWVudDY4OTE4NTM5Mw== | 9599 | 2020-09-08T23:17:42Z | 2020-09-08T23:17:42Z | OWNER | That seems like a reasonable approach to me, especially since this is going to be a pretty rare edge-case. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
688668680 | |
https://github.com/simonw/sqlite-utils/pull/146#issuecomment-688573964 | https://api.github.com/repos/simonw/sqlite-utils/issues/146 | 688573964 | MDEyOklzc3VlQ29tbWVudDY4ODU3Mzk2NA== | 96218 | 2020-09-08T01:55:07Z | 2020-09-08T01:55:07Z | CONTRIBUTOR | Okay, I've rewritten this PR to preserve the batching behaviour but still fix #145, and rebased the branch to account for the `db.execute()` api change. It's not terribly sophisticated -- if it attempts to insert a batch which has too many variables, the exception is caught, the batch is split in two and each half is inserted separately, and then it carries on as before with the same `batch_size`. In the edge case where this gets triggered, subsequent batches will all be inserted in two groups too if they continue to have the same number of columns (which is presumably reasonably likely). Do you reckon this is acceptable when set against the awkwardness of recalculating the `batch_size` on the fly? | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
688668680 | |
https://github.com/simonw/sqlite-utils/pull/146#issuecomment-688508510 | https://api.github.com/repos/simonw/sqlite-utils/issues/146 | 688508510 | MDEyOklzc3VlQ29tbWVudDY4ODUwODUxMA== | 9599 | 2020-09-07T20:56:03Z | 2020-09-07T20:56:24Z | OWNER | The problem with this approach is that it requires us to consume the entire iterator before we can start inserting rows into the table - here on line 1052: https://github.com/simonw/sqlite-utils/blob/bb131793feac16bc7181ab997568f941b0220ef2/sqlite_utils/db.py#L1047-L1054 I designed the `.insert_all()` to avoid doing this, because I want to be able to pass it an iterator (or more likely a generator) that could produce potentially millions of records. Doing things one batch of 100 records at a time means that the Python process doesn't need to pull millions of records into memory at once. `db-to-sqlite` is one example of a tool that uses that characteristic, in https://github.com/simonw/db-to-sqlite/blob/63e4ee972f292de13bb11767c0fb64b35339d954/db_to_sqlite/cli.py#L94-L106 So we need to solve this issue without consuming the entire iterator with a `records = list(records)` call. I think one way to do this is to execute each chunk one at a time and watch out for an exception that indicates that we sent too many parameters - then adjust the chunk size down and try again. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
688668680 | |
https://github.com/simonw/sqlite-utils/pull/146#issuecomment-688481317 | https://api.github.com/repos/simonw/sqlite-utils/issues/146 | 688481317 | MDEyOklzc3VlQ29tbWVudDY4ODQ4MTMxNw== | 96218 | 2020-09-07T19:18:55Z | 2020-09-07T19:18:55Z | CONTRIBUTOR | Just force-pushed to update d042f9c with more formatting changes to satisfy `black==20.8b1` and pass the GitHub Actions "Test" workflow. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
688668680 | |
https://github.com/simonw/sqlite-utils/pull/146#issuecomment-688479163 | https://api.github.com/repos/simonw/sqlite-utils/issues/146 | 688479163 | MDEyOklzc3VlQ29tbWVudDY4ODQ3OTE2Mw== | 96218 | 2020-09-07T19:10:33Z | 2020-09-07T19:11:57Z | CONTRIBUTOR | @simonw -- I've gone ahead updated the documentation to reflect the changes introduced in this PR. IMO it's ready to merge now. In writing the documentation changes, I begin to wonder about the value and role of `batch_size` at all, tbh. May I assume it was originally intended to prevent using the entire row set to determine columns and column types, and that this was a performance consideration? If so, this PR entirely undermines its purpose. I've been passing in excess of 500,000 rows at a time to `insert_all()` with these changes and although I'm sure the performance difference is measurable it's not really noticeable; given #145, I don't know that any performance advantages outweigh the problems doing it this way removes. What do you think about just dropping the argument and defaulting to the maximum `batch_size` permissible given `SQLITE_MAX_VARS`? Are there other reasons one might want to restrict `batch_size` that I've overlooked? I could open a new issue to discuss/implement this. Of course the documentation will need to change again too if/when something is done about #147. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
688668680 | |
https://github.com/simonw/sqlite-utils/pull/142#issuecomment-683173375 | https://api.github.com/repos/simonw/sqlite-utils/issues/142 | 683173375 | MDEyOklzc3VlQ29tbWVudDY4MzE3MzM3NQ== | 9599 | 2020-08-28T22:29:02Z | 2020-08-28T22:29:02Z | OWNER | Yeah I think that failure is actually because there's a brand new release of Black out and it subtly changes some of the formatting rules. I'll merge this and then run Black against the entire codebase. | { "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
688386219 | |
https://github.com/simonw/sqlite-utils/pull/142#issuecomment-683172829 | https://api.github.com/repos/simonw/sqlite-utils/issues/142 | 683172829 | MDEyOklzc3VlQ29tbWVudDY4MzE3MjgyOQ== | 9599 | 2020-08-28T22:27:05Z | 2020-08-28T22:27:05Z | OWNER | Looks like it failed the "black" formatting test - possibly because there's a new release if black out. I'm going to merge despite that failure. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
688386219 | |
https://github.com/simonw/sqlite-utils/pull/142#issuecomment-683172082 | https://api.github.com/repos/simonw/sqlite-utils/issues/142 | 683172082 | MDEyOklzc3VlQ29tbWVudDY4MzE3MjA4Mg== | 9599 | 2020-08-28T22:24:25Z | 2020-08-28T22:24:25Z | OWNER | Thanks very much! | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
688386219 | |
https://github.com/simonw/sqlite-utils/pull/120#issuecomment-655289686 | https://api.github.com/repos/simonw/sqlite-utils/issues/120 | 655289686 | MDEyOklzc3VlQ29tbWVudDY1NTI4OTY4Ng== | 9599 | 2020-07-08T05:13:11Z | 2020-07-08T05:13:11Z | OWNER | This is an excellent fix, thanks! | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
652816158 | |
https://github.com/simonw/sqlite-utils/pull/118#issuecomment-655653292 | https://api.github.com/repos/simonw/sqlite-utils/issues/118 | 655653292 | MDEyOklzc3VlQ29tbWVudDY1NTY1MzI5Mg== | 9599 | 2020-07-08T17:26:02Z | 2020-07-08T17:26:02Z | OWNER | Awesome, thank you very much. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
651844316 | |
https://github.com/simonw/sqlite-utils/pull/118#issuecomment-655643078 | https://api.github.com/repos/simonw/sqlite-utils/issues/118 | 655643078 | MDEyOklzc3VlQ29tbWVudDY1NTY0MzA3OA== | 79913 | 2020-07-08T17:05:59Z | 2020-07-08T17:05:59Z | CONTRIBUTOR | > The only thing missing from this PR is updates to the documentation. Ah, yes, thanks for this reminder! I've repushed with doc bits added. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
651844316 | |
https://github.com/simonw/sqlite-utils/pull/118#issuecomment-655286864 | https://api.github.com/repos/simonw/sqlite-utils/issues/118 | 655286864 | MDEyOklzc3VlQ29tbWVudDY1NTI4Njg2NA== | 9599 | 2020-07-08T05:05:27Z | 2020-07-08T05:05:36Z | OWNER | The only thing missing from this PR is updates to the documentation. Those need to go in two places: - In the Python API docs. I suggest adding a note to this section about bulk inserts: https://github.com/simonw/sqlite-utils/blob/d0cdaaaf00249230e847be3a3b393ee2689fbfe4/docs/python-api.rst#bulk-inserts - In the CLI docs, in this section: https://github.com/simonw/sqlite-utils/blob/d0cdaaaf00249230e847be3a3b393ee2689fbfe4/docs/cli.rst#inserting-json-data Here's an example of a previous commit that includes updates to both CLI and API documentation: https://github.com/simonw/sqlite-utils/commit/f9473ace14878212c1fa968b7bd2f51e4f064dba#diff-e3e2a9bfd88566b05001b02a3f51d286 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
651844316 | |
https://github.com/simonw/sqlite-utils/pull/118#issuecomment-655284168 | https://api.github.com/repos/simonw/sqlite-utils/issues/118 | 655284168 | MDEyOklzc3VlQ29tbWVudDY1NTI4NDE2OA== | 9599 | 2020-07-08T04:58:00Z | 2020-07-08T04:58:00Z | OWNER | Oops didn't mean to click "close" there. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
651844316 | |
https://github.com/simonw/sqlite-utils/pull/118#issuecomment-655284054 | https://api.github.com/repos/simonw/sqlite-utils/issues/118 | 655284054 | MDEyOklzc3VlQ29tbWVudDY1NTI4NDA1NA== | 9599 | 2020-07-08T04:57:38Z | 2020-07-08T04:57:38Z | OWNER | Thoughts on transactions would be much appreciated in #121 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
651844316 | |
https://github.com/simonw/sqlite-utils/pull/118#issuecomment-655283393 | https://api.github.com/repos/simonw/sqlite-utils/issues/118 | 655283393 | MDEyOklzc3VlQ29tbWVudDY1NTI4MzM5Mw== | 9599 | 2020-07-08T04:55:18Z | 2020-07-08T04:55:18Z | OWNER | This is a really good idea - and thank you for the detailed discussion in the pull request. I'm keen to discuss how transactions can work better. I tend to use this pattern in my own code: with db.conn: db["table"].insert(...) But it's not documented and I've not though very hard about it! I like having inserts that handle 10,000+ rows commit on every chunk so I can watch their progress from another process, but the library should absolutely support people who want to commit all of the rows in a single transaction - or combine changes with DML. Lots to discuss here. I'll start a new issue. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
651844316 | |
https://github.com/simonw/sqlite-utils/pull/118#issuecomment-655239728 | https://api.github.com/repos/simonw/sqlite-utils/issues/118 | 655239728 | MDEyOklzc3VlQ29tbWVudDY1NTIzOTcyOA== | 79913 | 2020-07-08T02:16:42Z | 2020-07-08T02:16:42Z | CONTRIBUTOR | I fixed my original oops by moving the `DELETE FROM $table` out of the chunking loop and repushed. I think this change can be considered in isolation from issues around transactions, which I discuss next. I wanted to make the DELETE + INSERT happen all in the same transaction so it was robust, but that was more complicated than I expected. The transaction handling in the Database/Table classes isn't systematic, and this poses big hurdles to making `Table.insert_all` (or other operations) consistent and robust in the face of errors. For example, I wanted to do this (whitespace ignored in diff, so indentation change not highlighted): ```diff diff --git a/sqlite_utils/db.py b/sqlite_utils/db.py index d6b9ecf..4107ceb 100644 --- a/sqlite_utils/db.py +++ b/sqlite_utils/db.py @@ -1028,6 +1028,11 @@ class Table(Queryable): batch_size = max(1, min(batch_size, SQLITE_MAX_VARS // num_columns)) self.last_rowid = None self.last_pk = None + with self.db.conn: + # Explicit BEGIN is necessary because Python's sqlite3 doesn't + # issue implicit BEGINs for DDL, only DML. We mix DDL and DML + # below and might execute DDL first, e.g. for table creation. + self.db.conn.execute("BEGIN") if truncate and self.exists(): self.db.conn.execute("DELETE FROM [{}];".format(self.name)) for chunk in chunks(itertools.chain([first_record], records), batch_size): @@ -1038,7 +1043,11 @@ class Table(Queryable): # Use the first batch to derive the table names column_types = suggest_column_types(chunk) column_types.update(columns or {}) - self.create( + # Not self.create() because that is wrapped in its own + # transaction and Python's sqlite3 doesn't support + # nested transactions. + self.db.create_table( + … | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
651844316 | |
https://github.com/simonw/sqlite-utils/pull/118#issuecomment-655052451 | https://api.github.com/repos/simonw/sqlite-utils/issues/118 | 655052451 | MDEyOklzc3VlQ29tbWVudDY1NTA1MjQ1MQ== | 79913 | 2020-07-07T18:45:23Z | 2020-07-07T18:45:23Z | CONTRIBUTOR | Ah, I see the problem. The truncate is inside a loop I didn't realize was there. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
651844316 | |
https://github.com/simonw/sqlite-utils/pull/118#issuecomment-655018966 | https://api.github.com/repos/simonw/sqlite-utils/issues/118 | 655018966 | MDEyOklzc3VlQ29tbWVudDY1NTAxODk2Ng== | 79913 | 2020-07-07T17:41:06Z | 2020-07-07T17:41:06Z | CONTRIBUTOR | Hmm, while tests pass, this may not work as intended on larger datasets. Looking into it. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
651844316 | |
https://github.com/simonw/sqlite-utils/issues/99#issuecomment-612727814 | https://api.github.com/repos/simonw/sqlite-utils/issues/99 | 612727814 | MDEyOklzc3VlQ29tbWVudDYxMjcyNzgxNA== | 9599 | 2020-04-13T03:05:04Z | 2020-04-13T03:05:04Z | OWNER | Bit trick from an implementation point of view this, since we want to be able to handle input that is a generator - so we can't scan through the input to validate that every dictionary has the same exact keys without consuming the entire iterator. The alternative would be to raise an error the first time we spot a dictionary with keys that differ... but that's weird because we commit changes in batches, so we may end up only applying half of the changes before exiting with the error. On that basis, I'm going to leave this as-is and mark this as wontfix. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
598640234 | |
https://github.com/simonw/sqlite-utils/issues/99#issuecomment-612727400 | https://api.github.com/repos/simonw/sqlite-utils/issues/99 | 612727400 | MDEyOklzc3VlQ29tbWVudDYxMjcyNzQwMA== | 9599 | 2020-04-13T03:03:09Z | 2020-04-13T03:03:09Z | OWNER | I think I'm going to leave this as intended behaviour. Or maybe passing multiple dictionaries to `.upsert_all()` with different numbers of keys should raise an error? | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
598640234 | |
https://github.com/simonw/sqlite-utils/issues/98#issuecomment-928790381 | https://api.github.com/repos/simonw/sqlite-utils/issues/98 | 928790381 | IC_kwDOCGYnMM43XDdt | 36834097 | 2021-09-28T04:38:44Z | 2021-09-28T04:38:44Z | NONE | Hi @simonw - wondering if you might be able to shed some light here. I've seemed to reproduce this issue. Here's the stacktrace: ``` ... db["potholes"].insert(pothole, pk='id', alter=True, replace=True) ... Traceback (most recent call last): File "<stdin>", line 3, in <module> File "/Users/patricktrainer/.pyenv/versions/3.9.0/lib/python3.9/site-packages/sqlite_utils/db.py", line 2481, in insert return self.insert_all( File "/Users/patricktrainer/.pyenv/versions/3.9.0/lib/python3.9/site-packages/sqlite_utils/db.py", line 2596, in insert_all self.insert_chunk( File "/Users/patricktrainer/.pyenv/versions/3.9.0/lib/python3.9/site-packages/sqlite_utils/db.py", line 2424, in insert_chunk row = list(self.rows_where("rowid = ?", [self.last_rowid]))[0] IndexError: list index out of range ``` Interesting enough, I found that omitting the `pk` param does not throw the error. Let me know how I can help out! | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
597671518 | |
https://github.com/simonw/sqlite-utils/issues/98#issuecomment-612728047 | https://api.github.com/repos/simonw/sqlite-utils/issues/98 | 612728047 | MDEyOklzc3VlQ29tbWVudDYxMjcyODA0Nw== | 9599 | 2020-04-13T03:06:10Z | 2020-04-13T03:06:10Z | OWNER | Implementation plan: `.insert_all()` and `.upsert_all()` should only set `.last_rowid` and `last_pk` if they were called with a single item. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
597671518 | |
https://github.com/simonw/sqlite-utils/issues/98#issuecomment-612708274 | https://api.github.com/repos/simonw/sqlite-utils/issues/98 | 612708274 | MDEyOklzc3VlQ29tbWVudDYxMjcwODI3NA== | 9599 | 2020-04-13T01:25:59Z | 2020-04-13T01:26:11Z | OWNER | In mucking around with `sqlite3` it looks like `result.lastrowid` is indeed populated for `UPDATE` - in this case with the last inserted rowid in the table. This differs from the documented behaviour I linked to above. ``` In [1]: import sqlite3 In [2]: c = sqlite3.connect(":memory:") In [3]: c Out[3]: <sqlite3.Connection at 0x103c4d490> In [4]: c.execute('create table foo (bar integer);') Out[4]: <sqlite3.Cursor at 0x103f760a0> In [5]: c.execute('insert into foo (bar) values (1)') Out[5]: <sqlite3.Cursor at 0x103f76650> In [6]: c.execute('select * from foo').fetchall() Out[6]: [(1,)] In [7]: c.execute('insert into foo (bar) values (1)') Out[7]: <sqlite3.Cursor at 0x103f766c0> In [8]: c.execute('select * from foo').fetchall() Out[8]: [(1,), (1,)] In [9]: c.execute('insert into foo (bar) values (1)').lastrowid Out[9]: 3 In [10]: c.execute('select * from foo').fetchall() Out[10]: [(1,), (1,), (1,)] In [11]: c.execute('select rowid, bar from foo').fetchall() Out[11]: [(1, 1), (2, 1), (3, 1)] In [12]: c.execute('insert into foo (bar) values (1)').lastrowid Out[12]: 4 In [13]: c.execute('select rowid, bar from foo').fetchall() Out[13]: [(1, 1), (2, 1), (3, 1), (4, 1)] In [14]: r = c.execute('update foo set bar =2 where rowid = 1') In [15]: r.lastrowid Out[15]: 4 In [16]: c.execute('select rowid, bar from foo').fetchall() Out[16]: [(1, 2), (2, 1), (3, 1), (4, 1)] In [17]: r = c.execute('select rowid, bar from foo') In… | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
597671518 | |
https://github.com/simonw/sqlite-utils/issues/98#issuecomment-612707828 | https://api.github.com/repos/simonw/sqlite-utils/issues/98 | 612707828 | MDEyOklzc3VlQ29tbWVudDYxMjcwNzgyOA== | 9599 | 2020-04-13T01:24:05Z | 2020-04-13T01:24:16Z | OWNER | Why do I even care about `lastrowid` here? I'm trying to ensure that after you insert or upsert a row you can use `table.last_pk` to start doing things like building additional foreign key relationships. So maybe it doesn't make sense to make `.last_pk` available _at all_ for cases where you called `.upsert_all()` or `.insert_all()` - it should just be populated for `.upsert()` and `.insert()`. The documentation doesn't say it should work for `.upsert_all()` - it's only documented for the single actions. https://github.com/simonw/sqlite-utils/blob/6161ebf4de44411b3f33feeacaf4501e803d1116/sqlite_utils/db.py#L1113-L1124 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
597671518 | |
https://github.com/simonw/sqlite-utils/issues/98#issuecomment-612707293 | https://api.github.com/repos/simonw/sqlite-utils/issues/98 | 612707293 | MDEyOklzc3VlQ29tbWVudDYxMjcwNzI5Mw== | 9599 | 2020-04-13T01:21:22Z | 2020-04-13T01:21:22Z | OWNER | I have a hunch that the root of the problem here is that accessing `result.lastrowid` during my version of an `.upsert()` doesn't actually make sense: https://github.com/simonw/sqlite-utils/blob/6161ebf4de44411b3f33feeacaf4501e803d1116/sqlite_utils/db.py#L1102-L1113 In the bug I'm seeing (which I still haven't reduced to a reproducible test) the debugger shows me this at that point: ``` (Pdb) query 'UPDATE [files] SET [createdAt] = ?, [ext] = ?, [updatedAt] = ?, [uri] = ?, [uriType] = ? WHERE [project] = ? AND [name] = ?' (Pdb) params ['2020-03-04T04:04:40.152000+00:00', 'csv', '2020-03-04T04:04:40.152000+00:00', 'https://storage.googleapis.com/bln_prod/...', 'download', 'UHJvamVjdDo4MTgyMjU2Ny01ZjI0LTQxM2ItYWZmNi05NTlmNGY3MjExMjI=', 'loans_to_documentation.csv'] (Pdb) result.lastrowid 100 ``` But here's the weird thing... there's no row in the table with a rowid of 100! ``` (Pdb) [r['rowid'] for r in self.db.execute_returning_dicts('select rowid, * from files')] [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99] ``` So what the heck is going on? The last SQL statement I executed here was an `UPDATE`. The `lastrowid` docs say: https://kite.com/python/docs/sqlite3.Cursor.lastrowid > This read-only attribute provides the rowid of the last modified row. It is only set if you issued a INSERT statement using the execute() method. For operations other than INSERT or when executemany() is called, lastrowid is set to None. So where did that `100` come from? It should be `None`! | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
597671518 | |
https://github.com/simonw/sqlite-utils/issues/98#issuecomment-612258687 | https://api.github.com/repos/simonw/sqlite-utils/issues/98 | 612258687 | MDEyOklzc3VlQ29tbWVudDYxMjI1ODY4Nw== | 9599 | 2020-04-10T23:08:48Z | 2020-04-10T23:08:48Z | OWNER | I need a test that reproduces this. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
597671518 | |
https://github.com/simonw/sqlite-utils/issues/98#issuecomment-612173156 | https://api.github.com/repos/simonw/sqlite-utils/issues/98 | 612173156 | MDEyOklzc3VlQ29tbWVudDYxMjE3MzE1Ng== | 9599 | 2020-04-10T19:03:32Z | 2020-04-10T23:08:28Z | OWNER | Investigate this traceback: ``` Traceback (most recent call last): File "fetch_projects.py", line 60, in <module> fetch_projects(db, token) File "fetch_projects.py", line 41, in fetch_projects db["projects"].upsert(project, pk="id") File "/Users/simonw/.local/share/virtualenvs/big-local-datasette-2jT6nJCT/lib/python3.7/site-packages/sqlite_utils/db.py", line 1139, in upsert conversions=conversions, File "/Users/simonw/.local/share/virtualenvs/big-local-datasette-2jT6nJCT/lib/python3.7/site-packages/sqlite_utils/db.py", line 1168, in upsert_all upsert=True, File "/Users/simonw/.local/share/virtualenvs/big-local-datasette-2jT6nJCT/lib/python3.7/site-packages/sqlite_utils/db.py", line 1107, in insert_all row = list(self.rows_where("rowid = ?", [self.last_rowid]))[0] IndexError: list index out of range ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
597671518 | |
https://github.com/simonw/sqlite-utils/issues/97#issuecomment-614073859 | https://api.github.com/repos/simonw/sqlite-utils/issues/97 | 614073859 | MDEyOklzc3VlQ29tbWVudDYxNDA3Mzg1OQ== | 1448859 | 2020-04-15T14:29:30Z | 2020-04-15T14:29:30Z | NONE | Woah! Thanks a lot. Next time I will add a more obvious/explicit "if you like this idea let me know I'd love to work on it to get my feet wet here" :D | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
593751293 | |
https://github.com/simonw/sqlite-utils/issues/97#issuecomment-612738311 | https://api.github.com/repos/simonw/sqlite-utils/issues/97 | 612738311 | MDEyOklzc3VlQ29tbWVudDYxMjczODMxMQ== | 9599 | 2020-04-13T03:55:11Z | 2020-04-13T03:55:11Z | OWNER | Shipped in 2.5 - documentation is here: https://sqlite-utils.readthedocs.io/en/stable/python-api.html#connecting-to-or-creating-a-database | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
593751293 | |
https://github.com/simonw/sqlite-utils/issues/97#issuecomment-612732453 | https://api.github.com/repos/simonw/sqlite-utils/issues/97 | 612732453 | MDEyOklzc3VlQ29tbWVudDYxMjczMjQ1Mw== | 9599 | 2020-04-13T03:26:46Z | 2020-04-13T03:26:46Z | OWNER | I wonder if it should delete an recreate the file or if it would be safer to drop every table instead? Dropping tables gets messy: then you need to drop triggers and views, and you need to run `vacuum` to clean up the space. My worry with deleting and recreating the file is that it could trigger errors in other processes that are currently attached to that database file. But... if you know that's going to be likely, maybe you shouldn't use the `recreate=True` feature? | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
593751293 | |
https://github.com/simonw/sqlite-utils/issues/97#issuecomment-612732129 | https://api.github.com/repos/simonw/sqlite-utils/issues/97 | 612732129 | MDEyOklzc3VlQ29tbWVudDYxMjczMjEyOQ== | 9599 | 2020-04-13T03:25:29Z | 2020-04-13T03:25:29Z | OWNER | Interesting thought. I've run into this myself a lot - many of my scripts intend to create the database from scratch, so I end up running `!rm /tmp/blah.db` in Jupyter and occasionally getting errors if the file doesn't exist. I think adding `recreate=True` could make sense. It could throw an error if you attempt to use it after passing in something other than a path to a file on disk. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
593751293 |