```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",986829194,
https://github.com/simonw/datasette/pull/1453#issuecomment-908337614,https://api.github.com/repos/simonw/datasette/issues/1453,908337614,IC_kwDOBm6k_c42JCHO,22429695,2021-08-30T13:20:26Z,2021-08-30T13:20:26Z,NONE,"# [Codecov](https://codecov.io/gh/simonw/datasette/pull/1453?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report
> Merging [#1453](https://codecov.io/gh/simonw/datasette/pull/1453?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (4f492a7) into [main](https://codecov.io/gh/simonw/datasette/commit/67cbf0ae7243431bf13702e6e3ba466b619c4d6f?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (67cbf0a) will **not change** coverage.
> The diff coverage is `n/a`.
[![Impacted file tree graph](https://codecov.io/gh/simonw/datasette/pull/1453/graphs/tree.svg?width=650&height=150&src=pr&token=eSahVY7kw1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)](https://codecov.io/gh/simonw/datasette/pull/1453?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)
```diff
@@ Coverage Diff @@
## main #1453 +/- ##
=======================================
Coverage 91.83% 91.83%
=======================================
Files 34 34
Lines 4421 4421
=======================================
Hits 4060 4060
Misses 361 361
```
------
[Continue to review full report at Codecov](https://codecov.io/gh/simonw/datasette/pull/1453?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)
> `Δ = absolute
(impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/simonw/datasette/pull/1453?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Last update [67cbf0a...4f492a7](https://codecov.io/gh/simonw/datasette/pull/1453?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",982780906,
https://github.com/simonw/datasette/pull/1448#issuecomment-906414811,https://api.github.com/repos/simonw/datasette/issues/1448,906414811,IC_kwDOBm6k_c42Bsrb,22429695,2021-08-26T13:29:02Z,2021-08-26T13:29:02Z,NONE,"# [Codecov](https://codecov.io/gh/simonw/datasette/pull/1448?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report
> Merging [#1448](https://codecov.io/gh/simonw/datasette/pull/1448?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (a211747) into [main](https://codecov.io/gh/simonw/datasette/commit/a1a33bb5822214be1cebd98cd858b2058d91a4aa?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (a1a33bb) will **not change** coverage.
> The diff coverage is `n/a`.
[![Impacted file tree graph](https://codecov.io/gh/simonw/datasette/pull/1448/graphs/tree.svg?width=650&height=150&src=pr&token=eSahVY7kw1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)](https://codecov.io/gh/simonw/datasette/pull/1448?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)
```diff
@@ Coverage Diff @@
## main #1448 +/- ##
=======================================
Coverage 91.82% 91.82%
=======================================
Files 34 34
Lines 4418 4418
=======================================
Hits 4057 4057
Misses 361 361
```
------
[Continue to review full report at Codecov](https://codecov.io/gh/simonw/datasette/pull/1448?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)
> `Δ = absolute (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/simonw/datasette/pull/1448?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Last update [a1a33bb...a211747](https://codecov.io/gh/simonw/datasette/pull/1448?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",980228553,
https://github.com/dogsheep/dogsheep-photos/issues/7#issuecomment-906015471,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/7,906015471,IC_kwDOD079W842ALLv,18232,2021-08-26T02:01:01Z,2021-08-26T02:01:01Z,NONE,Perceptual hashes might be what you're after : http://phash.org,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",602585497,
https://github.com/simonw/sqlite-utils/pull/321#issuecomment-905022931,https://api.github.com/repos/simonw/sqlite-utils/issues/321,905022931,IC_kwDOCGYnMM418Y3T,22429695,2021-08-24T22:38:38Z,2021-08-24T23:27:26Z,NONE,"# [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/321?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report
> Merging [#321](https://codecov.io/gh/simonw/sqlite-utils/pull/321?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (db2dd6d) into [main](https://codecov.io/gh/simonw/sqlite-utils/commit/9258f4bd8450c951900de998a7bf81ca9b45a014?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (9258f4b) will **increase** coverage by `0.16%`.
> The diff coverage is `100.00%`.
[![Impacted file tree graph](https://codecov.io/gh/simonw/sqlite-utils/pull/321/graphs/tree.svg?width=650&height=150&src=pr&token=O0X3703L9P&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)](https://codecov.io/gh/simonw/sqlite-utils/pull/321?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)
```diff
@@ Coverage Diff @@
## main #321 +/- ##
==========================================
+ Coverage 96.41% 96.58% +0.16%
==========================================
Files 5 5
Lines 2206 2223 +17
==========================================
+ Hits 2127 2147 +20
+ Misses 79 76 -3
```
| [Impacted Files](https://codecov.io/gh/simonw/sqlite-utils/pull/321?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage Δ | |
|---|---|---|
| [sqlite\_utils/cli.py](https://codecov.io/gh/simonw/sqlite-utils/pull/321/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-c3FsaXRlX3V0aWxzL2NsaS5weQ==) | `95.69% <100.00%> (+0.42%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/321?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)
> `Δ = absolute (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/321?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Last update [9258f4b...db2dd6d](https://codecov.io/gh/simonw/sqlite-utils/pull/321?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",978537855,
https://github.com/simonw/sqlite-utils/issues/319#issuecomment-905024066,https://api.github.com/repos/simonw/sqlite-utils/issues/319,905024066,IC_kwDOCGYnMM418ZJC,66709385,2021-08-24T22:41:39Z,2021-08-24T22:41:39Z,NONE,"I'm happy with this functionality left the way you describe. In my case the data is homogeneous but other cases would work just by being consistent on the encoding. Thanks a lot, Simon!","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",976399638,
https://github.com/simonw/sqlite-utils/issues/319#issuecomment-905021010,https://api.github.com/repos/simonw/sqlite-utils/issues/319,905021010,IC_kwDOCGYnMM418YZS,66709385,2021-08-24T22:33:42Z,2021-08-24T22:33:42Z,NONE,"Oh, I misread. Yes some files will not be valid UTF-8, I'd throw a warning and continue (not adding that file) but if you want to get more elaborate you could allow to define a policy on what to do. Not adding the file, index binary content or use a conversion policy like the ones available on Python's decode.
From https://stackoverflow.com/questions/24616678/unicodedecodeerror-in-python-when-reading-a-file-how-to-ignore-the-error-and-ju :
- 'ignore' ignores errors. Note that ignoring encoding errors can lead to data loss.
- 'replace' causes a replacement marker (such as '?') to be inserted where there is malformed data.
- 'surrogateescape' will represent any incorrect bytes as code points in the Unicode Private Use Area ranging from U+DC80 to U+DCFF. These private code points will then be turned back into the same bytes when the surrogateescape error handler is used when writing data. This is useful for processing files in an unknown encoding.
- 'xmlcharrefreplace' is only supported when writing to a file. Characters not supported by the encoding are replaced with the appropriate XML character reference nnn;.
- 'backslashreplace' (also only supported when writing) replaces unsupported characters with Python’s backslashed escape sequences.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",976399638,
https://github.com/simonw/sqlite-utils/issues/319#issuecomment-905003381,https://api.github.com/repos/simonw/sqlite-utils/issues/319,905003381,IC_kwDOCGYnMM418UF1,66709385,2021-08-24T21:56:49Z,2021-08-24T21:56:49Z,NONE,"I was thinking that an approach could be making FILE_COLUMNS a generator (_get_file_columns(mode)) or you can just have a different set of columns (is there something else that makes sense to be changed on the text scenario?).
About UTF-8 I was referring to the encoding to use when reading files. This can be difficult to auto-detect but I believe that UTF-8 is pretty much the standard for text files.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",976399638,
https://github.com/dogsheep/healthkit-to-sqlite/pull/13#issuecomment-904642396,https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/13,904642396,IC_kwDOC8tyDs41679c,32016596,2021-08-24T13:27:40Z,2021-08-24T13:28:26Z,NONE,This would fix #21 and make #22 obsolete.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",743071410,
https://github.com/dogsheep/healthkit-to-sqlite/pull/22#issuecomment-904641261,https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/22,904641261,IC_kwDOC8tyDs4167rt,32016596,2021-08-24T13:26:20Z,2021-08-24T13:26:20Z,NONE,Did not see that #13 fixes the same issue in a similar way. You can decide which one to merge ;),"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",978086284,
https://github.com/dogsheep/healthkit-to-sqlite/issues/21#issuecomment-903950096,https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/21,903950096,IC_kwDOC8tyDs414S8Q,32016596,2021-08-23T17:00:59Z,2021-08-23T17:00:59Z,NONE,"I think the issue is that I have records like these:
```xml
```
And if sqlite is case insensitive, then `metadata_meal` and `metadata_Meal` result in the same column.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",977128935,
https://github.com/simonw/sqlite-utils/pull/247#issuecomment-901344634,https://api.github.com/repos/simonw/sqlite-utils/issues/247,901344634,IC_kwDOCGYnMM41uW16,22429695,2021-08-18T18:42:54Z,2021-08-18T18:42:54Z,NONE,"# [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/247?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report
> Merging [#247](https://codecov.io/gh/simonw/sqlite-utils/pull/247?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (af989af) into [main](https://codecov.io/gh/simonw/sqlite-utils/commit/1fe73c898b44695052f1a9ca832818d50cecf662?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (1fe73c8) will **decrease** coverage by `0.03%`.
> The diff coverage is `85.71%`.
[![Impacted file tree graph](https://codecov.io/gh/simonw/sqlite-utils/pull/247/graphs/tree.svg?width=650&height=150&src=pr&token=O0X3703L9P&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)](https://codecov.io/gh/simonw/sqlite-utils/pull/247?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)
```diff
@@ Coverage Diff @@
## main #247 +/- ##
==========================================
- Coverage 96.28% 96.24% -0.04%
==========================================
Files 5 5
Lines 2179 2186 +7
==========================================
+ Hits 2098 2104 +6
- Misses 81 82 +1
```
| [Impacted Files](https://codecov.io/gh/simonw/sqlite-utils/pull/247?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage Δ | |
|---|---|---|
| [sqlite\_utils/db.py](https://codecov.io/gh/simonw/sqlite-utils/pull/247/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-c3FsaXRlX3V0aWxzL2RiLnB5) | `97.84% <85.71%> (-0.08%)` | :arrow_down: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/247?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)
> `Δ = absolute (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/247?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Last update [1fe73c8...af989af](https://codecov.io/gh/simonw/sqlite-utils/pull/247?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",832687563,
https://github.com/simonw/datasette/pull/1433#issuecomment-898450402,https://api.github.com/repos/simonw/datasette/issues/1433,898450402,IC_kwDOBm6k_c41jUPi,22429695,2021-08-13T13:15:55Z,2021-08-13T13:15:55Z,NONE,"# [Codecov](https://codecov.io/gh/simonw/datasette/pull/1433?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report
> Merging [#1433](https://codecov.io/gh/simonw/datasette/pull/1433?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (ddba6cc) into [main](https://codecov.io/gh/simonw/datasette/commit/2883098770fc66e50183b2b231edbde20848d4d6?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (2883098) will **not change** coverage.
> The diff coverage is `n/a`.
[![Impacted file tree graph](https://codecov.io/gh/simonw/datasette/pull/1433/graphs/tree.svg?width=650&height=150&src=pr&token=eSahVY7kw1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)](https://codecov.io/gh/simonw/datasette/pull/1433?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)
```diff
@@ Coverage Diff @@
## main #1433 +/- ##
=======================================
Coverage 91.82% 91.82%
=======================================
Files 34 34
Lines 4418 4418
=======================================
Hits 4057 4057
Misses 361 361
```
------
[Continue to review full report at Codecov](https://codecov.io/gh/simonw/datasette/pull/1433?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)
> `Δ = absolute (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/simonw/datasette/pull/1433?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Last update [2883098...ddba6cc](https://codecov.io/gh/simonw/datasette/pull/1433?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",970386262,
https://github.com/simonw/datasette/pull/1430#issuecomment-898043575,https://api.github.com/repos/simonw/datasette/issues/1430,898043575,IC_kwDOBm6k_c41hw63,22429695,2021-08-12T23:39:36Z,2021-08-12T23:49:51Z,NONE,"# [Codecov](https://codecov.io/gh/simonw/datasette/pull/1430?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report
> Merging [#1430](https://codecov.io/gh/simonw/datasette/pull/1430?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (9419947) into [main](https://codecov.io/gh/simonw/datasette/commit/b1fed48a95516ae84c0f020582303ab50ab817e2?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (b1fed48) will **increase** coverage by `0.00%`.
> The diff coverage is `100.00%`.
[![Impacted file tree graph](https://codecov.io/gh/simonw/datasette/pull/1430/graphs/tree.svg?width=650&height=150&src=pr&token=eSahVY7kw1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)](https://codecov.io/gh/simonw/datasette/pull/1430?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)
```diff
@@ Coverage Diff @@
## main #1430 +/- ##
=======================================
Coverage 91.71% 91.71%
=======================================
Files 34 34
Lines 4417 4418 +1
=======================================
+ Hits 4051 4052 +1
Misses 366 366
```
| [Impacted Files](https://codecov.io/gh/simonw/datasette/pull/1430?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage Δ | |
|---|---|---|
| [datasette/views/table.py](https://codecov.io/gh/simonw/datasette/pull/1430/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL3ZpZXdzL3RhYmxlLnB5) | `96.00% <100.00%> (+<0.01%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/simonw/datasette/pull/1430?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)
> `Δ = absolute (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/simonw/datasette/pull/1430?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Last update [b1fed48...9419947](https://codecov.io/gh/simonw/datasette/pull/1430?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",969758038,
https://github.com/simonw/datasette/issues/942#issuecomment-898032118,https://api.github.com/repos/simonw/datasette/issues/942,898032118,IC_kwDOBm6k_c41huH2,596279,2021-08-12T23:12:00Z,2021-08-12T23:12:00Z,NONE,"This looks awesome. We'll definitely make extensive use of this feature!
On Thu, Aug 12, 2021 at 5:52 PM Simon Willison ***@***.***>
wrote:
> I like this. Need to solve for mobile though where the cog menu isn't
> visible - I think I'll do that with a definition list at the top of the
> page.
>
> —
> You are receiving this because you are subscribed to this thread.
> Reply to this email directly, view it on GitHub
> ,
> or unsubscribe
>
> .
> Triage notifications on the go with GitHub Mobile for iOS
>
> or Android
>
> .
>
--
Zane A. Selvans, PhD
Chief Data Wrangler
Catalyst Cooperative
https://catalyst.coop
***@***.***
Signal/WhatsApp/SMS: +1 720 443 1363
Twitter: @ZaneSelvans
PGP : 0x64F7B56F3A127B04
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",681334912,
https://github.com/simonw/sqlite-utils/issues/186#issuecomment-897600677,https://api.github.com/repos/simonw/sqlite-utils/issues/186,897600677,IC_kwDOCGYnMM41gEyl,9308268,2021-08-12T12:32:14Z,2021-08-12T12:32:14Z,NONE,"Actually, I forgot to include the `bib_pub_year` in the extract ...
But also, I tried again with empty string values instead of `NULL` values and it seems to place the foreign key properly / correctly...
```python3
sql = """"""\
INSERT INTO ""circulation_info"" (""item_id"", ""bib_title"", ""bib_creator"", ""bib_format"", ""bib_pub_year"", ""checkout_date"")
VALUES
(1, ""title one"", ""creator one"", ""Book"", 2018, ""2021-08-12 00:01""),
(2, ""title two"", ""creator one"", ""Book"", 2019, ""2021-08-12 00:02""),
(3, ""title three"", """", ""DVD"", 2020, ""2021-08-12 00:03""),
(4, ""title four"", """", ""DVD"", """", ""2021-08-12 00:04""),
(5, ""title five"", """", ""DVD"", """", ""2021-08-12 00:05"")
""""""
with sqlite3.connect('test_bib_2.db') as con:
con.execute(sql)
```
```python3
db[""circulation_info""].extract(
[
""bib_title"",
""bib_creator"",
""bib_format"",
""bib_pub_year""
],
table=""bib_info"",
fk_column=""bib_info_id""
)
```
```
{'id': 1, 'item_id': 1, 'bib_info_id': 1, 'bib_pub_year': 2018, 'checkout_date': '2021-08-12 00:01'}
{'id': 2, 'item_id': 2, 'bib_info_id': 2, 'bib_pub_year': 2019, 'checkout_date': '2021-08-12 00:02'}
{'id': 3, 'item_id': 3, 'bib_info_id': 3, 'bib_pub_year': 2020, 'checkout_date': '2021-08-12 00:03'}
{'id': 4, 'item_id': 4, 'bib_info_id': 4, 'bib_pub_year': '', 'checkout_date': '2021-08-12 00:04'}
{'id': 5, 'item_id': 5, 'bib_info_id': 5, 'bib_pub_year': '', 'checkout_date': '2021-08-12 00:05'}
---
{'id': 1, 'bib_title': 'title one', 'bib_creator': 'creator one', 'bib_format': 'Book'}
{'id': 2, 'bib_title': 'title two', 'bib_creator': 'creator one', 'bib_format': 'Book'}
{'id': 3, 'bib_title': 'title three', 'bib_creator': '', 'bib_format': 'DVD'}
{'id': 4, 'bib_title': 'title four', 'bib_creator': '', 'bib_format': 'DVD'}
{'id': 5, 'bib_title': 'title five', 'bib_creator': '', 'bib_format': 'DVD'}
```
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",722816436,
https://github.com/simonw/sqlite-utils/issues/186#issuecomment-897588624,https://api.github.com/repos/simonw/sqlite-utils/issues/186,897588624,IC_kwDOCGYnMM41gB2Q,9308268,2021-08-12T12:13:25Z,2021-08-12T12:13:25Z,NONE,"I think I ran into an issue that's perhaps related with `extract()`
I have a case where I want to create a lookup table for all the related title data where there are possibly multiple null values in the related columns ....
```python3
sql = """"""\
INSERT INTO ""circulation_info"" (""item_id"", ""bib_title"", ""bib_creator"", ""bib_format"", ""bib_pub_year"", ""checkout_date"")
VALUES
(1, ""title one"", ""creator one"", ""Book"", 2018, ""2021-08-12 00:01""),
(2, ""title two"", ""creator one"", ""Book"", 2019, ""2021-08-12 00:02""),
(3, ""title three"", NULL, ""DVD"", 2020, ""2021-08-12 00:03""),
(4, ""title four"", NULL, ""DVD"", NULL, ""2021-08-12 00:04""),
(5, ""title five"", NULL, ""DVD"", NULL, ""2021-08-12 00:05"")
""""""
with sqlite3.connect('test_bib.db') as con:
con.execute(sql)
```
when I run the `extract()` method ...
```python3
db[""circulation_info""].extract(
[
""bib_title"",
""bib_creator"",
""bib_format""
],
table=""bib_info"",
fk_column=""bib_info_id""
)
db = sqlite_utils.Database(""test_bib.db"")
for row in db[""circulation_info""].rows:
print(row)
print(""\n---\n"")
for row in db[""bib_info""].rows:
print(row)
```
results in this ..
```
{'id': 1, 'item_id': 1, 'bib_info_id': 1, 'bib_pub_year': 2018, 'checkout_date': '2021-08-12 00:01'}
{'id': 2, 'item_id': 2, 'bib_info_id': 2, 'bib_pub_year': 2019, 'checkout_date': '2021-08-12 00:02'}
{'id': 3, 'item_id': 3, 'bib_info_id': None, 'bib_pub_year': 2020, 'checkout_date': '2021-08-12 00:03'}
{'id': 4, 'item_id': 4, 'bib_info_id': None, 'bib_pub_year': None, 'checkout_date': '2021-08-12 00:04'}
{'id': 5, 'item_id': 5, 'bib_info_id': None, 'bib_pub_year': None, 'checkout_date': '2021-08-12 00:05'}
---
{'id': 1, 'bib_title': 'title one', 'bib_creator': 'creator one', 'bib_format': 'Book'}
{'id': 2, 'bib_title': 'title two', 'bib_creator': 'creator one', 'bib_format': 'Book'}
{'id': 3, 'bib_title': 'title three', 'bib_creator': None, 'bib_format': 'DVD'}
{'id': 4, 'bib_title': 'title four', 'bib_creator': None, 'bib_format': 'DVD'}
{'id': 5, 'bib_title': 'title five', 'bib_creator': None, 'bib_format': 'DVD'}
```
Seems like it's correctly generating the row data for those lookups, but it's not correctly updating the foreign key back to the primary table? Looks like it just results in a `NULL` value in that original table.
Any ideas on why? Thanks again!","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",722816436,
https://github.com/dogsheep/google-takeout-to-sqlite/pull/8#issuecomment-896378525,https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/8,896378525,IC_kwDODFE5qs41baad,28565,2021-08-10T23:28:45Z,2021-08-10T23:28:45Z,NONE,"I added parsing of text/html emails using BeautifulSoup.
Around half of the emails in my archive don't include a text/plain payload so adding html parsing makes a good chunk of them searchable.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",954546309,
https://github.com/simonw/sqlite-utils/pull/312#issuecomment-896162082,https://api.github.com/repos/simonw/sqlite-utils/issues/312,896162082,IC_kwDOCGYnMM41alki,22429695,2021-08-10T17:10:39Z,2021-08-10T23:07:35Z,NONE,"# [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/312?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report
> Merging [#312](https://codecov.io/gh/simonw/sqlite-utils/pull/312?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (43bc064) into [main](https://codecov.io/gh/simonw/sqlite-utils/commit/ee469e3122d6f5973ec2584c1580d930daca2e7c?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (ee469e3) will **decrease** coverage by `0.02%`.
> The diff coverage is `96.84%`.
[![Impacted file tree graph](https://codecov.io/gh/simonw/sqlite-utils/pull/312/graphs/tree.svg?width=650&height=150&src=pr&token=O0X3703L9P&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)](https://codecov.io/gh/simonw/sqlite-utils/pull/312?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)
```diff
@@ Coverage Diff @@
## main #312 +/- ##
==========================================
- Coverage 96.30% 96.28% -0.03%
==========================================
Files 5 5
Lines 2168 2179 +11
==========================================
+ Hits 2088 2098 +10
- Misses 80 81 +1
```
| [Impacted Files](https://codecov.io/gh/simonw/sqlite-utils/pull/312?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage Δ | |
|---|---|---|
| [sqlite\_utils/db.py](https://codecov.io/gh/simonw/sqlite-utils/pull/312/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-c3FsaXRlX3V0aWxzL2RiLnB5) | `97.91% <96.84%> (-0.08%)` | :arrow_down: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/312?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)
> `Δ = absolute (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/312?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Last update [ee469e3...43bc064](https://codecov.io/gh/simonw/sqlite-utils/pull/312?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",965143346,
https://github.com/dogsheep/google-takeout-to-sqlite/pull/8#issuecomment-894581223,https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/8,894581223,IC_kwDODFE5qs41Ujnn,28565,2021-08-07T00:57:48Z,2021-08-07T00:57:48Z,NONE,"Just added two more fixes:
* Added parsing for rfc 2047 encoded unicode headers
* Body is now stored as TEXT rather than a BLOB regardless of what order the messages are parsed in.
I was able to run this on my Takeout export and everything seems to work fine. @simonw let me know if this looks good to merge.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",954546309,
https://github.com/simonw/datasette/pull/1418#issuecomment-891987129,https://api.github.com/repos/simonw/datasette/issues/1418,891987129,IC_kwDOBm6k_c41KqS5,22429695,2021-08-03T16:26:01Z,2021-08-03T16:32:20Z,NONE,"# [Codecov](https://codecov.io/gh/simonw/datasette/pull/1418?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report
> Merging [#1418](https://codecov.io/gh/simonw/datasette/pull/1418?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (532170f) into [main](https://codecov.io/gh/simonw/datasette/commit/54b6e96ee8aa553b6671e341a1944f93f3fb89c3?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (54b6e96) will **not change** coverage.
> The diff coverage is `n/a`.
[![Impacted file tree graph](https://codecov.io/gh/simonw/datasette/pull/1418/graphs/tree.svg?width=650&height=150&src=pr&token=eSahVY7kw1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)](https://codecov.io/gh/simonw/datasette/pull/1418?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)
```diff
@@ Coverage Diff @@
## main #1418 +/- ##
=======================================
Coverage 91.64% 91.64%
=======================================
Files 34 34
Lines 4382 4382
=======================================
Hits 4016 4016
Misses 366 366
```
------
[Continue to review full report at Codecov](https://codecov.io/gh/simonw/datasette/pull/1418?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)
> `Δ = absolute (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/simonw/datasette/pull/1418?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Last update [54b6e96...532170f](https://codecov.io/gh/simonw/datasette/pull/1418?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",959284434,
https://github.com/simonw/sqlite-utils/issues/298#issuecomment-891583131,https://api.github.com/repos/simonw/sqlite-utils/issues/298,891583131,IC_kwDOCGYnMM41JHqb,2172260,2021-08-03T06:50:47Z,2021-08-03T06:50:47Z,NONE,@simonw Awesome; thanks so much!,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",951581763,
https://github.com/simonw/sqlite-utils/pull/303#issuecomment-890553014,https://api.github.com/repos/simonw/sqlite-utils/issues/303,890553014,IC_kwDOCGYnMM41FMK2,22429695,2021-08-01T16:53:35Z,2021-08-02T04:45:19Z,NONE,"# [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/303?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report
> Merging [#303](https://codecov.io/gh/simonw/sqlite-utils/pull/303?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (4c3bf97) into [main](https://codecov.io/gh/simonw/sqlite-utils/commit/c7e8d72be9fe8fe0811f685a18eebc637662d41b?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (c7e8d72) will **increase** coverage by `0.24%`.
> The diff coverage is `100.00%`.
[![Impacted file tree graph](https://codecov.io/gh/simonw/sqlite-utils/pull/303/graphs/tree.svg?width=650&height=150&src=pr&token=O0X3703L9P&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)](https://codecov.io/gh/simonw/sqlite-utils/pull/303?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)
```diff
@@ Coverage Diff @@
## main #303 +/- ##
==========================================
+ Coverage 96.04% 96.28% +0.24%
==========================================
Files 4 5 +1
Lines 1998 2129 +131
==========================================
+ Hits 1919 2050 +131
Misses 79 79
```
| [Impacted Files](https://codecov.io/gh/simonw/sqlite-utils/pull/303?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage Δ | |
|---|---|---|
| [sqlite\_utils/cli.py](https://codecov.io/gh/simonw/sqlite-utils/pull/303/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-c3FsaXRlX3V0aWxzL2NsaS5weQ==) | `94.91% <100.00%> (+0.36%)` | :arrow_up: |
| [sqlite\_utils/db.py](https://codecov.io/gh/simonw/sqlite-utils/pull/303/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-c3FsaXRlX3V0aWxzL2RiLnB5) | `97.97% <100.00%> (+0.10%)` | :arrow_up: |
| [sqlite\_utils/recipes.py](https://codecov.io/gh/simonw/sqlite-utils/pull/303/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-c3FsaXRlX3V0aWxzL3JlY2lwZXMucHk=) | `100.00% <100.00%> (ø)` | |
| [sqlite\_utils/utils.py](https://codecov.io/gh/simonw/sqlite-utils/pull/303/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-c3FsaXRlX3V0aWxzL3V0aWxzLnB5) | `92.39% <100.00%> (+0.48%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/303?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)
> `Δ = absolute (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/303?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Last update [c7e8d72...4c3bf97](https://codecov.io/gh/simonw/sqlite-utils/pull/303?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",957536983,
https://github.com/simonw/datasette/issues/1398#issuecomment-889599513,https://api.github.com/repos/simonw/datasette/issues/1398,889599513,IC_kwDOBm6k_c41BjYZ,192984,2021-07-30T03:21:49Z,2021-07-30T03:21:49Z,NONE,Does the library support this now?,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",947044667,
https://github.com/dogsheep/google-takeout-to-sqlite/pull/5#issuecomment-888075098,https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/5,888075098,IC_kwDODFE5qs407vNa,28565,2021-07-28T07:18:56Z,2021-07-28T07:18:56Z,NONE,"> I'm not sure why but my most recent import, when displayed in Datasette, looks like this:
>
>
I did some investigation into this issue and made a fix [here](https://github.com/dogsheep/google-takeout-to-sqlite/pull/8/commits/8ee555c2889a38ff42b95664ee074b4a01a82f06). The problem was that some messages (like gchat logs) don't have a `Message-Id` and we need to use `X-GM-THRID` as the pkey instead.
@simonw While looking into this I found something unexpected about how sqlite_utils handles upserts if the pkey column is `None`. When the pkey is NULL I'd expect the function to either use rowid or throw an exception. Instead, it seems upsert_all creates a row where all columns are NULL instead of using the values provided as parameters.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",813880401,
https://github.com/dogsheep/github-to-sqlite/pull/65#issuecomment-885964242,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/65,885964242,IC_kwDODFdgUs40zr3S,231498,2021-07-23T23:45:35Z,2021-07-23T23:45:35Z,NONE,@simonw is this PR of interest to you?,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",923270900,
https://github.com/dogsheep/google-takeout-to-sqlite/pull/5#issuecomment-885098025,https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/5,885098025,IC_kwDODFE5qs40wYYp,306240,2021-07-22T17:47:50Z,2021-07-22T17:47:50Z,NONE,"Hi @maxhawkins , I'm sorry, I haven't had any time to work on this. I'll have some time tomorrow to test your commits. I think they look great. I'm great with your commits superseding my initial attempt here.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",813880401,
https://github.com/dogsheep/google-takeout-to-sqlite/pull/5#issuecomment-885094284,https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/5,885094284,IC_kwDODFE5qs40wXeM,28565,2021-07-22T17:41:32Z,2021-07-22T17:41:32Z,NONE,I added a follow-up commit that deals with emails that don't have a `Date` header: https://github.com/maxhawkins/google-takeout-to-sqlite/commit/4bc70103582c10802c85a523ef1e99a8a2154aa9,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",813880401,
https://github.com/dogsheep/google-takeout-to-sqlite/pull/5#issuecomment-885022230,https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/5,885022230,IC_kwDODFE5qs40wF4W,28565,2021-07-22T15:51:46Z,2021-07-22T15:51:46Z,NONE,One thing I noticed is this importer doesn't save attachments along with the body of the emails. It would be nice if those got stored as blobs in a separate attachments table so attachments can be included while fetching search results.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",813880401,
https://github.com/dogsheep/google-takeout-to-sqlite/pull/5#issuecomment-884672647,https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/5,884672647,IC_kwDODFE5qs40uwiH,28565,2021-07-22T05:56:31Z,2021-07-22T14:03:08Z,NONE,"How does this commit look? https://github.com/maxhawkins/google-takeout-to-sqlite/commit/72802a83fee282eb5d02d388567731ba4301050d
It seems that Takeout's mbox format is pretty simple, so we can get away with just splitting the file on lines begining with `From `. My commit just splits the file every time a line starts with `From ` and uses `email.message_from_bytes` to parse each chunk.
I was able to load a 12GB takeout mbox without the program using more than a couple hundred MB of memory during the import process. It does make us lose the progress bar, but maybe I can add that back in a later commit.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",813880401,
https://github.com/dogsheep/dogsheep-photos/issues/32#issuecomment-884688833,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/32,884688833,IC_kwDOD079W840u0fB,10793464,2021-07-22T06:40:25Z,2021-07-22T06:40:25Z,NONE,The solution here is to upload an image to the bucket first. It is caused because it does not properly handle the case when there are no images in the bucket.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",803333769,
https://github.com/simonw/datasette/pull/1296#issuecomment-817403642,https://api.github.com/repos/simonw/datasette/issues/1296,817403642,MDEyOklzc3VlQ29tbWVudDgxNzQwMzY0Mg==,22429695,2021-04-12T00:29:05Z,2021-07-20T08:52:12Z,NONE,"# [Codecov](https://codecov.io/gh/simonw/datasette/pull/1296?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report
> Merging [#1296](https://codecov.io/gh/simonw/datasette/pull/1296?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (527a056) into [main](https://codecov.io/gh/simonw/datasette/commit/c73af5dd72305f6a01ea94a2c76d52e5e26de38b?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (c73af5d) will **decrease** coverage by `0.11%`.
> The diff coverage is `n/a`.
> :exclamation: Current head 527a056 differs from pull request most recent head 8f00c31. Consider uploading reports for the commit 8f00c31 to get more accurate results
[![Impacted file tree graph](https://codecov.io/gh/simonw/datasette/pull/1296/graphs/tree.svg?width=650&height=150&src=pr&token=eSahVY7kw1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)](https://codecov.io/gh/simonw/datasette/pull/1296?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)
```diff
@@ Coverage Diff @@
## main #1296 +/- ##
==========================================
- Coverage 91.62% 91.51% -0.12%
==========================================
Files 34 34
Lines 4371 4255 -116
==========================================
- Hits 4005 3894 -111
+ Misses 366 361 -5
```
| [Impacted Files](https://codecov.io/gh/simonw/datasette/pull/1296?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage Δ | |
|---|---|---|
| [datasette/tracer.py](https://codecov.io/gh/simonw/datasette/pull/1296/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL3RyYWNlci5weQ==) | `81.60% <0.00%> (-1.35%)` | :arrow_down: |
| [datasette/views/base.py](https://codecov.io/gh/simonw/datasette/pull/1296/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL3ZpZXdzL2Jhc2UucHk=) | `95.01% <0.00%> (-0.42%)` | :arrow_down: |
| [datasette/facets.py](https://codecov.io/gh/simonw/datasette/pull/1296/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL2ZhY2V0cy5weQ==) | `89.04% <0.00%> (-0.41%)` | :arrow_down: |
| [datasette/utils/\_\_init\_\_.py](https://codecov.io/gh/simonw/datasette/pull/1296/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL3V0aWxzL19faW5pdF9fLnB5) | `94.13% <0.00%> (-0.21%)` | :arrow_down: |
| [datasette/renderer.py](https://codecov.io/gh/simonw/datasette/pull/1296/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL3JlbmRlcmVyLnB5) | `94.02% <0.00%> (-0.18%)` | :arrow_down: |
| [datasette/views/database.py](https://codecov.io/gh/simonw/datasette/pull/1296/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL3ZpZXdzL2RhdGFiYXNlLnB5) | `97.19% <0.00%> (-0.10%)` | :arrow_down: |
| [datasette/views/table.py](https://codecov.io/gh/simonw/datasette/pull/1296/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL3ZpZXdzL3RhYmxlLnB5) | `95.88% <0.00%> (-0.07%)` | :arrow_down: |
| [datasette/views/index.py](https://codecov.io/gh/simonw/datasette/pull/1296/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL3ZpZXdzL2luZGV4LnB5) | `96.36% <0.00%> (-0.07%)` | :arrow_down: |
| [datasette/hookspecs.py](https://codecov.io/gh/simonw/datasette/pull/1296/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL2hvb2tzcGVjcy5weQ==) | `100.00% <0.00%> (ø)` | |
| [datasette/utils/testing.py](https://codecov.io/gh/simonw/datasette/pull/1296/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL3V0aWxzL3Rlc3RpbmcucHk=) | `95.38% <0.00%> (ø)` | |
| ... and [5 more](https://codecov.io/gh/simonw/datasette/pull/1296/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/simonw/datasette/pull/1296?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)
> `Δ = absolute (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/simonw/datasette/pull/1296?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Last update [c73af5d...8f00c31](https://codecov.io/gh/simonw/datasette/pull/1296?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",855446829,
https://github.com/simonw/datasette/pull/1400#issuecomment-882542519,https://api.github.com/repos/simonw/datasette/issues/1400,882542519,IC_kwDOBm6k_c40moe3,22429695,2021-07-19T13:20:52Z,2021-07-19T13:20:52Z,NONE,"# [Codecov](https://codecov.io/gh/simonw/datasette/pull/1400?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report
> Merging [#1400](https://codecov.io/gh/simonw/datasette/pull/1400?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (e95c685) into [main](https://codecov.io/gh/simonw/datasette/commit/c73af5dd72305f6a01ea94a2c76d52e5e26de38b?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (c73af5d) will **not change** coverage.
> The diff coverage is `n/a`.
[![Impacted file tree graph](https://codecov.io/gh/simonw/datasette/pull/1400/graphs/tree.svg?width=650&height=150&src=pr&token=eSahVY7kw1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)](https://codecov.io/gh/simonw/datasette/pull/1400?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)
```diff
@@ Coverage Diff @@
## main #1400 +/- ##
=======================================
Coverage 91.62% 91.62%
=======================================
Files 34 34
Lines 4371 4371
=======================================
Hits 4005 4005
Misses 366 366
```
------
[Continue to review full report at Codecov](https://codecov.io/gh/simonw/datasette/pull/1400?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)
> `Δ = absolute (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/simonw/datasette/pull/1400?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Last update [c73af5d...e95c685](https://codecov.io/gh/simonw/datasette/pull/1400?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",947640902,
https://github.com/simonw/datasette/issues/123#issuecomment-882096402,https://api.github.com/repos/simonw/datasette/issues/123,882096402,IC_kwDOBm6k_c40k7kS,921217,2021-07-18T18:07:29Z,2021-07-18T18:07:29Z,NONE,"I also love the idea for this feature and wonder if it could work without having to download the whole database into memory at once if it's a rather large db. Obviously this could be slower but could have many use cases.
My comment is partially inspired by this post about streaming sqlite dbs from github pages or such
https://news.ycombinator.com/item?id=27016630
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",275125561,