html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,issue,performed_via_github_app
https://github.com/simonw/datasette/issues/1836#issuecomment-1271004167,https://api.github.com/repos/simonw/datasette/issues/1836,1271004167,IC_kwDOBm6k_c5LwfwH,9599,2022-10-07T01:53:05Z,2022-10-07T01:53:05Z,OWNER,"Oh this is interesting!
Is your hunch here that running this line is causing the file to be stored as a second layer?
https://github.com/simonw/datasette/blob/5aa359b86907d11b3ee601510775a85a90224da8/datasette/utils/__init__.py#L399
I guess it's possible that running a non-read-only query against the database causes one or two bytes to be changed (maybe a transaction ID or similar?)
Modifying the `inspect` command to use `?mode=ro` seems sensible to me. Except.... it should already be opening those files in immutable mode according to this line: https://github.com/simonw/datasette/blob/eff112498ecc499323c26612d707908831446d25/datasette/cli.py#L172-L173
Here's what opening as a `immutables` does:
https://github.com/simonw/datasette/blob/eff112498ecc499323c26612d707908831446d25/datasette/app.py#L258-L260
https://github.com/simonw/datasette/blob/eff112498ecc499323c26612d707908831446d25/datasette/database.py#L98
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1400374908,
https://github.com/simonw/datasette/issues/1836#issuecomment-1271006020,https://api.github.com/repos/simonw/datasette/issues/1836,1271006020,IC_kwDOBm6k_c5LwgNE,9599,2022-10-07T01:54:07Z,2022-10-07T01:54:07Z,OWNER,Just overlapped with your comment here: https://github.com/simonw/datasette/issues/1836#issuecomment-1271003212 - which notes that opening with `?immutable=1` DOES seem to cause the file to be duplicated!,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1400374908,
https://github.com/simonw/datasette/pull/1838#issuecomment-1271009214,https://api.github.com/repos/simonw/datasette/issues/1838,1271009214,IC_kwDOBm6k_c5Lwg--,9599,2022-10-07T02:01:07Z,2022-10-07T02:01:07Z,OWNER,"The argument that has always convinced me NOT to use `target=""_blank""` (even for links like this one) is that it breaks browser expectations.
If you click a link with `target=""_blank"" on it you get a new browser window... with a disabled back button. You have to then know to close that browser window in order to return to the previous page - as opposed to hitting the ""back"" button like usual.
You'll note that Datasette doesn't use `target=""_blank""` even on URLs presented in database tables - like these ones: https://latest.datasette.io/fixtures/roadside_attractions
So I'm very firmly in the anti-target-blank camp!
This is the kind of change which I'd suggest implementing as a plugin. `datasette-external-links-new-windows` could run a bit of JavaScript on every page that looks for `` elements that link to off-domain pages and adds `target=""_blank""` to them via the DOM.
That way people who like `target=""_blank""` can have it!
","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1400494162,
https://github.com/simonw/datasette/pull/1838#issuecomment-1271803298,https://api.github.com/repos/simonw/datasette/issues/1838,1271803298,IC_kwDOBm6k_c5Lzi2i,9599,2022-10-07T16:28:41Z,2022-10-07T16:28:41Z,OWNER,... and here's @ocdtrekkie's plugin! https://github.com/ocdtrekkie/datasette-external-links-new-tabs,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1400494162,
https://github.com/simonw/datasette/issues/1646#issuecomment-1272149176,https://api.github.com/repos/simonw/datasette/issues/1646,1272149176,IC_kwDOBm6k_c5L03S4,9599,2022-10-07T23:06:17Z,2022-10-07T23:06:17Z,OWNER,Updated documentation: https://docs.datasette.io/en/latest/settings.html#configuration-directory-mode,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1157182254,
https://github.com/simonw/datasette/issues/1836#issuecomment-1270923537,https://api.github.com/repos/simonw/datasette/issues/1836,1270923537,IC_kwDOBm6k_c5LwMER,536941,2022-10-07T00:46:08Z,2022-10-07T00:46:08Z,CONTRIBUTOR,"i thought it was maybe to do with reading through all the files, but that does not seem to be the case
if i make a little test file like:
```python
# test_read.py
import hashlib
import sys
import pathlib
HASH_BLOCK_SIZE = 1024 * 1024
def inspect_hash(path):
""""""Calculate the hash of a database, efficiently.""""""
m = hashlib.sha256()
with path.open(""rb"") as fp:
while True:
data = fp.read(HASH_BLOCK_SIZE)
if not data:
break
m.update(data)
return m.hexdigest()
inspect_hash(pathlib.Path(sys.argv[1]))
```
then a line in the Dockerfile like
```docker
RUN python test_read.py nlrb.db && echo ""[]"" > /etc/inspect.json
```
just produes a layer of `3B`
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1400374908,
https://github.com/simonw/datasette/issues/1836#issuecomment-1270936982,https://api.github.com/repos/simonw/datasette/issues/1836,1270936982,IC_kwDOBm6k_c5LwPWW,536941,2022-10-07T00:52:41Z,2022-10-07T00:52:41Z,CONTRIBUTOR,"it's not that the inspect command is somehow changing the db files. if i set them to only read-only, the ""inspect"" layer still has the same very large size.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1400374908,
https://github.com/simonw/datasette/issues/1836#issuecomment-1270988081,https://api.github.com/repos/simonw/datasette/issues/1836,1270988081,IC_kwDOBm6k_c5Lwb0x,536941,2022-10-07T01:19:01Z,2022-10-07T01:27:35Z,CONTRIBUTOR,"okay, some progress!! running some sql against a database file causes that file to get duplicated even if it doesn't apparently change the file.
make a little test script like this:
```python
# test_sql.py
import sqlite3
import sys
db_name = sys.argv[1]
conn = sqlite3.connect(f'file:/app/{db_name}', uri=True)
cur = conn.cursor()
cur.execute('select count(*) from filing')
print(cur.fetchone())
```
then
```docker
RUN python test_sql.py nlrb.db
```
produced a layer that's the same size as `nlrb.db`!!
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1400374908,
https://github.com/simonw/datasette/issues/1836#issuecomment-1270992795,https://api.github.com/repos/simonw/datasette/issues/1836,1270992795,IC_kwDOBm6k_c5Lwc-b,536941,2022-10-07T01:29:15Z,2022-10-07T01:50:14Z,CONTRIBUTOR,"fascinatingly, telling python to open sqlite in read only mode makes this layer have a size of 0
```python
# test_sql_ro.py
import sqlite3
import sys
db_name = sys.argv[1]
conn = sqlite3.connect(f'file:/app/{db_name}?mode=ro', uri=True)
cur = conn.cursor()
cur.execute('select count(*) from filing')
print(cur.fetchone())
```
that's quite weird because setting the file permissions to read only didn't do anything. (on reflection, that chmod isn't doing anything because the dockerfile commands are run as root)","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1400374908,
https://github.com/simonw/datasette/issues/1836#issuecomment-1271003212,https://api.github.com/repos/simonw/datasette/issues/1836,1271003212,IC_kwDOBm6k_c5LwfhM,536941,2022-10-07T01:52:04Z,2022-10-07T01:52:04Z,CONTRIBUTOR,"and if we try immutable mode, which is how things are opened by `datasette inspect` we duplicate the files!!!
```python
# test_sql_immutable.py
import sqlite3
import sys
db_name = sys.argv[1]
conn = sqlite3.connect(f'file:/app/{db_name}?immutable=1', uri=True)
cur = conn.cursor()
cur.execute('select count(*) from filing')
print(cur.fetchone())
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1400374908,
https://github.com/simonw/datasette/issues/1836#issuecomment-1271008997,https://api.github.com/repos/simonw/datasette/issues/1836,1271008997,IC_kwDOBm6k_c5Lwg7l,536941,2022-10-07T02:00:37Z,2022-10-07T02:00:49Z,CONTRIBUTOR,"yes, and i also think that this is causing the apparent memory problems in #1480. when the container starts up, it will make some operation on the database in `immutable` mode which apparently makes some small change to the db file. if that's so, then the db files will be copied to the read/write layer which counts against cloudrun's memory allocation!
running a test of that now. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1400374908,
https://github.com/simonw/datasette/issues/1836#issuecomment-1271020193,https://api.github.com/repos/simonw/datasette/issues/1836,1271020193,IC_kwDOBm6k_c5Lwjqh,536941,2022-10-07T02:15:05Z,2022-10-07T02:21:08Z,CONTRIBUTOR,"when i hack the connect method to open non mutable files with ""mode=ro"" and not ""immutable=1"" https://github.com/simonw/datasette/blob/eff112498ecc499323c26612d707908831446d25/datasette/database.py#L79
then:
```bash
870 B RUN /bin/sh -c datasette inspect nlrb.db --inspect-file inspect-data.json
```
the `datasette inspect` layer is only the size of the json file!","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1400374908,
https://github.com/simonw/datasette/issues/1301#issuecomment-1271035998,https://api.github.com/repos/simonw/datasette/issues/1301,1271035998,IC_kwDOBm6k_c5Lwnhe,536941,2022-10-07T02:38:04Z,2022-10-07T02:38:04Z,CONTRIBUTOR,the only mode that `publish cloudrun` supports right now is immutable,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",860722711,
https://github.com/simonw/datasette/issues/1836#issuecomment-1271100651,https://api.github.com/repos/simonw/datasette/issues/1836,1271100651,IC_kwDOBm6k_c5Lw3Tr,536941,2022-10-07T04:38:14Z,2022-10-07T04:38:14Z,CONTRIBUTOR,"> yes, and i also think that this is causing the apparent memory problems in #1480. when the container starts up, it will make some operation on the database in `immutable` mode which apparently makes some small change to the db file. if that's so, then the db files will be copied to the read/write layer which counts against cloudrun's memory allocation!
>
> running a test of that now.
this completely addressed #1480 ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1400374908,
https://github.com/simonw/datasette/issues/1480#issuecomment-1271101072,https://api.github.com/repos/simonw/datasette/issues/1480,1271101072,IC_kwDOBm6k_c5Lw3aQ,536941,2022-10-07T04:39:10Z,2022-10-07T04:39:10Z,CONTRIBUTOR,switching from `immutable=1` to `mode=ro` completely addressed this. see https://github.com/simonw/datasette/issues/1836#issuecomment-1271100651 for details.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1015646369,
https://github.com/simonw/datasette/issues/1836#issuecomment-1271103097,https://api.github.com/repos/simonw/datasette/issues/1836,1271103097,IC_kwDOBm6k_c5Lw355,536941,2022-10-07T04:43:41Z,2022-10-07T04:43:41Z,CONTRIBUTOR,"@simonw, should i open up a new issue for investigating the differences between ""immutable=1"" and ""mode=ro"" and possibly switching to ""mode=ro"". Or would you like to keep that conversation in this issue?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1400374908,
https://github.com/simonw/datasette/pull/1838#issuecomment-1271024708,https://api.github.com/repos/simonw/datasette/issues/1838,1271024708,IC_kwDOBm6k_c5LwkxE,4399499,2022-10-07T02:19:49Z,2022-10-07T02:19:49Z,NONE,"Ooh, I didn't even think about links in tables! You're definitely right on the approach to this. It might also be a really good ""stupidly simple"" plugin for me to try to build myself, which could be fun.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1400494162,
https://github.com/simonw/datasette/pull/1820#issuecomment-1258601033,https://api.github.com/repos/simonw/datasette/issues/1820,1258601033,IC_kwDOBm6k_c5LBLpJ,22429695,2022-09-26T20:32:47Z,2022-10-07T03:58:13Z,NONE,"# [Codecov](https://codecov.io/gh/simonw/datasette/pull/1820?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report
Base: **92.50**% // Head: **92.51**% // Increases project coverage by **`+0.01%`** :tada:
> Coverage data is based on head [(`9bead2a`)](https://codecov.io/gh/simonw/datasette/pull/1820?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) compared to base [(`eff1124`)](https://codecov.io/gh/simonw/datasette/commit/eff112498ecc499323c26612d707908831446d25?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).
> Patch coverage: 100.00% of modified lines in pull request are covered.
Additional details and impacted files
```diff
@@ Coverage Diff @@
## main #1820 +/- ##
==========================================
+ Coverage 92.50% 92.51% +0.01%
==========================================
Files 35 35
Lines 4400 4406 +6
==========================================
+ Hits 4070 4076 +6
Misses 330 330
```
| [Impacted Files](https://codecov.io/gh/simonw/datasette/pull/1820?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage Δ | |
|---|---|---|
| [datasette/app.py](https://codecov.io/gh/simonw/datasette/pull/1820/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL2FwcC5weQ==) | `94.11% <ø> (ø)` | |
| [datasette/views/base.py](https://codecov.io/gh/simonw/datasette/pull/1820/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL3ZpZXdzL2Jhc2UucHk=) | `94.80% <100.00%> (+0.05%)` | :arrow_up: |
| [datasette/views/database.py](https://codecov.io/gh/simonw/datasette/pull/1820/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL3ZpZXdzL2RhdGFiYXNlLnB5) | `95.29% <100.00%> (+0.06%)` | :arrow_up: |
Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)
[:umbrella: View full report at Codecov](https://codecov.io/gh/simonw/datasette/pull/1820?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).
:loudspeaker: Do you have feedback about the report comment? [Let us know in this issue](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1386456717,
https://github.com/simonw/datasette/pull/1837#issuecomment-1270855853,https://api.github.com/repos/simonw/datasette/issues/1837,1270855853,IC_kwDOBm6k_c5Lv7it,22429695,2022-10-07T00:01:20Z,2022-10-07T00:01:20Z,NONE,"# [Codecov](https://codecov.io/gh/simonw/datasette/pull/1837?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report
Base: **92.50**% // Head: **92.50**% // No change to project coverage :thumbsup:
> Coverage data is based on head [(`c12447e`)](https://codecov.io/gh/simonw/datasette/pull/1837?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) compared to base [(`eff1124`)](https://codecov.io/gh/simonw/datasette/commit/eff112498ecc499323c26612d707908831446d25?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).
> Patch has no changes to coverable lines.
Additional details and impacted files
```diff
@@ Coverage Diff @@
## main #1837 +/- ##
=======================================
Coverage 92.50% 92.50%
=======================================
Files 35 35
Lines 4400 4400
=======================================
Hits 4070 4070
Misses 330 330
```
Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)
[:umbrella: View full report at Codecov](https://codecov.io/gh/simonw/datasette/pull/1837?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).
:loudspeaker: Do you have feedback about the report comment? [Let us know in this issue](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1400431789,