html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,issue,performed_via_github_app
https://github.com/simonw/datasette/pull/2191#issuecomment-1724480716,https://api.github.com/repos/simonw/datasette/issues/2191,1724480716,IC_kwDOBm6k_c5myXzM,22429695,2023-09-18T21:28:36Z,2023-10-12T16:15:40Z,NONE,"## [Codecov](https://app.codecov.io/gh/simonw/datasette/pull/2191?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report
All modified lines are covered by tests :white_check_mark:
> Comparison is base [(`6ed7908`)](https://app.codecov.io/gh/simonw/datasette/commit/6ed7908580fa2ba9297c3225d85c56f8b08b9937?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) 92.69% compared to head [(`0135e7c`)](https://app.codecov.io/gh/simonw/datasette/pull/2191?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) 92.68%.
> Report is 14 commits behind head on main.
> :exclamation: Current head 0135e7c differs from pull request most recent head 18b48f8. Consider uploading reports for the commit 18b48f8 to get more accurate results
Additional details and impacted files
```diff
@@ Coverage Diff @@
## main #2191 +/- ##
==========================================
- Coverage 92.69% 92.68% -0.02%
==========================================
Files 40 40
Lines 6039 6042 +3
==========================================
+ Hits 5598 5600 +2
- Misses 441 442 +1
```
| [Files](https://app.codecov.io/gh/simonw/datasette/pull/2191?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage Δ | |
|---|---|---|
| [datasette/app.py](https://app.codecov.io/gh/simonw/datasette/pull/2191?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL2FwcC5weQ==) | `94.09% <100.00%> (-0.11%)` | :arrow_down: |
| [datasette/default\_permissions.py](https://app.codecov.io/gh/simonw/datasette/pull/2191?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL2RlZmF1bHRfcGVybWlzc2lvbnMucHk=) | `97.36% <100.00%> (+0.01%)` | :arrow_up: |
... and [3 files with indirect coverage changes](https://app.codecov.io/gh/simonw/datasette/pull/2191/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/simonw/datasette/pull/2191?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1901768721,
https://github.com/simonw/datasette/pull/2190#issuecomment-1759947021,https://api.github.com/repos/simonw/datasette/issues/2190,1759947021,IC_kwDOBm6k_c5o5qkN,9599,2023-10-12T16:19:38Z,2023-10-12T16:19:38Z,OWNER,"
This looks good and works well. The error from this currently looks like:
```
datasette -m metadata.json -p 8844
Traceback (most recent call last):
File ""/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/bin/datasette"", line 33, in
sys.exit(load_entry_point('datasette', 'console_scripts', 'datasette')())
File ""/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/click/core.py"", line 1130, in __call__
return self.main(*args, **kwargs)
File ""/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/click/core.py"", line 1055, in main
rv = self.invoke(ctx)
File ""/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/click/core.py"", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File ""/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/click/core.py"", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File ""/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/click/core.py"", line 760, in invoke
return __callback(*args, **kwargs)
File ""/Users/simon/Dropbox/Development/datasette/datasette/cli.py"", line 98, in wrapped
return fn(*args, **kwargs)
File ""/Users/simon/Dropbox/Development/datasette/datasette/cli.py"", line 546, in serve
metadata_data = fail_if_plugins_in_metadata(parse_metadata(metadata.read()))
File ""/Users/simon/Dropbox/Development/datasette/datasette/utils/__init__.py"", line 1282, in fail_if_plugins_in_metadata
raise Exception(
Exception: Datasette no longer accepts plugin configuration in --metadata. Move your ""plugins"" configuration blocks to a separate file - we suggest calling that datasette..json - and start Datasette with datasette -c datasette..json. See https://docs.datasette.io/en/latest/configuration.html for more details.
```
With wrapping:
`Exception: Datasette no longer accepts plugin configuration in --metadata. Move your ""plugins"" configuration blocks to a separate file - we suggest calling that datasette..json - and start Datasette with datasette -c datasette..json. See https://docs.datasette.io/en/latest/configuration.html for more details.`
I think we should link directly to documentation that tells people how to perform this upgrade.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1901483874,
https://github.com/simonw/datasette/pull/2190#issuecomment-1759947534,https://api.github.com/repos/simonw/datasette/issues/2190,1759947534,IC_kwDOBm6k_c5o5qsO,9599,2023-10-12T16:19:59Z,2023-10-12T16:19:59Z,OWNER,It would be nice if we could catch that and turn that into a less intimidating Click exception too.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1901483874,
https://github.com/simonw/datasette/pull/2190#issuecomment-1759948683,https://api.github.com/repos/simonw/datasette/issues/2190,1759948683,IC_kwDOBm6k_c5o5q-L,9599,2023-10-12T16:20:41Z,2023-10-12T16:20:41Z,OWNER,I'm going to land this and open a new issue for the upgrade instructions.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1901483874,
https://github.com/simonw/datasette/issues/2199#issuecomment-1759952247,https://api.github.com/repos/simonw/datasette/issues/2199,1759952247,IC_kwDOBm6k_c5o5r13,9599,2023-10-12T16:23:10Z,2023-10-12T16:23:10Z,OWNER,"Some options for where this could go:
- Directly in the release notes? I'm not sure about that, those are getting pretty long already. I think the release notes should link to relevant upgrade guides.
- On a new page? We could have a ""upgrade instructions"" page in the documentation.
- At the bottom of the new https://docs.datasette.io/en/latest/configuration.html page
I'm leaning towards the third option at the moment.
But... we may also need to provide upgrade instructions for plugin authors. Those could live in a separate area of the documentation though, since issues affecting end-users who configure Datasette and issues affecting plugin authors are unlikely to overlap much.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1940346034,
https://github.com/simonw/datasette/issues/2199#issuecomment-1760396195,https://api.github.com/repos/simonw/datasette/issues/2199,1760396195,IC_kwDOBm6k_c5o7YOj,9599,2023-10-12T21:36:25Z,2023-10-12T21:36:25Z,OWNER,"Related idea: how about a `datasette-upgrade` plugin which adds a `datasette upgrade` command that can be used to automate this process?
Maybe something like this:
```bash
datasette install datasette-upgrade
datasette upgrade metadata-to-config metadata.json
```
This would output two new files: `metadata.yaml` and `datasette.yaml`. If files with those names existed already in the current directory they would be called `metadata-new.yaml` and `datasette-new.yaml`.
The command would tell you what it did:
```
Your metadata.json file has been rewritten as two files:
metadata-new.yaml
datasette.yaml
Start Datasette like this to try them out:
datasette -m metadata-new.yaml -c datasette.yaml
```
The command is `datasette upgrade metadata-to-config` because `metadata-to-config` is the name of the upgrade recipe. The first version of the plugin would only have that single recipe, but we could add more recipes in the future for other upgrades.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1940346034,
https://github.com/simonw/datasette/issues/2199#issuecomment-1760401731,https://api.github.com/repos/simonw/datasette/issues/2199,1760401731,IC_kwDOBm6k_c5o7ZlD,15178711,2023-10-12T21:41:42Z,2023-10-12T21:41:42Z,CONTRIBUTOR,"I dig it - I was thinking an Observable notebook where you paste your `metadata.json`/`metadata.yaml` and it would generate the new metadata + datasette.yaml files, but an extensible `datasette upgrade` plugin would be nice for future plugins.
One thing to think about: If someone has comments in their original `metadata.yaml`, could we preserve them in the new files? tbh maybe not too important bc if people cared that much they could just copy + paste, and it might be too distracting
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1940346034,
https://github.com/simonw/datasette/issues/2199#issuecomment-1760411937,https://api.github.com/repos/simonw/datasette/issues/2199,1760411937,IC_kwDOBm6k_c5o7cEh,9599,2023-10-12T21:51:16Z,2023-10-12T21:51:16Z,OWNER,"I think I'm OK with not preserving comments, just because it adds a level of complexity to the tool which I don't think is worth the value it provides.
If people want to keep their comments I'm happy to leave them to copy those over by hand.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1940346034,
https://github.com/simonw/datasette/issues/2199#issuecomment-1760412424,https://api.github.com/repos/simonw/datasette/issues/2199,1760412424,IC_kwDOBm6k_c5o7cMI,9599,2023-10-12T21:51:44Z,2023-10-12T21:51:44Z,OWNER,"Started playing with this plugin idea, now tearing myself away to work on something more important:
```python
from datasette import hookimpl
import click
import pathlib
@hookimpl
def register_commands(cli):
@cli.group()
def upgrade():
""""""
Apply configuration upgrades to an existing Datasette instance
""""""
pass
@upgrade.command()
@click.argument(
""metadata"", type=click.Path(exists=True)
)
@click.option(
""new_metadata"", ""-m"", ""--new-metadata"", help=""Path to new metadata.yaml file"", type=click.Path(exists=False)
)
@click.option(
""new_datasette"", ""-c"", ""--new-datasette"", help=""Path to new datasette.yaml file"", type=click.Path(exists=False)
)
@click.option(
""output_dir"", ""-e"", ""--output-dir"", help=""Directory to write new files to"", type=click.Path(), default="".""
)
def metadata_to_config(metadata, new_metadata, new_datasette, output_dir):
""""""
Upgrade an existing metadata.json/yaml file to the new metadata.yaml and
datasette.yaml split introduced prior to Datasette 1.0.
""""""
print(""Upgrading {} to new metadata.yaml format"".format(metadata))
output_dir = pathlib.Path(output_dir)
if not new_metadata:
# Pick a filename for the new metadata.yaml file that does not yet exist
new_metadata = pick_filename(""metadata"", output_dir)
if not new_datasette:
new_datasette = pick_filename(""datasette"", output_dir)
print(""New metadata.yaml file will be written to {}"".format(new_metadata))
print(""New datasette.yaml file will be written to {}"".format(new_datasette))
def pick_filename(base, output_dir):
options = [""{}.yaml"".format(base), ""{}-new.yaml"".format(base)]
i = 0
while True:
option = options.pop(0)
option_path = output_dir / option
if not option_path.exists():
return option_path
# If we ran out
if not options:
i += 1
options = [""{}-new-{}.yaml"".format(base, i)]
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1940346034,
https://github.com/simonw/datasette/issues/2199#issuecomment-1760413191,https://api.github.com/repos/simonw/datasette/issues/2199,1760413191,IC_kwDOBm6k_c5o7cYH,9599,2023-10-12T21:52:25Z,2023-10-12T21:52:25Z,OWNER,"Demo of that logic:
```
$ datasette upgrade metadata-to-config ../datasette/metadata.json
Upgrading ../datasette/metadata.json to new metadata.yaml format
New metadata.yaml file will be written to metadata-new-1.yaml
New datasette.yaml file will be written to datasette.yaml
$ touch metadata-new-1.yaml
$ datasette upgrade metadata-to-config ../datasette/metadata.json
Upgrading ../datasette/metadata.json to new metadata.yaml format
New metadata.yaml file will be written to metadata-new-2.yaml
New datasette.yaml file will be written to datasette.yaml
$ touch datasette.yaml
$ datasette upgrade metadata-to-config ../datasette/metadata.json
Upgrading ../datasette/metadata.json to new metadata.yaml format
New metadata.yaml file will be written to metadata-new-2.yaml
New datasette.yaml file will be written to datasette-new.yaml
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1940346034,
https://github.com/simonw/datasette/issues/2196#issuecomment-1760417555,https://api.github.com/repos/simonw/datasette/issues/2196,1760417555,IC_kwDOBm6k_c5o7dcT,9599,2023-10-12T21:54:52Z,2023-10-12T21:54:52Z,OWNER,"I can't replicate this bug. Closing, but please re-open if it's still happening.
As an aside, the link I promote is https://datasette.io/discord which redirects:
```
curl -i 'https://datasette.io/discord'
HTTP/2 301
location: https://discord.gg/ktd74dm5mw
content-type: text/plain
x-cloud-trace-context: 8dcfd08d3d1fa44f7ee78568e0f5305e;o=1
date: Thu, 12 Oct 2023 21:54:17 GMT
server: Google Frontend
content-length: 0
```
```
curl -i 'https://discord.gg/ktd74dm5mw'
HTTP/2 301
date: Thu, 12 Oct 2023 21:54:28 GMT
content-type: text/plain;charset=UTF-8
content-length: 0
location: https://discord.com/invite/ktd74dm5mw
strict-transport-security: max-age=31536000; includeSubDomains; preload
permissions-policy: interest-cohort=()
x-content-type-options: nosniff
x-frame-options: DENY
x-robots-tag: noindex, nofollow, noarchive, nocache, noimageindex, noodp
x-xss-protection: 1; mode=block
report-to: {""endpoints"":[{""url"":""https:\/\/a.nel.cloudflare.com\/report\/v3?s=Dzzrf%2FgGkfFxtzSAQ46slMVDLcFjsH9fsvVkzHtgUUiZ891rXAa6LvTRpHK%2BdSMSQ54F57hS9z1mZXXklIbONZW1bfBuFjSK9J4XmjjLjsFUulMXvpjfCLkB6PI%3D""}],""group"":""cf-nel"",""max_age"":604800}
nel: {""success_fraction"":0,""report_to"":""cf-nel"",""max_age"":604800}
server: cloudflare
cf-ray: 815294ddff282511-SJC
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1910269679,
https://github.com/simonw/datasette/issues/2199#issuecomment-1760441535,https://api.github.com/repos/simonw/datasette/issues/2199,1760441535,IC_kwDOBm6k_c5o7jS_,9599,2023-10-12T22:08:42Z,2023-10-12T22:08:42Z,OWNER,Pushed that incomplete code here: https://github.com/datasette/datasette-upgrade,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1940346034,
https://github.com/simonw/datasette/pull/2052#issuecomment-1760542865,https://api.github.com/repos/simonw/datasette/issues/2052,1760542865,IC_kwDOBm6k_c5o78CR,9599,2023-10-12T23:44:53Z,2023-10-12T23:45:15Z,OWNER,"Weird, the `cog` check is failing in CI.
```
Run cog --check docs/*.rst
cog --check docs/*.rst
shell: /usr/bin/bash -e {0}
env:
pythonLocation: /opt/hostedtoolcache/Python/3.9.18/x64
PKG_CONFIG_PATH: /opt/hostedtoolcache/Python/3.9.18/x64/lib/pkgconfig
Python_ROOT_DIR: /opt/hostedtoolcache/Python/3.9.18/x64
Python2_ROOT_DIR: /opt/hostedtoolcache/Python/3.9.18/x64
Python3_ROOT_DIR: /opt/hostedtoolcache/Python/3.9.18/x64
LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.9.18/x64/lib
Check failed
Checking docs/authentication.rst
Checking docs/binary_data.rst
Checking docs/changelog.rst
Checking docs/cli-reference.rst
Checking docs/configuration.rst (changed)
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1651082214,
https://github.com/simonw/datasette/pull/2052#issuecomment-1760545012,https://api.github.com/repos/simonw/datasette/issues/2052,1760545012,IC_kwDOBm6k_c5o78j0,9599,2023-10-12T23:48:16Z,2023-10-12T23:48:16Z,OWNER,Oh! I think I broke Cog on `main` and these tests are running against this branch rebased against main.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1651082214,
https://github.com/simonw/datasette/pull/2052#issuecomment-1632867333,https://api.github.com/repos/simonw/datasette/issues/2052,1632867333,IC_kwDOBm6k_c5hU5QF,22429695,2023-07-12T16:38:27Z,2023-10-12T23:52:24Z,NONE,"## [Codecov](https://app.codecov.io/gh/simonw/datasette/pull/2052?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report
All modified lines are covered by tests :white_check_mark:
> Comparison is base [(`3feed1f`)](https://app.codecov.io/gh/simonw/datasette/commit/3feed1f66e2b746f349ee56970a62246a18bb164?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) 92.46% compared to head [(`8ae479c`)](https://app.codecov.io/gh/simonw/datasette/pull/2052?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) 92.69%.
> Report is 112 commits behind head on main.
Additional details and impacted files
```diff
@@ Coverage Diff @@
## main #2052 +/- ##
==========================================
+ Coverage 92.46% 92.69% +0.22%
==========================================
Files 38 40 +2
Lines 5750 6047 +297
==========================================
+ Hits 5317 5605 +288
- Misses 433 442 +9
```
[see 19 files with indirect coverage changes](https://app.codecov.io/gh/simonw/datasette/pull/2052/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/simonw/datasette/pull/2052?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1651082214,
https://github.com/simonw/datasette/pull/2052#issuecomment-1760552652,https://api.github.com/repos/simonw/datasette/issues/2052,1760552652,IC_kwDOBm6k_c5o7-bM,9599,2023-10-12T23:59:21Z,2023-10-12T23:59:21Z,OWNER,I'm landing this despite the cog failures. I'll fix them on main if I have to.,"{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 1, ""rocket"": 0, ""eyes"": 0}",1651082214,