html_url,issue_url,id,node_id,user,user_label,created_at,updated_at,author_association,body,reactions,issue,issue_label,performed_via_github_app https://github.com/simonw/datasette/issues/2195#issuecomment-1730458954,https://api.github.com/repos/simonw/datasette/issues/2195,1730458954,IC_kwDOBm6k_c5nJLVK,9599,simonw,2023-09-21T22:57:39Z,2023-09-21T22:57:48Z,OWNER,"Worth noting that it already sets `--cors` automatically without you needing to specify it: https://github.com/simonw/datasette/blob/d97e82df3c8a3f2e97038d7080167be9bb74a68d/datasette/utils/__init__.py#L374-L374 I wonder if that's actually surprising behaviour that we should change before 1.0.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1907765514,`datasette publish` needs support for the new config/metadata split, https://github.com/simonw/datasette/issues/2195#issuecomment-1730457374,https://api.github.com/repos/simonw/datasette/issues/2195,1730457374,IC_kwDOBm6k_c5nJK8e,9599,simonw,2023-09-21T22:56:18Z,2023-09-21T22:56:18Z,OWNER,Maybe I should add `--cors` and `--crossdb` to `datasette publish cloudrun` as well?,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1907765514,`datasette publish` needs support for the new config/metadata split, https://github.com/simonw/datasette/issues/2195#issuecomment-1730446937,https://api.github.com/repos/simonw/datasette/issues/2195,1730446937,IC_kwDOBm6k_c5nJIZZ,9599,simonw,2023-09-21T22:46:42Z,2023-09-21T22:46:52Z,OWNER,"Found more when I [searched for YAML](https://github.com/search?q=datasette+publish+extra-options++language%3AYAML&type=code). Here's the most interesting: https://github.com/labordata/warehouse/blob/0029a72fc1ceae9091932da6566f891167179012/.github/workflows/build.yml#L59 `--extra-options=""--crossdb --setting sql_time_limit_ms 100000 --cors --setting facet_time_limit_ms 500 --setting allow_facet off --setting trace_debug 1""` Uses both `--cors` and `--crossdb`.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1907765514,`datasette publish` needs support for the new config/metadata split, https://github.com/simonw/datasette/issues/2195#issuecomment-1730441613,https://api.github.com/repos/simonw/datasette/issues/2195,1730441613,IC_kwDOBm6k_c5nJHGN,9599,simonw,2023-09-21T22:42:12Z,2023-09-21T22:42:12Z,OWNER,"https://github.com/search?q=datasette+publish+extra-options+language%3AShell&type=code&l=Shell shows 17 matches, I'll copy in illustrative examples here: ``` --extra-options=""--setting sql_time_limit_ms 5000"" --extra-options=""--config default_cache_ttl:3600 --config hash_urls:1"" --extra-options ""--setting sql_time_limit_ms 3500 --setting default_page_size 20 --setting trace_debug 1"" --extra-options=""--config default_page_size:50 --config sql_time_limit_ms:30000 --config facet_time_limit_ms:10000"" --extra-options=""--setting sql_time_limit_ms 5000"" --extra-options ""--setting suggest_facets off --setting allow_download on --setting truncate_cells_html 0 --setting max_csv_mb 0 --setting sql_time_limit_ms 2000"" ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1907765514,`datasette publish` needs support for the new config/metadata split, https://github.com/simonw/datasette/issues/2195#issuecomment-1730438503,https://api.github.com/repos/simonw/datasette/issues/2195,1730438503,IC_kwDOBm6k_c5nJGVn,9599,simonw,2023-09-21T22:38:10Z,2023-09-21T22:38:10Z,OWNER,"I'd really like to remove `--extra-options`. I think the new design makes that completely obsolete? Maybe it doesn't. You still need `--extra-options` for the `--crossdb` option for example.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1907765514,`datasette publish` needs support for the new config/metadata split, https://github.com/simonw/datasette/issues/2195#issuecomment-1730437934,https://api.github.com/repos/simonw/datasette/issues/2195,1730437934,IC_kwDOBm6k_c5nJGMu,9599,simonw,2023-09-21T22:37:22Z,2023-09-21T22:37:22Z,OWNER,"Here's the full help for Cloud Run at the moment: ```bash datasette publish cloudrun --help ``` ``` Usage: datasette publish cloudrun [OPTIONS] [FILES]... Publish databases to Datasette running on Cloud Run Options: -m, --metadata FILENAME Path to JSON/YAML file containing metadata to publish --extra-options TEXT Extra options to pass to datasette serve --branch TEXT Install datasette from a GitHub branch e.g. main --template-dir DIRECTORY Path to directory containing custom templates --plugins-dir DIRECTORY Path to directory containing custom plugins --static MOUNT:DIRECTORY Serve static files from this directory at /MOUNT/... --install TEXT Additional packages (e.g. plugins) to install --plugin-secret ... Secrets to pass to plugins, e.g. --plugin- secret datasette-auth-github client_id xxx --version-note TEXT Additional note to show on /-/versions --secret TEXT Secret used for signing secure values, such as signed cookies --title TEXT Title for metadata --license TEXT License label for metadata --license_url TEXT License URL for metadata --source TEXT Source label for metadata --source_url TEXT Source URL for metadata --about TEXT About label for metadata --about_url TEXT About URL for metadata -n, --name TEXT Application name to use when building --service TEXT Cloud Run service to deploy (or over-write) --spatialite Enable SpatialLite extension --show-files Output the generated Dockerfile and metadata.json --memory TEXT Memory to allocate in Cloud Run, e.g. 1Gi --cpu [1|2|4] Number of vCPUs to allocate in Cloud Run --timeout INTEGER Build timeout in seconds --apt-get-install TEXT Additional packages to apt-get install --max-instances INTEGER Maximum Cloud Run instances --min-instances INTEGER Minimum Cloud Run instances --help Show this message and exit. ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1907765514,`datasette publish` needs support for the new config/metadata split, https://github.com/simonw/datasette/issues/2195#issuecomment-1730437237,https://api.github.com/repos/simonw/datasette/issues/2195,1730437237,IC_kwDOBm6k_c5nJGB1,9599,simonw,2023-09-21T22:36:22Z,2023-09-21T22:36:22Z,OWNER,"I think the actual design of this is pretty simple. Current help starts like this: ``` Usage: datasette publish cloudrun [OPTIONS] [FILES]... Publish databases to Datasette running on Cloud Run Options: -m, --metadata FILENAME Path to JSON/YAML file containing metadata to publish --extra-options TEXT Extra options to pass to datasette serve ``` The `-s` and `-c` short options are not being used. So I think `-c/--config` can point to a JSON or YAML `datasette.yaml` file, and `-s/--setting key value` can mirror the new `-s/--setting` option in `datasette serve` itself (a shortcut for populating the config file directly from the CLI). Here's the relevant help section from `datasette serve`: ``` -m, --metadata FILENAME Path to JSON/YAML file containing license/source metadata -c, --config FILENAME Path to JSON/YAML Datasette configuration file -s, --setting SETTING... nested.key, value setting to use in Datasette configuration ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1907765514,`datasette publish` needs support for the new config/metadata split, https://github.com/simonw/datasette/issues/2189#issuecomment-1730388418,https://api.github.com/repos/simonw/datasette/issues/2189,1730388418,IC_kwDOBm6k_c5nI6HC,9599,simonw,2023-09-21T22:26:19Z,2023-09-21T22:26:19Z,OWNER,1.0a7 is out with this fix as well now: https://docs.datasette.io/en/1.0a7/changelog.html#a7-2023-09-21,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1901416155,Server hang on parallel execution of queries to named in-memory databases, https://github.com/simonw/datasette/issues/2057#issuecomment-1730363182,https://api.github.com/repos/simonw/datasette/issues/2057,1730363182,IC_kwDOBm6k_c5nIz8u,9599,simonw,2023-09-21T22:09:10Z,2023-09-21T22:09:10Z,OWNER,Tests all pass now.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1662951875,DeprecationWarning: pkg_resources is deprecated as an API, https://github.com/simonw/datasette/issues/2194#issuecomment-1730362441,https://api.github.com/repos/simonw/datasette/issues/2194,1730362441,IC_kwDOBm6k_c5nIzxJ,9599,simonw,2023-09-21T22:08:19Z,2023-09-21T22:08:19Z,OWNER,"That worked https://github.com/simonw/datasette/commit/e4f868801a6633400045f59584cfe650961c3fa6 is the latest commit right now and https://latest.datasette.io/-/versions shows that as the deployed version.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1907695234,"Deploy failing with ""plugins/alternative_route.py: Not a directory""", https://github.com/simonw/datasette/issues/2057#issuecomment-1730356422,https://api.github.com/repos/simonw/datasette/issues/2057,1730356422,IC_kwDOBm6k_c5nIyTG,9599,simonw,2023-09-21T22:01:00Z,2023-09-21T22:01:00Z,OWNER,Tested that locally with Python 3.9 from `pyenv` and it worked.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1662951875,DeprecationWarning: pkg_resources is deprecated as an API, https://github.com/simonw/datasette/issues/2057#issuecomment-1730353462,https://api.github.com/repos/simonw/datasette/issues/2057,1730353462,IC_kwDOBm6k_c5nIxk2,9599,simonw,2023-09-21T21:57:17Z,2023-09-21T21:57:17Z,OWNER,"Still fails in Python 3.9: https://github.com/simonw/datasette/actions/runs/6266752548/job/17018363302 ``` plugin_info[""name""] = distinfo.name or distinfo.project_name AttributeError: 'PathDistribution' object has no attribute 'name' Test failed: datasette-json-html should not have been loaded ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1662951875,DeprecationWarning: pkg_resources is deprecated as an API, https://github.com/simonw/datasette/issues/2193#issuecomment-1730353006,https://api.github.com/repos/simonw/datasette/issues/2193,1730353006,IC_kwDOBm6k_c5nIxdu,9599,simonw,2023-09-21T21:56:43Z,2023-09-21T21:56:43Z,OWNER,"The test fails as expected now. Closing this issue, will solve the remaining problems in: - #2057","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1907655261,"""Test DATASETTE_LOAD_PLUGINS"" test shows errors but did not fail the CI run", https://github.com/simonw/datasette/issues/2193#issuecomment-1730352111,https://api.github.com/repos/simonw/datasette/issues/2193,1730352111,IC_kwDOBm6k_c5nIxPv,9599,simonw,2023-09-21T21:55:41Z,2023-09-21T21:55:41Z,OWNER,"https://github.com/simonw/datasette/actions/runs/6267146158/job/17019594849 failed on 3.9 this time. ``` plugin_info[""name""] = distinfo.name or distinfo.project_name AttributeError: 'PathDistribution' object has no attribute 'name' Test failed: datasette-json-html should not have been loaded ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1907655261,"""Test DATASETTE_LOAD_PLUGINS"" test shows errors but did not fail the CI run", https://github.com/simonw/datasette/issues/2195#issuecomment-1730313565,https://api.github.com/repos/simonw/datasette/issues/2195,1730313565,IC_kwDOBm6k_c5nIn1d,9599,simonw,2023-09-21T21:16:31Z,2023-09-21T21:16:31Z,OWNER,"The `@add_common_publish_arguments_and_options` decorator described here is bad. If I update it to support a new `config` option all plugins that use it will break. https://github.com/simonw/datasette/blob/f130c7c0a88e50cea4121ea18d1f6db2431b6fab/docs/plugin_hooks.rst#L347-L355 I want to deprecate it and switch to a different, better design to address the same problem.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1907765514,`datasette publish` needs support for the new config/metadata split, https://github.com/simonw/datasette/issues/2195#issuecomment-1730312128,https://api.github.com/repos/simonw/datasette/issues/2195,1730312128,IC_kwDOBm6k_c5nInfA,9599,simonw,2023-09-21T21:15:11Z,2023-09-21T21:15:11Z,OWNER,"As soon as `datasette publish cloudrun` has this I can re-enable this bit of the demo deploy: https://github.com/simonw/datasette/blob/2da1a6acec915b81a16127008fd739c7d6075681/.github/workflows/deploy-latest.yml#L91-L97 Which should fix this broken demo from https://simonwillison.net/2022/Dec/2/datasette-write-api/ https://todomvc.datasette.io/","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1907765514,`datasette publish` needs support for the new config/metadata split, https://github.com/simonw/datasette/issues/2194#issuecomment-1730305920,https://api.github.com/repos/simonw/datasette/issues/2194,1730305920,IC_kwDOBm6k_c5nIl-A,9599,simonw,2023-09-21T21:09:21Z,2023-09-21T21:09:21Z,OWNER,"I'm going to disable this bit of the deploy for the moment, which will break the demo linked to from https://simonwillison.net/2022/Dec/2/datasette-write-api/ https://github.com/simonw/datasette/blob/2da1a6acec915b81a16127008fd739c7d6075681/.github/workflows/deploy-latest.yml#L91-L97 ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1907695234,"Deploy failing with ""plugins/alternative_route.py: Not a directory""", https://github.com/simonw/datasette/issues/2194#issuecomment-1730259871,https://api.github.com/repos/simonw/datasette/issues/2194,1730259871,IC_kwDOBm6k_c5nIauf,9599,simonw,2023-09-21T20:34:09Z,2023-09-21T20:34:09Z,OWNER,"... which raises the challenge that `datasette publish` doesn't yet know what to do with a config file! https://github.com/simonw/datasette/blob/2da1a6acec915b81a16127008fd739c7d6075681/.github/workflows/deploy-latest.yml#L114-L122","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1907695234,"Deploy failing with ""plugins/alternative_route.py: Not a directory""", https://github.com/simonw/datasette/issues/2194#issuecomment-1730258302,https://api.github.com/repos/simonw/datasette/issues/2194,1730258302,IC_kwDOBm6k_c5nIaV-,9599,simonw,2023-09-21T20:32:53Z,2023-09-21T20:33:02Z,OWNER,"Correct usage is now: ```bash python tests/fixtures.py fixtures.db fixtures-config.json fixtures-metadata.json \ plugins --extra-db-filename extra_database.db ``` ``` Test tables written to fixtures.db - metadata written to fixtures-metadata.json - config written to fixtures-config.json Wrote plugin: plugins/register_output_renderer.py Wrote plugin: plugins/view_name.py Wrote plugin: plugins/my_plugin.py Wrote plugin: plugins/messages_output_renderer.py Wrote plugin: plugins/sleep_sql_function.py Wrote plugin: plugins/my_plugin_2.py Test tables written to extra_database.db ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1907695234,"Deploy failing with ""plugins/alternative_route.py: Not a directory""", https://github.com/simonw/datasette/issues/2194#issuecomment-1730256435,https://api.github.com/repos/simonw/datasette/issues/2194,1730256435,IC_kwDOBm6k_c5nIZ4z,9599,simonw,2023-09-21T20:31:22Z,2023-09-21T20:31:31Z,OWNER,"New error: ""Error: Metadata should end with .json"" https://github.com/simonw/datasette/actions/runs/6266720924/job/17018265851 ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1907695234,"Deploy failing with ""plugins/alternative_route.py: Not a directory""", https://github.com/simonw/datasette/issues/2057#issuecomment-1730250337,https://api.github.com/repos/simonw/datasette/issues/2057,1730250337,IC_kwDOBm6k_c5nIYZh,9599,simonw,2023-09-21T20:26:12Z,2023-09-21T20:26:12Z,OWNER,That does seem to fix the problem! ,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1662951875,DeprecationWarning: pkg_resources is deprecated as an API, https://github.com/simonw/datasette/issues/2057#issuecomment-1730247545,https://api.github.com/repos/simonw/datasette/issues/2057,1730247545,IC_kwDOBm6k_c5nIXt5,9599,simonw,2023-09-21T20:23:47Z,2023-09-21T20:23:47Z,OWNER,Hunch: https://pypi.org/project/importlib-metadata/ may help here.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1662951875,DeprecationWarning: pkg_resources is deprecated as an API, https://github.com/simonw/datasette/issues/2194#issuecomment-1730245204,https://api.github.com/repos/simonw/datasette/issues/2194,1730245204,IC_kwDOBm6k_c5nIXJU,9599,simonw,2023-09-21T20:21:42Z,2023-09-21T20:21:42Z,OWNER,"I think I see the problem - it's from here: https://github.com/simonw/datasette/commit/b2ec8717c3619260a1b535eea20e618bf95aa30b#diff-5dbc88d6e5c3615caf10e32a9d6fc6ff683f5b5814948928cb84c3ab91c038b6L770 The `config` and `metadata` Click options are the wrong way round: https://github.com/simonw/datasette/blob/80a9cd9620fddf2695d12d8386a91e7c6b145ef2/tests/fixtures.py#L785-L786 https://github.com/simonw/datasette/blob/80a9cd9620fddf2695d12d8386a91e7c6b145ef2/tests/fixtures.py#L801","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1907695234,"Deploy failing with ""plugins/alternative_route.py: Not a directory""", https://github.com/simonw/datasette/issues/2194#issuecomment-1730242734,https://api.github.com/repos/simonw/datasette/issues/2194,1730242734,IC_kwDOBm6k_c5nIWiu,9599,simonw,2023-09-21T20:19:29Z,2023-09-21T20:19:29Z,OWNER,"Maybe `plugins/` does not exist? It should have been created by this line: https://github.com/simonw/datasette/blob/80a9cd9620fddf2695d12d8386a91e7c6b145ef2/.github/workflows/deploy-latest.yml#L41-L42","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1907695234,"Deploy failing with ""plugins/alternative_route.py: Not a directory""", https://github.com/simonw/datasette/issues/2194#issuecomment-1730241813,https://api.github.com/repos/simonw/datasette/issues/2194,1730241813,IC_kwDOBm6k_c5nIWUV,9599,simonw,2023-09-21T20:18:40Z,2023-09-21T20:18:40Z,OWNER,"This looks to be the step that is failing: https://github.com/simonw/datasette/blob/80a9cd9620fddf2695d12d8386a91e7c6b145ef2/.github/workflows/deploy-latest.yml#L50-L60","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1907695234,"Deploy failing with ""plugins/alternative_route.py: Not a directory""", https://github.com/simonw/datasette/issues/2189#issuecomment-1730232308,https://api.github.com/repos/simonw/datasette/issues/2189,1730232308,IC_kwDOBm6k_c5nIT_0,9599,simonw,2023-09-21T20:11:16Z,2023-09-21T20:11:16Z,OWNER,"We're planning a breaking change in `1.0a7`: - #2191 Since that's a breaking change I'm going to ship 1.0a7 right now with this fix, then ship that breaking change as `1.0a8` instead.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1901416155,Server hang on parallel execution of queries to named in-memory databases, https://github.com/simonw/datasette/issues/2189#issuecomment-1730231404,https://api.github.com/repos/simonw/datasette/issues/2189,1730231404,IC_kwDOBm6k_c5nITxs,9599,simonw,2023-09-21T20:10:28Z,2023-09-21T20:10:28Z,OWNER,Release 0.64.4: https://docs.datasette.io/en/stable/changelog.html#v0-64-4,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1901416155,Server hang on parallel execution of queries to named in-memory databases, https://github.com/simonw/datasette/issues/2057#issuecomment-1730226107,https://api.github.com/repos/simonw/datasette/issues/2057,1730226107,IC_kwDOBm6k_c5nISe7,9599,simonw,2023-09-21T20:06:19Z,2023-09-21T20:06:19Z,OWNER,"No that's not it actually, it's something else. Got to this point: ```bash DATASETTE_LOAD_PLUGINS=datasette-init python -i $(which datasette) plugins ``` That fails and drops me into a debugger: ``` File ""/Users/simon/Dropbox/Development/datasette/datasette/cli.py"", line 186, in plugins app = Datasette([], plugins_dir=plugins_dir) File ""/Users/simon/Dropbox/Development/datasette/datasette/app.py"", line 405, in __init__ for plugin in get_plugins() File ""/Users/simon/Dropbox/Development/datasette/datasette/plugins.py"", line 89, in get_plugins plugin_info[""name""] = distinfo.name or distinfo.project_name AttributeError: 'PathDistribution' object has no attribute 'name' ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1662951875,DeprecationWarning: pkg_resources is deprecated as an API, https://github.com/simonw/datasette/issues/2057#issuecomment-1730219703,https://api.github.com/repos/simonw/datasette/issues/2057,1730219703,IC_kwDOBm6k_c5nIQ63,9599,simonw,2023-09-21T20:01:54Z,2023-09-21T20:01:54Z,OWNER,"The problem is here: ``` 86 distinfo = plugin_to_distinfo.get(plugin) 87 if distinfo is None: 88 breakpoint() 89 -> assert False 90 if distinfo.name is None: 91 breakpoint() 92 assert False 93 if distinfo: 94 plugin_info[""version""] = distinfo.version (Pdb) distinfo (Pdb) plugin ``` That `plugin_to_distinfo` is missing some stuff.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1662951875,DeprecationWarning: pkg_resources is deprecated as an API, https://github.com/simonw/datasette/issues/2057#issuecomment-1730214654,https://api.github.com/repos/simonw/datasette/issues/2057,1730214654,IC_kwDOBm6k_c5nIPr-,9599,simonw,2023-09-21T19:59:51Z,2023-09-21T19:59:51Z,OWNER,"So the problem is the `get_plugins()` function returning plugins with `None` for their name: https://github.com/simonw/datasette/blob/80a9cd9620fddf2695d12d8386a91e7c6b145ef2/datasette/plugins.py#L61-L91","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1662951875,DeprecationWarning: pkg_resources is deprecated as an API, https://github.com/simonw/datasette/issues/2057#issuecomment-1730212597,https://api.github.com/repos/simonw/datasette/issues/2057,1730212597,IC_kwDOBm6k_c5nIPL1,9599,simonw,2023-09-21T19:58:38Z,2023-09-21T19:58:38Z,OWNER,Relevant code: https://github.com/simonw/datasette/blob/80a9cd9620fddf2695d12d8386a91e7c6b145ef2/datasette/app.py#L1127-L1146,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1662951875,DeprecationWarning: pkg_resources is deprecated as an API, https://github.com/simonw/datasette/issues/2057#issuecomment-1730211445,https://api.github.com/repos/simonw/datasette/issues/2057,1730211445,IC_kwDOBm6k_c5nIO51,9599,simonw,2023-09-21T19:57:44Z,2023-09-21T19:57:44Z,OWNER,"In the debugger: ``` >>> import pdb >>> pdb.pm() > /Users/simon/Dropbox/Development/datasette/datasette/app.py(1136)_plugins() -> ps.sort(key=lambda p: p[""name""]) (Pdb) ps [{'name': None, 'static_path': None, 'templates_path': None, 'hooks': ['prepare_connection', 'render_cell'], 'version': '1.0.1'}, {'name': None, 'static_path': None, 'templates_path': None, 'hooks': ['startup'], 'version': '0.2'}] ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1662951875,DeprecationWarning: pkg_resources is deprecated as an API, https://github.com/simonw/datasette/issues/2057#issuecomment-1730210728,https://api.github.com/repos/simonw/datasette/issues/2057,1730210728,IC_kwDOBm6k_c5nIOuo,9599,simonw,2023-09-21T19:57:08Z,2023-09-21T19:57:08Z,OWNER,"In my Python 3.8 environment I ran: ```bash datasette install datasette-init datasette-json-html ``` And now `datasette plugins` produces this error: ``` File ""/Users/simon/Dropbox/Development/datasette/datasette/cli.py"", line 192, in plugins click.echo(json.dumps(app._plugins(all=all), indent=4)) File ""/Users/simon/Dropbox/Development/datasette/datasette/app.py"", line 1136, in _plugins ps.sort(key=lambda p: p[""name""]) TypeError: '<' not supported between instances of 'NoneType' and 'NoneType' ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1662951875,DeprecationWarning: pkg_resources is deprecated as an API, https://github.com/simonw/datasette/issues/2193#issuecomment-1730208566,https://api.github.com/repos/simonw/datasette/issues/2193,1730208566,IC_kwDOBm6k_c5nIOM2,9599,simonw,2023-09-21T19:55:19Z,2023-09-21T19:55:19Z,OWNER,"Yes, the new script seems to work. On Python 3.11: ``` tests/test-datasette-load-plugins.sh echo $? 0 ``` On Python 3.8: ``` tests/test-datasette-load-plugins.sh Test failed: datasette-json-html not found echo $? 1 ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1907655261,"""Test DATASETTE_LOAD_PLUGINS"" test shows errors but did not fail the CI run", https://github.com/simonw/datasette/issues/2193#issuecomment-1730206629,https://api.github.com/repos/simonw/datasette/issues/2193,1730206629,IC_kwDOBm6k_c5nINul,9599,simonw,2023-09-21T19:53:39Z,2023-09-21T19:53:39Z,OWNER,"[GPT-4 says](https://chat.openai.com/share/65a95561-8f3c-4f9b-a4ab-c2508e87d2a9): > In the script, you're using a subshell `( ... )` to group commands. If you `exit 1` within the subshell, it will only exit the subshell and not the main script. This is why GitHub Actions does not see it as a failure. It suggested doing this instead: ```bash #!/bin/bash PLUGINS=$(datasette plugins) if ! echo ""$PLUGINS"" | jq 'any(.[]; .name == ""datasette-json-html"")' | grep -q true; then echo ""Test failed: datasette-json-html not found"" exit 1 fi PLUGINS2=$(DATASETTE_LOAD_PLUGINS=datasette-init datasette plugins) if ! echo ""$PLUGINS2"" | jq 'any(.[]; .name == ""datasette-json-html"")' | grep -q false; then echo ""Test failed: datasette-json-html should not have been loaded"" exit 1 fi if ! echo ""$PLUGINS2"" | jq 'any(.[]; .name == ""datasette-init"")' | grep -q true; then echo ""Test failed: datasette-init should have been loaded"" exit 1 fi PLUGINS3=$(DATASETTE_LOAD_PLUGINS='' datasette plugins) if ! echo ""$PLUGINS3"" | grep -q '\[\]'; then echo ""Test failed: datasette plugins should have returned []"" exit 1 fi ``` ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1907655261,"""Test DATASETTE_LOAD_PLUGINS"" test shows errors but did not fail the CI run", https://github.com/simonw/datasette/issues/2193#issuecomment-1730203356,https://api.github.com/repos/simonw/datasette/issues/2193,1730203356,IC_kwDOBm6k_c5nIM7c,9599,simonw,2023-09-21T19:51:04Z,2023-09-21T19:51:04Z,OWNER,"The script: https://github.com/simonw/datasette/blob/b0d0a0e5de8bb5b9b6c253e8af451a532266bcf1/tests/test-datasette-load-plugins.sh#L1-L29 I'm not sure why those `exit 1` lines did not cause a CI failure.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1907655261,"""Test DATASETTE_LOAD_PLUGINS"" test shows errors but did not fail the CI run", https://github.com/simonw/datasette/issues/2193#issuecomment-1730202533,https://api.github.com/repos/simonw/datasette/issues/2193,1730202533,IC_kwDOBm6k_c5nIMul,9599,simonw,2023-09-21T19:50:22Z,2023-09-21T19:50:22Z,OWNER,"Here's the failure in CI, which did not cause the workflow to fail even though it should have: ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1907655261,"""Test DATASETTE_LOAD_PLUGINS"" test shows errors but did not fail the CI run", https://github.com/simonw/datasette/issues/2057#issuecomment-1730201226,https://api.github.com/repos/simonw/datasette/issues/2057,1730201226,IC_kwDOBm6k_c5nIMaK,9599,simonw,2023-09-21T19:49:20Z,2023-09-21T19:49:20Z,OWNER,"That passed on 3.8 but should have failed: https://github.com/simonw/datasette/actions/runs/6266341481/job/17017099801 - the ""Test DATASETTE_LOAD_PLUGINS"" test shows errors but did not fail the CI run.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1662951875,DeprecationWarning: pkg_resources is deprecated as an API, https://github.com/simonw/datasette/issues/2057#issuecomment-1730188367,https://api.github.com/repos/simonw/datasette/issues/2057,1730188367,IC_kwDOBm6k_c5nIJRP,9599,simonw,2023-09-21T19:38:28Z,2023-09-21T19:40:38Z,OWNER,"I'll imitate `certbot`: https://github.com/certbot/certbot/blob/694c758db7fcd8410b5dadcd136c61b3eb028fdc/certbot-ci/setup.py#L9 ```python 'importlib_resources>=1.3.1; python_version < ""3.9""', ``` Looks like `1.3` is the minimum version needed for compatibility with the 3.9 standard library, according to https://github.com/python/importlib_resources/blob/main/README.rst#compatibility https://github.com/certbot/certbot/blob/694c758db7fcd8410b5dadcd136c61b3eb028fdc/certbot/certbot/_internal/constants.py#L13C29-L16 ```python if sys.version_info >= (3, 9): # pragma: no cover import importlib.resources as importlib_resources else: # pragma: no cover import importlib_resources ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1662951875,DeprecationWarning: pkg_resources is deprecated as an API, https://github.com/simonw/datasette/issues/2057#issuecomment-1730185322,https://api.github.com/repos/simonw/datasette/issues/2057,1730185322,IC_kwDOBm6k_c5nIIhq,9599,simonw,2023-09-21T19:35:49Z,2023-09-21T19:35:49Z,OWNER,I think I can fix this using https://importlib-resources.readthedocs.io/en/latest/using.html - maybe as a dependency only installed if the Python version is less than 3.9.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1662951875,DeprecationWarning: pkg_resources is deprecated as an API, https://github.com/simonw/datasette/issues/2057#issuecomment-1730183405,https://api.github.com/repos/simonw/datasette/issues/2057,1730183405,IC_kwDOBm6k_c5nIIDt,9599,simonw,2023-09-21T19:34:09Z,2023-09-21T19:34:09Z,OWNER,"Confirmed: https://docs.python.org/3/library/importlib.resources.html#importlib.resources.files > `importlib.resources.files(package)` > [...] > New in version 3.9.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1662951875,DeprecationWarning: pkg_resources is deprecated as an API, https://github.com/simonw/datasette/issues/2057#issuecomment-1730171241,https://api.github.com/repos/simonw/datasette/issues/2057,1730171241,IC_kwDOBm6k_c5nIFFp,9599,simonw,2023-09-21T19:27:25Z,2023-09-21T19:27:25Z,OWNER,"This broke in Python 3.8: ``` if plugin.__name__ not in DEFAULT_PLUGINS: try: if (importlib.resources.files(plugin.__name__) / ""static"").is_dir(): E AttributeError: module 'importlib.resources' has no attribute 'files' ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1662951875,DeprecationWarning: pkg_resources is deprecated as an API, https://github.com/simonw/datasette/issues/2189#issuecomment-1730162283,https://api.github.com/repos/simonw/datasette/issues/2189,1730162283,IC_kwDOBm6k_c5nIC5r,9599,simonw,2023-09-21T19:19:47Z,2023-09-21T19:19:47Z,OWNER,"I'm going to release this in `1.0a7`, and I'll backport it to a `0.64.4` release too.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1901416155,Server hang on parallel execution of queries to named in-memory databases,