home / github

Menu
  • Search all tables
  • GraphQL API

issues

Table actions
  • GraphQL API for issues

2,081 rows where repo = 107914493 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: milestone, author_association, draft, state_reason, created_at (date), updated_at (date), closed_at (date)

type 2

  • issue 1,649
  • pull 432

state 2

  • closed 1,536
  • open 545

repo 1

  • datasette · 2,081 ✖
id node_id number title user state locked assignee milestone comments created_at updated_at ▲ closed_at author_association pull_request body repo type active_lock_reason performed_via_github_app reactions draft state_reason
1910269679 I_kwDOBm6k_c5x3Gbv 2196 Discord invite link returns 401 Olshansk 1892194 open 0     0 2023-09-24T15:16:54Z 2023-09-24T15:16:54Z   NONE  

I found the link to the datasette discord channel via this query.

The following video should be self explanatory:

https://github.com/simonw/datasette/assets/1892194/8cd33e88-bcaa-41f3-9818-ab4d589c3f02

Link for reference: https://discord.com/invite/ktd74dm5mw

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2196/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1907765514 I_kwDOBm6k_c5xtjEK 2195 `datasette publish` needs support for the new config/metadata split simonw 9599 open 0     9 2023-09-21T21:08:12Z 2023-09-21T22:57:48Z   OWNER  

... which raises the challenge that datasette publish doesn't yet know what to do with a config file!

Originally posted by @simonw in https://github.com/simonw/datasette/issues/2194#issuecomment-1730259871

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2195/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1901416155 I_kwDOBm6k_c5xVU7b 2189 Server hang on parallel execution of queries to named in-memory databases simonw 9599 closed 0     31 2023-09-18T17:23:18Z 2023-09-21T22:26:21Z 2023-09-21T22:26:21Z OWNER  

I've started to encounter a bug where queries to tables inside named in-memory databases sometimes trigger server hangs.

I'm still trying to figure out what's going on here - on one occasion I managed to Ctrl+C the server and saw an exception that mentioned a thread lock, but usually hitting Ctrl+C does nothing and I have to kill -9 the PID instead.

This is all running on my M2 Mac.

I've seen the bug in the Datasette 1.0 alphas and in Datasette 0.64.3 - but reverting to 0.61 appeared to fix it.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2189/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1662951875 I_kwDOBm6k_c5jHqHD 2057 DeprecationWarning: pkg_resources is deprecated as an API simonw 9599 closed 0     25 2023-04-11T17:41:20Z 2023-09-21T22:09:10Z 2023-09-21T22:09:10Z OWNER  

Got this running tests against Python 3.11.

../../../.local/share/virtualenvs/datasette-big-local-6Yn-280V/lib/python3.11/site-packages/datasette/app.py:14: in <module> import pkg_resources ../../../.local/share/virtualenvs/datasette-big-local-6Yn-280V/lib/python3.11/site-packages/pkg_resources/__init__.py:121: in <module> warnings.warn("pkg_resources is deprecated as an API", DeprecationWarning) E DeprecationWarning: pkg_resources is deprecated as an API

I ran with pytest -Werror --pdb -x to get the debugger for that warning, but it turned out searching the code worked better. It's used in these two places:

https://github.com/simonw/datasette/blob/5890a20c374fb0812d88c9b0ef26a838bfa06c76/datasette/plugins.py#L43-L50

https://github.com/simonw/datasette/blob/5890a20c374fb0812d88c9b0ef26a838bfa06c76/datasette/app.py#L1037

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2057/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1907695234 I_kwDOBm6k_c5xtR6C 2194 Deploy failing with "plugins/alternative_route.py: Not a directory" simonw 9599 closed 0     8 2023-09-21T20:17:49Z 2023-09-21T22:08:19Z 2023-09-21T22:08:19Z OWNER  

https://github.com/simonw/datasette/actions/runs/6266449018/job/17017460074

This is a bit of a mystery, I don't think I've changed anything recently that could have broken this.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2194/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1907655261 I_kwDOBm6k_c5xtIJd 2193 "Test DATASETTE_LOAD_PLUGINS" test shows errors but did not fail the CI run simonw 9599 closed 0     6 2023-09-21T19:49:34Z 2023-09-21T21:56:43Z 2023-09-21T21:56:43Z OWNER  

That passed on 3.8 but should have failed: https://github.com/simonw/datasette/actions/runs/6266341481/job/17017099801 - the "Test DATASETTE_LOAD_PLUGINS" test shows errors but did not fail the CI run.

Originally posted by @simonw in https://github.com/simonw/datasette/issues/2057#issuecomment-1730201226

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2193/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1901483874 PR_kwDOBm6k_c5amULw 2190 Raise an exception if a "plugins" block exists in metadata.json asg017 15178711 open 0     2 2023-09-18T18:08:56Z 2023-09-21T16:56:58Z   CONTRIBUTOR simonw/datasette/pulls/2190

refs #2183 #2093

From this comment in #2183: If a "plugins" block appears in metadata.json, it means that a user hasn't migrated over their plugin configuration from metadata.json to datasette.yaml, which is a breaking change in Datasette 1.0.

This PR will ensure that an error is raised whenever that happens.


:books: Documentation preview :books:: https://datasette--2190.org.readthedocs.build/en/2190/

datasette 107914493 pull    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2190/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
0  
1901768721 PR_kwDOBm6k_c5anSg5 2191 DRAFT: Move `permissions` and `allow` blocks out of `metadata.yaml` and into `datasette.yaml` asg017 15178711 open 0     4 2023-09-18T21:21:16Z 2023-09-20T22:25:16Z   CONTRIBUTOR simonw/datasette/pulls/2191
  • [ ] Make sure tests pass
  • [ ] Docs
datasette 107914493 pull    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2191/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
1  
1896578249 PR_kwDOBm6k_c5aWACP 2185 Bump the python-packages group with 3 updates dependabot[bot] 49699333 closed 0     1 2023-09-14T13:27:40Z 2023-09-20T22:11:25Z 2023-09-20T22:11:24Z CONTRIBUTOR simonw/datasette/pulls/2185

Bumps the python-packages group with 3 updates: sphinx, furo and black.

Updates sphinx from 7.2.5 to 7.2.6

Release notes

Sourced from sphinx's releases.

Sphinx 7.2.6

Changelog: https://www.sphinx-doc.org/en/master/changes.html

Changelog

Sourced from sphinx's changelog.

Release 7.2.6 (released Sep 13, 2023)

Bugs fixed

  • #11679: Add the :envvar:!SPHINX_AUTODOC_RELOAD_MODULES environment variable, which if set reloads modules when using autodoc with TYPE_CHECKING = True. Patch by Matt Wozniski and Adam Turner.
  • #11679: Use :py:func:importlib.reload to reload modules in autodoc. Patch by Matt Wozniski and Adam Turner.
Commits
  • cf7d275 Bump to 7.2.6 final
  • 43d6975 Leverage importlib.reload for reloading modules (#11679)
  • 13da5d7 Inline makecmd in make mode
  • 3d0110a Enable test_cython on Python 3.12
  • 22759fb Bump version
  • See full diff in compare view


Updates furo from 2023.8.19 to 2023.9.10

Changelog

Sourced from furo's changelog.

Changelog

2023.09.10 -- Zesty Zaffre

  • Make asset hash injection idempotent, fixing Sphinx 6 compatibility.
  • Fix the check for HTML builders, fixing non-HTML Read the Docs builds.

2023.08.19 -- Xenolithic Xanadu

  • Fix missing search context with Sphinx 7.2, for dirhtml builds.
  • Drop support for Python 3.7.
  • Present configuration errors in a better format -- thanks @​AA-Turner!
  • Bump require_sphinx() to Sphinx 6.0, in line with dependency changes in Unassuming Ultramarine.

2023.08.17 -- Wonderous White

  • Fix compatiblity with Sphinx 7.2.0 and 7.2.1.

2023.07.26 -- Vigilant Volt

  • Fix compatiblity with Sphinx 7.1.
  • Improve how content overflow is handled.
  • Improve how literal blocks containing inline code are handled.

2023.05.20 -- Unassuming Ultramarine

  • ✨ Add support for Sphinx 7.
  • Drop support for Sphinx 5.
  • Improve the screen-reader label for sidebar collapse.
  • Make it easier to create derived themes from Furo.
  • Bump all JS dependencies (NodeJS and npm packages).

2023.03.27 -- Tasty Tangerine

  • Regenerate with newer version of sphinx-theme-builder, to fix RECORD hashes.
  • Add missing class to Font Awesome examples

2023.03.23 -- Sassy Saffron

... (truncated)

Commits
  • 2718ca4 Prepare release: 2023.09.10
  • c22c99d Update changelog
  • c37e849 Quote a not-runtime-generic type annotation
  • 9cfdf44 Rework infrastructure for linting
  • 5abeb9f Fix the check for HTML builders
  • ee2ab54 Tweak how tests are run with nox
  • cdae236 Test against Sphinx minor versions in CI
  • 9e40071 Make asset hash injection idempotent
  • aab86f4 Revert "Exclude incompatible Sphinx releases (#711)"
  • 4dd6eec Exclude incompatible Sphinx releases (#711)
  • Additional commits viewable in compare view


Updates black from 23.7.0 to 23.9.1

Release notes

Sourced from black's releases.

23.9.1

Due to various issues, the previous release (23.9.0) did not include compiled mypyc wheels, which make Black significantly faster. These issues have now been fixed, and this release should come with compiled wheels once again.

There will be no wheels for Python 3.12 due to a bug in mypyc. We will provide 3.12 wheels in a future release as soon as the mypyc bug is fixed.

Packaging

  • Upgrade to mypy 1.5.1 (#3864)

Performance

  • Store raw tuples instead of NamedTuples in Black's cache, improving performance and decreasing the size of the cache (#3877)

23.9.0

Preview style

  • More concise formatting for dummy implementations (#3796)
  • In stub files, add a blank line between a statement with a body (e.g an if sys.version_info > (3, x):) and a function definition on the same level (#3862)
  • Fix a bug whereby spaces were removed from walrus operators within subscript(#3823)

Configuration

  • Black now applies exclusion and ignore logic before resolving symlinks (#3846)

Performance

  • Avoid importing IPython if notebook cells do not contain magics (#3782)
  • Improve caching by comparing file hashes as fallback for mtime and size (#3821)

Blackd

  • Fix an issue in blackd with single character input (#3558)

Integrations

  • Black now has an official pre-commit mirror. Swapping https://github.com/psf/black to https://github.com/psf/black-pre-commit-mirror in your .pre-commit-config.yaml will make Black about 2x faster (#3828)
  • The .black.env folder specified by ENV_PATH will now be removed on the completion of the GitHub Action (#3759)
Changelog

Sourced from black's changelog.

23.9.1

Due to various issues, the previous release (23.9.0) did not include compiled mypyc wheels, which make Black significantly faster. These issues have now been fixed, and this release should come with compiled wheels once again.

There will be no wheels for Python 3.12 due to a bug in mypyc. We will provide 3.12 wheels in a future release as soon as the mypyc bug is fixed.

Packaging

  • Upgrade to mypy 1.5.1 (#3864)

Performance

  • Store raw tuples instead of NamedTuples in Black's cache, improving performance and decreasing the size of the cache (#3877)

23.9.0

Preview style

  • More concise formatting for dummy implementations (#3796)
  • In stub files, add a blank line between a statement with a body (e.g an if sys.version_info > (3, x):) and a function definition on the same level (#3862)
  • Fix a bug whereby spaces were removed from walrus operators within subscript(#3823)

Configuration

  • Black now applies exclusion and ignore logic before resolving symlinks (#3846)

Performance

  • Avoid importing IPython if notebook cells do not contain magics (#3782)
  • Improve caching by comparing file hashes as fallback for mtime and size (#3821)

Blackd

  • Fix an issue in blackd with single character input (#3558)

Integrations

  • Black now has an official pre-commit mirror. Swapping https://github.com/psf/black to https://github.com/psf/black-pre-commit-mirror in your .pre-commit-config.yaml will make Black about 2x faster (#3828)
  • The .black.env folder specified by ENV_PATH will now be removed on the completion of the GitHub Action (#3759)
Commits
  • e877371 Prepare release 23.9.1 (#3878)
  • 62dca32 mypyc builds on PRs, skip mypyc wheels for 3.12 (#3870)
  • 751583a Pickle raw tuples in FileData cache (#3877)
  • f791745 Re-export black.Mode (#3875)
  • 0b62b9c Ignore aiohttp DeprecationWarning for 3.12 (#3876)
  • c83ad6c Upgrade to Furo 2023.9.10 to fix docs build (#3873)
  • 4eebfd1 Add mypyc test marks to new tests that patch (#3871)
  • add161b Bump RTD Python version from 3.8 to 3.11 (#3868)
  • 4e93f2a Add classifier for 3.12 (#3866)
  • 716fa08 Upgrade mypy (#3864)
  • Additional commits viewable in compare view


Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore <dependency name> major version` will close this group update PR and stop Dependabot creating any more for the specific dependency's major version (unless you unignore this specific dependency's major version or upgrade to it yourself) - `@dependabot ignore <dependency name> minor version` will close this group update PR and stop Dependabot creating any more for the specific dependency's minor version (unless you unignore this specific dependency's minor version or upgrade to it yourself) - `@dependabot ignore <dependency name>` will close this group update PR and stop Dependabot creating any more for the specific dependency (unless you unignore this specific dependency or upgrade to it yourself) - `@dependabot unignore <dependency name>` will remove all of the ignore conditions of the specified dependency - `@dependabot unignore <dependency name> <ignore condition>` will remove the ignore condition of the specified dependency and ignore conditions

:books: Documentation preview :books:: https://datasette--2185.org.readthedocs.build/en/2185/

datasette 107914493 pull    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2185/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
0  
1903932086 PR_kwDOBm6k_c5aumyn 2192 Stop using parallel SQL queries for tables simonw 9599 closed 0     1 2023-09-20T01:28:43Z 2023-09-20T22:10:56Z 2023-09-20T22:10:55Z OWNER simonw/datasette/pulls/2192

Refs: - #2189


:books: Documentation preview :books:: https://datasette--2192.org.readthedocs.build/en/2192/

datasette 107914493 pull    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2192/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
0  
1825007061 I_kwDOBm6k_c5sx2XV 2123 datasette serve when invoked with --reload interprets the serve command as a file cadeef 79087 open 0     2 2023-07-27T19:07:22Z 2023-09-18T13:02:46Z   NONE  

When running datasette serve with the --reload flag, the serve command is picked up as a file argument:

$ datasette serve --reload test_db Starting monitor for PID 13574. Error: Invalid value for '[FILES]...': Path 'serve' does not exist. Press ENTER or change a file to reload.

If a 'serve' file is created it launches properly (albeit with an empty database called serve):

$ touch serve; datasette serve --reload test_db Starting monitor for PID 13628. INFO: Started server process [13628] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://127.0.0.1:8001 (Press CTRL+C to quit)

Version (running from HEAD on main):

$ datasette --version datasette, version 1.0a2

This issue appears to have existed for awhile as https://github.com/simonw/datasette/issues/1380#issuecomment-953366110 mentions the error in a different context.

I'm happy to debug and land a patch if it's welcome.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2123/reactions",
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1651082214 PR_kwDOBm6k_c5NcFCf 2052 feat: Javascript Plugin API (Custom panels, column menu items with JS actions) hydrosquall 9020979 open 0 simonw 9599   17 2023-04-02T20:23:44Z 2023-09-18T08:14:47Z   FIRST_TIME_CONTRIBUTOR simonw/datasette/pulls/2052

Motivation

  • Allow plugins that add data visualizations datasette-vega, datasette-leaflet, and datasette-nteract-data-explorer to co-exist safely
  • Standardize APIs / hooks to ease development for new JS plugin developers (better compat with datasette-lite) through standardized DOM selectors, methods for extending the existing Table UI. This has come up as a feature request several times (see research notes for examples)
  • Discussion w/ @simonw about a general-purpose Datasette JS API

Changes

Summary: Provide 2 new surface areas for Datasette JS plugin developers

  1. Custom column header items: https://a.cl.ly/Kou97wJr
  2. Basic "panels" controlled by buttons: https://a.cl.ly/rRugWobd

User Facing Changes

  • Allow creating menu items under table header that triggers JS (instead of opening hrefs per the existing menu_link hook). Items can respond to any column metadata provided by the column header (e.g. label). The proof of concept plugins log data to the console, or copy the column name to clipboard.
  • Allow plugins to register UI elements in a panel controller. The parent component handles switching the visibility of active plugins.
  • Because native button elements are used, the panel is keyboard-accessible - use tab / shift-tab to cycle through tab options, and enter to select.
  • There's room to improve the styling, but the focus of this PR is on the API rather than the UX.

(plugin) Developer Facing Changes

  • Dispatch a datasette_init CustomEvent when the datasetteManager is finished loading.
  • Provide manager.registerPlugin API for adding new functionality that coordinates with Datasette lifecycle events.
  • Provide a manager.selectors map of DOM elements that users may want to hook into.
  • Updated table.js to use refer to these to motivating keeping things in sync
  • Allow plugins to register themselves with 2 hooks:
  • makeColumnActions: Add items to menu in table column headers. Users can provide a label, and either href or onClick with full access to the metadata for the clicked column (name, type, misc. booleans)
  • makeAboveTablePanelConfigs: Add items to the panel. Each panel has a unique ID (namespaced within that plugin), a render function, and a string label.

See this file for example plugin usage.

Core Developer Facing Changes

  • Modified table.js to make use of the datasetteManager API.
  • Added example plugins to the demos/plugins folder, and stored the test js in the statics/ folder

Testing

For Datasette plugin developers, please see the alpha-level documentation .

To run the examples:

bash datasette serve fixtures.db --plugins-dir=demos/plugins/

Open local server: http://127.0.0.1:8001/fixtures/facetable

Open to all feedback on this PR, from API design to variable naming, to what additional hooks might be useful for the future.

My focus was more on the general shape of the API for developers, rather than on the UX of the test plugins.

Design notes

  • The manager tab panel could be a separate plugin if the implementation is too custom.
  • The makeColumnHeaderItems benefits from hooking into the logic of table.js
  • I wanted to offer this to the Datasette core, since the datasette-manager would be more powerful if it were connected to lifecycle and JS events that are part of the existing table.js.
  • Non-goals:
  • Dependency management (for now) - there's no "build" step, we don't know when new plugins will be added. While there are some valid use cases (for example, allow multiple plugins to wait for a global leaflet object to be loaded), I don't see enough use-cases to justify doing this yet.
  • Enabling single-page-app features - for now, most datasette actions lead to a new page being loaded. SPA development offers some benefits (no page jumping after clicking on a link), but also complexity that doesn't need to be in the core Datasette project.

Research Notes

  • Relocated to a comment, as this isn't required to review when evaluating the plugin. Including it just for those who are curious.
datasette 107914493 pull    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2052/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
0  
1900026059 I_kwDOBm6k_c5xQBjL 2188 Plugin Hooks for "compile to SQL" languages asg017 15178711 open 0     2 2023-09-18T01:37:15Z 2023-09-18T06:58:53Z   CONTRIBUTOR  

There's a ton of tools/languages that compile to SQL, which may be nice in Datasette. Some examples:

  • Logica https://logica.dev
  • PRQL https://prql-lang.org
  • Malloy, but not sure if it works with SQLite? https://github.com/malloydata/malloy

It would be cool if plugins could extend Datasette to use these languages, in both the code editor and API usage.

A few things I'd imagine a datasette-prql or datasette-logica plugin would do:

  • prql= instead of sql=
  • Code editor support (syntax highlighting, autocomplete)
  • Hide/show SQL
datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2188/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
787098345 MDU6SXNzdWU3ODcwOTgzNDU= 1191 Ability for plugins to collaborate when adding extra HTML to blocks in default templates simonw 9599 open 0   Datasette 1.0 3268330 12 2021-01-15T18:18:51Z 2023-09-18T06:55:52Z   OWNER  

Sometimes a plugin may want to add content to an existing default template - for example datasette-search-all adds a new search box at the top of index.html. I also want datasette-upload-csvs to add a CTA on the database.html page: https://github.com/simonw/datasette-upload-csvs/issues/18

Currently plugins can do this by providing a new version of the index.html template - but if multiple plugins try to do that only one of them will succeed.

It would be better if there were known areas of those templates which plugins could add additional content to, such that multiple plugins can use the same spot.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1191/reactions",
    "total_count": 4,
    "+1": 4,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1899310542 I_kwDOBm6k_c5xNS3O 2187 Datasette for serving JSON only geofinder 19705106 open 0     0 2023-09-16T05:48:29Z 2023-09-16T05:48:29Z   NONE  

Hi, is there any way to use datasette for serving json only without displaying webpage? I've tried to search about this in documentation but didn't get any information

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2187/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1898927976 I_kwDOBm6k_c5xL1do 2186 Mechanism for register_output_renderer hooks to access full count simonw 9599 open 0   Datasette 1.0 3268330 2 2023-09-15T18:57:54Z 2023-09-15T19:27:59Z   OWNER  

The cause of this bug: - https://github.com/simonw/datasette-export-notebook/issues/17

Is that datasette-export-notebook was consulting data["filtered_table_rows_count"] in the render output plugin function in order to show the total number of rows that would be exported.

That field is no longer available by default - the "count" field is only available if ?_extra=count was passed.

It would be useful if plugins like this could access the total count on demand, should they need to.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2186/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1890593563 PR_kwDOBm6k_c5aBx3g 2182 Bump the python-packages group with 2 updates dependabot[bot] 49699333 closed 0     1 2023-09-11T14:01:25Z 2023-09-14T13:27:30Z 2023-09-14T13:27:28Z CONTRIBUTOR simonw/datasette/pulls/2182

Bumps the python-packages group with 2 updates: furo and black.

Updates furo from 2023.8.19 to 2023.9.10

Changelog

Sourced from furo's changelog.

Changelog

2023.09.10 -- Zesty Zaffre

  • Make asset hash injection idempotent, fixing Sphinx 6 compatibility.
  • Fix the check for HTML builders, fixing non-HTML Read the Docs builds.

2023.08.19 -- Xenolithic Xanadu

  • Fix missing search context with Sphinx 7.2, for dirhtml builds.
  • Drop support for Python 3.7.
  • Present configuration errors in a better format -- thanks @​AA-Turner!
  • Bump require_sphinx() to Sphinx 6.0, in line with dependency changes in Unassuming Ultramarine.

2023.08.17 -- Wonderous White

  • Fix compatiblity with Sphinx 7.2.0 and 7.2.1.

2023.07.26 -- Vigilant Volt

  • Fix compatiblity with Sphinx 7.1.
  • Improve how content overflow is handled.
  • Improve how literal blocks containing inline code are handled.

2023.05.20 -- Unassuming Ultramarine

  • ✨ Add support for Sphinx 7.
  • Drop support for Sphinx 5.
  • Improve the screen-reader label for sidebar collapse.
  • Make it easier to create derived themes from Furo.
  • Bump all JS dependencies (NodeJS and npm packages).

2023.03.27 -- Tasty Tangerine

  • Regenerate with newer version of sphinx-theme-builder, to fix RECORD hashes.
  • Add missing class to Font Awesome examples

2023.03.23 -- Sassy Saffron

... (truncated)

Commits
  • 2718ca4 Prepare release: 2023.09.10
  • c22c99d Update changelog
  • c37e849 Quote a not-runtime-generic type annotation
  • 9cfdf44 Rework infrastructure for linting
  • 5abeb9f Fix the check for HTML builders
  • ee2ab54 Tweak how tests are run with nox
  • cdae236 Test against Sphinx minor versions in CI
  • 9e40071 Make asset hash injection idempotent
  • aab86f4 Revert "Exclude incompatible Sphinx releases (#711)"
  • 4dd6eec Exclude incompatible Sphinx releases (#711)
  • Additional commits viewable in compare view


Updates black from 23.7.0 to 23.9.1

Release notes

Sourced from black's releases.

23.9.1

Due to various issues, the previous release (23.9.0) did not include compiled mypyc wheels, which make Black significantly faster. These issues have now been fixed, and this release should come with compiled wheels once again.

There will be no wheels for Python 3.12 due to a bug in mypyc. We will provide 3.12 wheels in a future release as soon as the mypyc bug is fixed.

Packaging

  • Upgrade to mypy 1.5.1 (#3864)

Performance

  • Store raw tuples instead of NamedTuples in Black's cache, improving performance and decreasing the size of the cache (#3877)

23.9.0

Preview style

  • More concise formatting for dummy implementations (#3796)
  • In stub files, add a blank line between a statement with a body (e.g an if sys.version_info > (3, x):) and a function definition on the same level (#3862)
  • Fix a bug whereby spaces were removed from walrus operators within subscript(#3823)

Configuration

  • Black now applies exclusion and ignore logic before resolving symlinks (#3846)

Performance

  • Avoid importing IPython if notebook cells do not contain magics (#3782)
  • Improve caching by comparing file hashes as fallback for mtime and size (#3821)

Blackd

  • Fix an issue in blackd with single character input (#3558)

Integrations

  • Black now has an official pre-commit mirror. Swapping https://github.com/psf/black to https://github.com/psf/black-pre-commit-mirror in your .pre-commit-config.yaml will make Black about 2x faster (#3828)
  • The .black.env folder specified by ENV_PATH will now be removed on the completion of the GitHub Action (#3759)
Changelog

Sourced from black's changelog.

23.9.1

Due to various issues, the previous release (23.9.0) did not include compiled mypyc wheels, which make Black significantly faster. These issues have now been fixed, and this release should come with compiled wheels once again.

There will be no wheels for Python 3.12 due to a bug in mypyc. We will provide 3.12 wheels in a future release as soon as the mypyc bug is fixed.

Packaging

  • Upgrade to mypy 1.5.1 (#3864)

Performance

  • Store raw tuples instead of NamedTuples in Black's cache, improving performance and decreasing the size of the cache (#3877)

23.9.0

Preview style

  • More concise formatting for dummy implementations (#3796)
  • In stub files, add a blank line between a statement with a body (e.g an if sys.version_info > (3, x):) and a function definition on the same level (#3862)
  • Fix a bug whereby spaces were removed from walrus operators within subscript(#3823)

Configuration

  • Black now applies exclusion and ignore logic before resolving symlinks (#3846)

Performance

  • Avoid importing IPython if notebook cells do not contain magics (#3782)
  • Improve caching by comparing file hashes as fallback for mtime and size (#3821)

Blackd

  • Fix an issue in blackd with single character input (#3558)

Integrations

  • Black now has an official pre-commit mirror. Swapping https://github.com/psf/black to https://github.com/psf/black-pre-commit-mirror in your .pre-commit-config.yaml will make Black about 2x faster (#3828)
  • The .black.env folder specified by ENV_PATH will now be removed on the completion of the GitHub Action (#3759)
Commits
  • e877371 Prepare release 23.9.1 (#3878)
  • 62dca32 mypyc builds on PRs, skip mypyc wheels for 3.12 (#3870)
  • 751583a Pickle raw tuples in FileData cache (#3877)
  • f791745 Re-export black.Mode (#3875)
  • 0b62b9c Ignore aiohttp DeprecationWarning for 3.12 (#3876)
  • c83ad6c Upgrade to Furo 2023.9.10 to fix docs build (#3873)
  • 4eebfd1 Add mypyc test marks to new tests that patch (#3871)
  • add161b Bump RTD Python version from 3.8 to 3.11 (#3868)
  • 4e93f2a Add classifier for 3.12 (#3866)
  • 716fa08 Upgrade mypy (#3864)
  • Additional commits viewable in compare view


Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore <dependency name> major version` will close this group update PR and stop Dependabot creating any more for the specific dependency's major version (unless you unignore this specific dependency's major version or upgrade to it yourself) - `@dependabot ignore <dependency name> minor version` will close this group update PR and stop Dependabot creating any more for the specific dependency's minor version (unless you unignore this specific dependency's minor version or upgrade to it yourself) - `@dependabot ignore <dependency name>` will close this group update PR and stop Dependabot creating any more for the specific dependency (unless you unignore this specific dependency or upgrade to it yourself) - `@dependabot unignore <dependency name>` will remove all of the ignore conditions of the specified dependency - `@dependabot unignore <dependency name> <ignore condition>` will remove the ignore condition of the specified dependency and ignore conditions

:books: Documentation preview :books:: https://datasette--2182.org.readthedocs.build/en/2182/

datasette 107914493 pull    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2182/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
0  
1895266807 I_kwDOBm6k_c5w93n3 2184 Design decision - should configuration be exposed at /-/config ? simonw 9599 open 0     0 2023-09-13T21:07:08Z 2023-09-13T21:07:38Z   OWNER  

This made me think. That {"$env": "ENV_VAR"} hack was introduced back here:

  • https://github.com/simonw/datasette/issues/538

The problem it was solving was that metadata was visible to everyone with access to the instance at /-/metadata but plugins clearly needed a way to set secret settings.

Now that this stuff is moving to config, we have some decisions to make:

  1. Add /-/config to let people see the configuration of their instance, and keep the $env trick for secret settings.
  2. Say all configuration aside from metadata is secret and make $env optional or ditch it entirely.
  3. Allow plugins to announce which of their configuration options are secret so we can automatically redact them from /-/config

I've found /-/metadata extraordinarily useful as a user of Datasette - it really helps me understand exactly what's going on if I run into any problems with a plugin, if I can quickly check what the settings look like.

So I'm leaning towards option 1 or 3.

Originally posted by @simonw in https://github.com/simonw/datasette/pull/2183#discussion_r1325076924

Also refs: - #2093

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2184/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1891212159 PR_kwDOBm6k_c5aD33C 2183 `datasette.yaml` plugin support asg017 15178711 closed 0     4 2023-09-11T20:26:04Z 2023-09-13T21:06:25Z 2023-09-13T21:06:25Z CONTRIBUTOR simonw/datasette/pulls/2183

Part of #2093

In #2149 , we ported over "settings.json" into the new datasette.yaml config file, with a top-level "settings" key. This PR ports over plugin configuration into top-level "plugins" key, as well as nested database/table plugin config.

From now on, no plugin-related configuration is allowed in metadata.yaml, and must be in datasette.yaml in this new format. This is a pretty significant breaking change. Thankfully, you should be able to copy-paste your legacy plugin key/values into the new datasette.yaml format.

An example of what datasette.yaml would look like with this new plugin config:

```yaml

plugins: datasette-my-plugin: config_key: value

databases: fixtures: plugins: datasette-my-plugin: config_key: fixtures-db-value tables: students: plugins: datasette-my-plugin: config_key: fixtures-students-table-value

```

As an additional benefit, this now works with the new -s flag:

bash datasette --memory -s 'plugins.datasette-my-plugin.config_key' new_value

Marked as a "Draft" right now until I add better documentation. We also should have a plan for the next alpha release to document and publicize this change, especially for plugin authors (since their docs will have to change to say datasette.yaml instead of metadata.yaml


:books: Documentation preview :books:: https://datasette--2183.org.readthedocs.build/en/2183/

datasette 107914493 pull    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2183/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
0  
1781530343 I_kwDOBm6k_c5qL_7n 2093 Proposal: Combine settings, metadata, static, etc. into a single `datasette.yaml` File asg017 15178711 open 0     8 2023-06-29T21:18:23Z 2023-09-11T20:19:32Z   CONTRIBUTOR  

Very often I get tripped up when trying to configure my Datasette instances. For example: if I want to change the port my app listen too, do I do that with a CLI flag, a --setting flag, inside metadata.json, or an env var? If I want to up the time limit of SQL statements, is that under metadata.json or a setting? Where does my plugin configuration go?

Normally I need to look it up in Datasette docs, and I quickly find my answer, but the number of places where "config" goes it overwhelming.

  • Flat CLI flags like --port, --host, --cors, etc.
  • --setting, like default_page_size, sql_time_limit_ms etc
  • Inside metadata.json, including plugin configuration

Typically my Datasette deploys are extremely long shell commands, with multiple --setting and other CLI flags.

Proposal: Consolidate all "config" into datasette.toml

I propose that we add a new datasette.toml that combines "settings", "metadata", and other common CLI flags like --port and --cors into a single file. It would be similar to "Cargo.toml" in Rust projects, "package.json" in Node projects, and "pyproject.toml" in Python, etc.

A sample of what it could look like:

```toml

"top level" configuration that are currently CLI flags on datasette serve

[config] port = 8020 host = "0.0.0.0" cors = true

replaces multiple --setting flags

[settings] base_url = "/app/datasette/" default_allow_sql = true sql_time_limit_ms = 3500

replaces metadata.json.

The contents of datasette-metadata.json could be defined in this file instead, but supporting separate files is nice (since those are easy to machine-generate)

[metadata] include="./datasette-metadata.json"

plugin-specific

[plugins] [plugins.datasette-auth-github] client_id = {env = "DATASETTE_AUTH_GITHUB_CLIENT_ID"} client_secret = {env = "GITHUB_CLIENT_SECRET"}

[plugins.datasette-cluster-map]

latitude_column = "lat" longitude_column = "lon" ```

Pros

  • Instead of multiple files and CLI flags, everything could be in one tidy file
  • Editing config in a separate file is easier than editing CLI flags, since you don't have to kill a process + edit a command every time
  • New users will know "just edit my datasette.toml instead of needing to learn metadata + settings + CLI flags
  • Better dev experience for multiple environment. For example, could have datasette -c datasette-dev.toml for local dev environments (enables SQL, debug plugins, long timeouts, etc.), and a datasette -c datasette-prod.toml for "production" (lower timeouts, less plugins, monitoring plugins, etc.)

Cons

  • Yet another config-management system. Now Datasette users will need to know about metadata, settings, CLI flags, and datasette.toml. However with enough documentation + announcements + examples, I think we can get ahead of it.
  • If toml is chosen, would need to add a toml parser for Python version <3.11
  • Multiple sources of config require priority. For example: Would --setting default_allow_sql off override the value inside [settings]? What about --port?

Other Notes

Toml

I chose toml over json because toml supports comments. I chose toml over yaml because Python 3.11 has builtin support for it. I also find toml easier to work with since it doesn't have the odd "gotchas" that YAML has ("ex 3.10 resolving to 3.1, Norway NO resolving to false, etc.). It also mimics pyproject.toml which is nice. Happy to change my mind about this however

Plugin config will be difficult

Plugin config is currently in metadata.json in two places:

  1. Top level, under "plugins.[plugin-name]". This fits well into datasette.toml as [plugins.plugin-name]
  2. Table level, under "databases.[db-name].tables.[table-name].plugins.[plugin-name]. This doesn't fit that well into datasette.toml, unless it's nested under [metadata]?

Extensions, static, one-off plugins?

We could also include equivalents of --plugins-dir, --static, and --load-extension into datasette.toml, but I'd imagine there's a few security concerns there to think through.

Explicitly list with plugins to use?

I believe Datasette by default will load all install plugins on startup, but maybe datasette.toml can specify a list of plugins to use? For example, a dev version of datasette.toml can specify datasette-pretty-traces, but the prod version can leave it out

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2093/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1876353656 I_kwDOBm6k_c5v1uJ4 2168 Consider a request/response wrapping hook slightly higher level than asgi_wrapper() simonw 9599 open 0     6 2023-08-31T21:42:04Z 2023-09-10T17:54:08Z   OWNER  

There's a long justification for why this might be needed here: - https://github.com/simonw/datasette-auth-tokens/issues/10#issuecomment-1701820001

Short version: it would be neat if it was possible to stash some data on the request object such that a later plugin/middleware-type-thing could use that to influence the final returned response - similar to the kinds of things you can do with Django middleware.

The asgi_wrapper() mechanism doesn't have access to the request or response objects - it gets scope and can mess around with receive and send, but those are pretty low-level primitives.

Since Datasette has well-defined request and response objects now it might be nice to have a middleware layer that can manipulate those directly.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2168/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1886791100 I_kwDOBm6k_c5wdiW8 2180 Plugin hook: `actors_from_ids()` simonw 9599 closed 0     6 2023-09-08T01:16:41Z 2023-09-10T17:44:14Z 2023-09-08T04:28:03Z OWNER  

In building Datasette Cloud we realized that a bunch of the features we are building need a way of resolving an actor ID to the actual actor, in order to display something more interesting than just an integer ID.

Social plugins in particular need this - comments by X, CSV uploaded by X, that kind of thing.

I think the solution is a new plugin hook: actors_from_ids(datasette, ids) which can return a list of actor dictionaries.

The default implementation can return [{"id": "..."}] for the IDs passed to it.

Pluggy has a firstresult=True option which is relevant here, since this is the first plugin hook we will have implemented where only one plugin should provide an answer.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2180/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1874255116 I_kwDOBm6k_c5vtt0M 2164 Ability to only load a specific list of plugins simonw 9599 closed 0     1 2023-08-30T19:33:41Z 2023-09-08T04:35:46Z 2023-08-30T22:12:27Z OWNER  

I'm going to try and get this working through an environment variable, so that you can start Datasette and it will only load a subset of plugins including those that use the register_commands() hook.

Initial research on this: - https://github.com/pytest-dev/pluggy/issues/422

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2164/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1886812002 PR_kwDOBm6k_c5Z1N2L 2181 actors_from_ids plugin hook and datasette.actors_from_ids() method simonw 9599 closed 0     3 2023-09-08T01:51:07Z 2023-09-08T04:24:00Z 2023-09-08T04:23:59Z OWNER simonw/datasette/pulls/2181

Refs: - #2180

This plugin hook is feature complete - including documentation and tests.

I'm not going to land it in Datasette main until we've used it at least once though, which should happen promptly in development for Datasette Cloud.


:books: Documentation preview :books:: https://datasette--2181.org.readthedocs.build/en/2181/

datasette 107914493 pull    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2181/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
0  
1884330740 PR_kwDOBm6k_c5ZszDF 2174 Use $DATASETTE_INTERNAL in absence of --internal asg017 15178711 open 0     3 2023-09-06T16:07:15Z 2023-09-08T00:46:13Z   CONTRIBUTOR simonw/datasette/pulls/2174

refs 2157, specifically this comment

Passing in --internal my_internal.db over and over again can get repetitive.

This PR adds a new configurable env variable DATASETTE_INTERNAL_DB_PATH. If it's defined, then it takes place as the path to the internal database. Users can still overwrite this behavior by passing in their own --internal internal.db flag.

In draft mode for now, needs tests and documentation.

Side note: Maybe we can have a sections in the docs that lists all the "configuration environment variables" that Datasette respects? I did a quick grep and found:

  • DATASETTE_LOAD_PLUGINS
  • DATASETTE_SECRETS

:books: Documentation preview :books:: https://datasette--2174.org.readthedocs.build/en/2174/

datasette 107914493 pull    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2174/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
0  
1886350562 I_kwDOBm6k_c5wb2zi 2178 Don't show foreign key links to tables the user cannot access simonw 9599 closed 0     5 2023-09-07T17:56:41Z 2023-09-07T23:28:27Z 2023-09-07T23:28:27Z OWNER  

Spotted this problem while working on this plugin: - https://github.com/simonw/datasette-public

It's possible to make a table public to any users - but then you may end up with situations like this:

That table is public, but the foreign key links go to tables that are NOT public.

We're also leaking the names of the values in those private tables here, which we shouldn't do. So this is a tiny bit of an information leak.

Since this only affects people who have configured a table to be public that has foreign keys to a table that is private I don't think this is worth issuing a vulnerability report about - I very much doubt anyone is running Datasette configured in a way that could result in problems because of this.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2178/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1886649402 I_kwDOBm6k_c5wc_w6 2179 Flaky test: test_hidden_sqlite_stat1_table simonw 9599 closed 0     0 2023-09-07T22:48:43Z 2023-09-07T22:51:19Z 2023-09-07T22:51:19Z OWNER  

This test here: https://github.com/simonw/datasette/blob/fbcb103c0cb6668018ace539a01a6a1f156e8d6a/tests/test_api.py#L1011-L1020

It failed for me like this:

E AssertionError: assert [('normal', False), ('sqlite_stat1', True), ('sqlite_stat4', True)] in ([('normal', False), ('sqlite_stat1', True)],)

Looks like some builds of SQLite include a sqlite_stat4 table.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2179/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1010112818 I_kwDOBm6k_c48NRky 1479 Win32 "used by another process" error with datasette publish kirajano 76450761 open 0     7 2021-09-28T19:12:00Z 2023-09-07T02:14:16Z   NONE  

I unfortunately was not successful to deploy to fly.io. Please see the details above of the three scenarios that I took. I am also new to datasette.

Failed to deploy. Attaching logs: 1. Tried with an app created via flyctl apps create frosty-fog-8565 and the ran datasette publish fly covid.db --app frosty-fog-8565 ``` Deploying frosty-fog-8565 ==> Validating app configuration --> Validating app configuration done Services TCP 80/443 ⇢ 8080

Error error connecting to docker: An unknown error occured.

Traceback (most recent call last): File "c:\users\grott\anaconda3\lib\runpy.py", line 193, in _run_module_as_main "main", mod_spec) File "c:\users\grott\anaconda3\lib\runpy.py", line 85, in _run_code exec(code, run_globals) File "C:\Users\grott\Anaconda3\Scripts\datasette.exe__main__.py", line 7, in <module> File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 829, in call return self.main(args, kwargs) File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 782, in main rv = self.invoke(ctx) File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 1259, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 1259, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 1066, in invoke return ctx.invoke(self.callback, ctx.params) File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 610, in invoke return callback(args, **kwargs) File "c:\users\grott\anaconda3\lib\site-packages\datasette_publish_fly__init__.py", line 156, in fly "--remote-only", File "c:\users\grott\anaconda3\lib\contextlib.py", line 119, in exit next(self.gen) File "c:\users\grott\anaconda3\lib\site-packages\datasette\utils__init__.py", line 451, in temporary_docker_directory tmp.cleanup() File "c:\users\grott\anaconda3\lib\tempfile.py", line 811, in cleanup _shutil.rmtree(self.name) File "c:\users\grott\anaconda3\lib\shutil.py", line 516, in rmtree return _rmtree_unsafe(path, onerror) File "c:\users\grott\anaconda3\lib\shutil.py", line 395, in _rmtree_unsafe _rmtree_unsafe(fullname, onerror) File "c:\users\grott\anaconda3\lib\shutil.py", line 404, in _rmtree_unsafe onerror(os.rmdir, path, sys.exc_info()) File "c:\users\grott\anaconda3\lib\shutil.py", line 402, in _rmtree_unsafe os.rmdir(path) PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\Users\grott\AppData\Local\Temp\tmpgcm8cz66\frosty-fog-8565' ```

  1. Tried also with an app that gets autogenerate when running flyctl launch. This also generates the .toml file. Ran then datasette publish fly covid.db --app dark-feather-168 but different error now ```Deploying dark-feather-168 ==> Validating app configuration

Error not possible to validate configuration: server returned Post "https://api.fly.io/graphql": unexpected EOF

Traceback (most recent call last): File "c:\users\grott\anaconda3\lib\runpy.py", line 193, in _run_module_as_main
"main", mod_spec) exec(code, run_globals) File "C:\Users\grott\Anaconda3\Scripts\datasette.exe__main__.py", line 7, in <module> File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 829, in call return self.main(args, kwargs) File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 782, in main rv = self.invoke(ctx) File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 1259, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 1259, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 1066, in invoke return ctx.invoke(self.callback, ctx.params) File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 610, in invoke return callback(args, **kwargs) File "c:\users\grott\anaconda3\lib\site-packages\datasette_publish_fly__init__.py", line 156, in fly "--remote-only", File "c:\users\grott\anaconda3\lib\contextlib.py", line 119, in exit next(self.gen) File "c:\users\grott\anaconda3\lib\site-packages\datasette\utils__init__.py", line 451, in temporary_docker_directory tmp.cleanup() File "c:\users\grott\anaconda3\lib\tempfile.py", line 811, in cleanup _shutil.rmtree(self.name) File "c:\users\grott\anaconda3\lib\shutil.py", line 516, in rmtree return _rmtree_unsafe(path, onerror) File "c:\users\grott\anaconda3\lib\shutil.py", line 395, in _rmtree_unsafe _rmtree_unsafe(fullname, onerror) File "c:\users\grott\anaconda3\lib\shutil.py", line 404, in _rmtree_unsafe onerror(os.rmdir, path, sys.exc_info()) File "c:\users\grott\anaconda3\lib\shutil.py", line 402, in _rmtree_unsafe os.rmdir(path) PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\Users\grott\AppData\Local\Temp\tmpnoyewcre\dark-feather-168' ```

These are also the contents of the generated .toml file in 2 scenario:

```

fly.toml file generated for dark-feather-168 on 2021-09-28T20:35:44+02:00

app = "dark-feather-168"

kill_signal = "SIGINT" kill_timeout = 5 processes = []

[env]

[experimental] allowed_public_ports = [] auto_rollback = true

[[services]] http_checks = [] internal_port = 8080 processes = ["app"] protocol = "tcp" script_checks = []

[services.concurrency] hard_limit = 25 soft_limit = 20 type = "connections"

[[services.ports]] handlers = ["http"] port = 80

[[services.ports]] handlers = ["tls", "http"] port = 443

[[services.tcp_checks]] grace_period = "1s" interval = "15s" restart_limit = 6 timeout = "2s" ```

  1. But also trying datasette package covid.db to create a local DOCKERFILE to later try to push it via flyctl deploy fails as well.

```[+] Building 147.3s (11/11) FINISHED => [internal] load build definition from Dockerfile 0.2s => => transferring dockerfile: 396B 0.0s => [internal] load .dockerignore 0.1s => => transferring context: 2B 0.0s => [internal] load metadata for docker.io/library/python:3.8 4.7s => [auth] library/python:pull token for registry-1.docker.io 0.0s => [internal] load build context 0.1s => => transferring context: 82.37kB 0.0s => [1/5] FROM docker.io/library/python:3.8@sha256:530de807b46a11734e2587a784573c12c5034f2f14025f838589e6c0e3 108.3s => => resolve docker.io/library/python:3.8@sha256:530de807b46a11734e2587a784573c12c5034f2f14025f838589e6c0e3b5 0.0s => => sha256:56182bcdf4d4283aa1f46944b4ef7ac881e28b4d5526720a4e9ba03a4730846a 2.22kB / 2.22kB 0.0s => => sha256:955615a668ce169f8a1443fc6b6e6215f43fe0babfb4790712a2d3171f34d366 54.93MB / 54.93MB 21.6s => => sha256:911ea9f2bd51e53a455297e0631e18a72a86d7e2c8e1807176e80f991bde5d64 10.87MB / 10.87MB 15.5s => => sha256:530de807b46a11734e2587a784573c12c5034f2f14025f838589e6c0e3b5c5b6 1.86kB / 1.86kB 0.0s => => sha256:ff08f08727e50193dcf499afc30594c47e70cc96f6fcfd1a01240524624264d0 8.65kB / 8.65kB 0.0s => => sha256:2756ef5f69a5190f4308619e0f446d95f5515eef4a814dbad0bcebbbbc7b25a8 5.15MB / 5.15MB 6.4s => => sha256:27b0a22ee906271a6ce9ddd1754fdd7d3b59078e0b57b6cc054c7ed7ac301587 54.57MB / 54.57MB 37.7s => => sha256:8584d51a9262f9a3a436dea09ba40fa50f85802018f9bd299eee1bf538481077 196.45MB / 196.45MB 82.3s => => sha256:524774b7d3638702fe9ae0ea3fcfb81b027dfd75cc2fc14f0119e764b9543d58 6.29MB / 6.29MB 26.6s => => extracting sha256:955615a668ce169f8a1443fc6b6e6215f43fe0babfb4790712a2d3171f34d366 5.4s => => sha256:9460f6b75036e38367e2f27bb15e85777c5d6cd52ad168741c9566186415aa26 16.81MB / 16.81MB 40.5s => => extracting sha256:2756ef5f69a5190f4308619e0f446d95f5515eef4a814dbad0bcebbbbc7b25a8 0.6s => => extracting sha256:911ea9f2bd51e53a455297e0631e18a72a86d7e2c8e1807176e80f991bde5d64 0.6s => => sha256:9bc548096c181514aa1253966a330134d939496027f92f57ab376cd236eb280b 232B / 232B 40.1s => => extracting sha256:27b0a22ee906271a6ce9ddd1754fdd7d3b59078e0b57b6cc054c7ed7ac301587 5.8s => => sha256:1d87379b86b89fd3b8bb1621128f00c8f962756e6aaaed264ec38db733273543 2.35MB / 2.35MB 41.8s => => extracting sha256:8584d51a9262f9a3a436dea09ba40fa50f85802018f9bd299eee1bf538481077 18.8s => => extracting sha256:524774b7d3638702fe9ae0ea3fcfb81b027dfd75cc2fc14f0119e764b9543d58 1.2s => => extracting sha256:9460f6b75036e38367e2f27bb15e85777c5d6cd52ad168741c9566186415aa26 2.9s => => extracting sha256:9bc548096c181514aa1253966a330134d939496027f92f57ab376cd236eb280b 0.0s => => extracting sha256:1d87379b86b89fd3b8bb1621128f00c8f962756e6aaaed264ec38db733273543 0.8s => [2/5] COPY . /app 2.3s => [3/5] WORKDIR /app 0.2s => [4/5] RUN pip install -U datasette 26.9s => [5/5] RUN datasette inspect covid.db --inspect-file inspect-data.json 3.1s => exporting to image 1.2s => => exporting layers 1.2s => => writing image sha256:b5db0c205cd3454c21fbb00ecf6043f261540bcf91c2dfc36d418f1a23a75d7a 0.0s

Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them Traceback (most recent call last): "main", mod_spec) File "c:\users\grott\anaconda3\lib\runpy.py", line 85, in _run_code exec(code, run_globals) File "C:\Users\grott\Anaconda3\Scripts\datasette.exe__main__.py", line 7, in <module> File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 829, in call return self.main(args, kwargs) File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 782, in main rv = self.invoke(ctx) File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 1259, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 1066, in invoke return ctx.invoke(self.callback, ctx.params) File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 610, in invoke return callback(args, **kwargs) File "c:\users\grott\anaconda3\lib\site-packages\datasette\cli.py", line 283, in package call(args) File "c:\users\grott\anaconda3\lib\contextlib.py", line 119, in exit next(self.gen) File "c:\users\grott\anaconda3\lib\site-packages\datasette\utils__init__.py", line 451, in temporary_docker_directory tmp.cleanup() File "c:\users\grott\anaconda3\lib\tempfile.py", line 811, in cleanup _shutil.rmtree(self.name) File "c:\users\grott\anaconda3\lib\shutil.py", line 516, in rmtree return _rmtree_unsafe(path, onerror) File "c:\users\grott\anaconda3\lib\shutil.py", line 395, in _rmtree_unsafe _rmtree_unsafe(fullname, onerror) File "c:\users\grott\anaconda3\lib\shutil.py", line 404, in _rmtree_unsafe onerror(os.rmdir, path, sys.exc_info()) File "c:\users\grott\anaconda3\lib\shutil.py", line 402, in _rmtree_unsafe os.rmdir(path) PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\Users\grott\AppData\Local\Temp\tmpkb27qid3\datasette'```

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1479/reactions",
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1884408624 I_kwDOBm6k_c5wUcsw 2177 Move schema tables from _internal to _catalog simonw 9599 open 0     1 2023-09-06T16:58:33Z 2023-09-06T17:04:30Z   OWNER  

This came up in discussion over: - https://github.com/simonw/datasette/pull/2174

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2177/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1875519316 PR_kwDOBm6k_c5ZPO5y 2166 Bump the python-packages group with 1 update dependabot[bot] 49699333 closed 0     1 2023-08-31T13:19:57Z 2023-09-06T16:34:32Z 2023-09-06T16:34:31Z CONTRIBUTOR simonw/datasette/pulls/2166

Bumps the python-packages group with 1 update: sphinx.

Release notes

Sourced from sphinx's releases.

Sphinx 7.2.5

Changelog: https://www.sphinx-doc.org/en/master/changes.html

Changelog

Sourced from sphinx's changelog.

Release 7.2.5 (released Aug 30, 2023)

Bugs fixed

  • #11645: Fix a regression preventing autodoc from importing modules within packages that make use of if typing.TYPE_CHECKING: to guard circular imports needed by type checkers. Patch by Matt Wozniski.
  • #11634: Fixed inheritance diagram relative link resolution for sibling files in a subdirectory. Patch by Albert Shih.
  • #11659: Allow ?config=... in :confval:mathjax_path.
  • #11654: autodoc: Fail with a more descriptive error message when an object claims to be an instance of type, but is not a class. Patch by James Braza.
  • 11620: Cease emitting :event:source-read events for files read via the :dudir:include directive.
  • 11620: Add a new :event:include-read for observing and transforming the content of included files via the :dudir:include directive.
  • #11627: Restore support for copyright lines of the form YYYY when SOURCE_DATE_EPOCH is set.
Commits
  • fcc3899 Bump to 7.2.5 final
  • 2a631f9 Restore support for YYYY copyright lines
  • 2730cc3 Remove double spaces in CHANGES
  • ff18318 Add an 'include-read' event (#11657)
  • 74329d9 Fail better in ExceptionDocumenter.can_document_member (#11660)
  • 7d046a8 Allow ?config=... in mathjax_path (#11659)
  • 4692208 Fix two relative link bugs in inheritance diagrams (#11634)
  • ca0fc7a Add git .mailmap file
  • 8248be3 autodoc: Reset sys.modules on partial import failure (#11645)
  • e494baa Recommend correct replacement names for deprecated APIs (#11655)
  • Additional commits viewable in compare view


Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore <dependency name> major version` will close this group update PR and stop Dependabot creating any more for the specific dependency's major version (unless you unignore this specific dependency's major version or upgrade to it yourself) - `@dependabot ignore <dependency name> minor version` will close this group update PR and stop Dependabot creating any more for the specific dependency's minor version (unless you unignore this specific dependency's minor version or upgrade to it yourself) - `@dependabot ignore <dependency name>` will close this group update PR and stop Dependabot creating any more for the specific dependency (unless you unignore this specific dependency or upgrade to it yourself) - `@dependabot unignore <dependency name>` will remove all of the ignore conditions of the specified dependency - `@dependabot unignore <dependency name> <ignore condition>` will remove the ignore condition of the specified dependency and ignore conditions

:books: Documentation preview :books:: https://datasette--2166.org.readthedocs.build/en/2166/

datasette 107914493 pull    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2166/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
0  
1884333600 PR_kwDOBm6k_c5Zszqk 2175 Test against Python 3.12 preview simonw 9599 closed 0     0 2023-09-06T16:09:05Z 2023-09-06T16:16:28Z 2023-09-06T16:16:27Z OWNER simonw/datasette/pulls/2175

https://dev.to/hugovk/help-test-python-312-beta-1508/


:books: Documentation preview :books:: https://datasette--2175.org.readthedocs.build/en/2175/

datasette 107914493 pull    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2175/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
0  
1883055640 PR_kwDOBm6k_c5ZociX 2173 click-default-group>=1.2.3 simonw 9599 closed 0     3 2023-09-06T02:33:28Z 2023-09-06T02:50:10Z 2023-09-06T02:50:10Z OWNER simonw/datasette/pulls/2173

Now available as a wheel: - https://github.com/click-contrib/click-default-group/issues/21


:books: Documentation preview :books:: https://datasette--2173.org.readthedocs.build/en/2173/

datasette 107914493 pull    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2173/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
0  
336464733 MDU6SXNzdWUzMzY0NjQ3MzM= 328 Installation instructions, including how to use the docker image simonw 9599 closed 0     4 2018-06-28T03:59:33Z 2023-09-05T14:10:39Z 2018-06-28T04:02:10Z OWNER  
datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/328/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1292370469 I_kwDOBm6k_c5NCAIl 1765 Document plugins providing new plugin hook- simonw 9599 closed 0     1 2022-07-03T17:05:14Z 2023-08-31T23:08:24Z 2023-08-31T23:06:31Z OWNER  

I've used this pattern twice now: https://til.simonwillison.net/datasette/register-new-plugin-hooks - in datasette-graphql and datasette-low-disk-space-hook. I should describe the pattern on https://docs.datasette.io/en/stable/writing_plugins.html

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1765/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1876407598 I_kwDOBm6k_c5v17Uu 2169 execute-sql on a database should imply view-database/view-permission simonw 9599 closed 0     0 2023-08-31T22:45:56Z 2023-08-31T22:46:28Z 2023-08-31T22:46:28Z OWNER  

I noticed that a token with execute-sql permission alone did not work, because it was not allowed to view the instance of the database.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2169/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1875739055 I_kwDOBm6k_c5vzYGv 2167 Document return type of await ds.permission_allowed() simonw 9599 open 0     0 2023-08-31T15:14:23Z 2023-08-31T15:14:23Z   OWNER  

The return type isn't documented here: https://github.com/simonw/datasette/blob/4c3ef033110407f3b3dbce501659d523724985e0/docs/internals.rst#L327-L350

On inspecting the code I'm not 100% sure if it's possible for this. method to return None, or if it can only return True or False. Need to confirm that.

https://github.com/simonw/datasette/blob/4c3ef033110407f3b3dbce501659d523724985e0/datasette/app.py#L822C15-L853

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2167/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1865869205 I_kwDOBm6k_c5vNueV 2157 Proposal: Make the `_internal` database persistent, customizable, and hidden asg017 15178711 open 0     3 2023-08-24T20:54:29Z 2023-08-31T02:45:56Z   CONTRIBUTOR  

The current _internal database is used by Datasette core to cache info about databases/tables/columns/foreign keys of databases in a Datasette instance. It's a temporary database created at startup, that can only be seen by the root user. See an example _internal DB here, after logging in as root.

The current _internal database has a few rough edges:

  • It's part of datasette.databases, so many plugins have to specifically exclude _internal from their queries examples here
  • It's only used by Datasette core and can't be used by plugins or 3rd parties
  • It's created from scratch at startup and stored in memory. Why is fine, the performance is great, but persistent storage would be nice.

Additionally, it would be really nice if plugins could use this _internal database to store their own configuration, secrets, and settings. For example:

  • datasette-auth-tokens creates a _datasette_auth_tokens table to store auth token metadata. This could be moved into the _internal database to avoid writing to the gues database
  • datasette-socrata creates a socrata_imports table, which also can be in _internal
  • datasette-upload-csvs creates a _csv_progress_ table, which can be in _internal
  • datasette-write-ui wants to have the ability for users to toggle whether a table appears editable, which can be either in datasette.yaml or on-the-fly by storing config in _internal

In general, these are specific features that Datasette plugins would have access to if there was a central internal database they could read/write to:

  • Dynamic configuration. Changing the datasette.yaml file works, but can be tedious to restart the server every time. Plugins can define their own configuration table in _internal, and could read/write to it to store configuration based on user actions (cell menu click, API access, etc.)
  • Caching. If a plugin or Datasette Core needs to cache some expensive computation, they can store it inside _internal (possibly as a temporary table) instead of managing their own caching solution.
  • Audit logs. If a plugin performs some sensitive operations, they can log usage info to _internal for others to audit later.
  • Long running process status. Many plugins (datasette-upload-csvs, datasette-litestream, datasette-socrata) perform tasks that run for a really long time, and want to give continue status updates to the user. They can store this info inside_internal
  • Safer authentication. Passwords and authentication plugins usually store credentials/hashed secrets in configuration files or environment variables, which can be difficult to handle. Now, they can store them in _internal

Proposal

  • We remove _internal from datasette.databases property.
  • We add new datasette.get_internal_db() method that returns the _internal database, for plugins to use
  • We add a new --internal internal.db flag. If provided, then the _internal DB will be sourced from that file, and further updates will be persisted to that file (instead of an in-memory database)
  • When creating internal.db, create a new _datasette_internal table to mark it a an "datasette internal database"
  • In datasette serve, we check for the existence of the _datasette_internal table. If it exists, we assume the user provided that file in error and raise an error. This is to limit the chance that someone accidentally publishes their internal database to the internet. We could optionally add a --unsafe-allow-internal flag (or database plugin) that allows someone to do this if they really want to.

New features unlocked with this

These features don't really need a standardized _internal table per-say (plugins could currently configure their own long-time storage features if they really wanted to), but it would make it much simpler to create these kinds of features with a persistent application database.

  • datasette-comments : A plugin for commenting on rows or specific values in a database. Comment contents + threads + email notification info can be stored in _internal
  • Bookmarks: "Bookmarking" an SQL query could be stored in _internal, or a URL link shortener
  • Webhooks: If a plugin wants to either consume a webhook or create a new one, they can store hashed credentials/API endpoints in _internal
datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2157/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
742041667 MDU6SXNzdWU3NDIwNDE2Njc= 1092 Make cascading permission checks available to plugins simonw 9599 closed 0     1 2020-11-13T01:02:55Z 2023-08-30T22:17:42Z 2023-08-30T22:17:41Z OWNER  

The BaseView class has a method for cascading permission checks, but it's not easily accessible to plugins.

https://github.com/simonw/datasette/blob/5eb8e9bf250b26e30b017d39a392c33973997656/datasette/views/base.py#L75-L99

This leaves plugins like datasette-graphql having to implement their own versions of this logic, which is bad: https://github.com/simonw/datasette-graphql/issues/65

First check view-database - if that says False then disallow access, if it says True then allow access. If it says None check view-instance.

This should become a supported API that plugins are encouraged to use.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1092/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
594237015 MDU6SXNzdWU1OTQyMzcwMTU= 718 Plugin idea: datasette-redirects simonw 9599 open 0     0 2020-04-05T03:41:38Z 2023-08-30T22:17:31Z   OWNER  

I just had to write a one-off custom plugin to redirect niche-musems.com to www.niche-museums.com (https://github.com/simonw/museums/issues/21) - it would be great if this kind of thing could be handled by a configurable plugin.

https://github.com/simonw/museums/blob/6b1faf00c463b2228860d4d62d104b11935e01b1/plugins/redirect_www.py

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/718/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  reopened
787098146 MDU6SXNzdWU3ODcwOTgxNDY= 1190 `datasette publish upload` mechanism for uploading databases to an existing Datasette instance tomershvueli 1024355 closed 0     3 2021-01-15T18:18:42Z 2023-08-30T22:16:39Z 2023-08-30T22:16:38Z NONE  

If I have a self-hosted instance of Datasette up and running, I'd like to be able to the use the CLI to publish databases to that instance, not only Google or Heroku. Ideally there'd be a url parameter or something similar to which one could point the publish command to their instance.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1190/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1874327336 PR_kwDOBm6k_c5ZLMSe 2165 DATASETTE_LOAD_PLUGINS environment variable for loading specific plugins simonw 9599 closed 0     6 2023-08-30T20:33:30Z 2023-08-30T22:12:25Z 2023-08-30T22:12:25Z OWNER simonw/datasette/pulls/2165
  • 2164

TODO:

  • [x] Automated tests
  • [ ] Documentation
  • [x] Make sure DATASETTE_LOAD_PLUGINS='' works for loading zero plugins
datasette 107914493 pull    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2165/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
0  
1872043170 I_kwDOBm6k_c5vlRyi 2163 Rename core_X to catalog_X in the internals simonw 9599 closed 0     1 2023-08-29T16:45:00Z 2023-08-29T17:01:31Z 2023-08-29T17:01:31Z OWNER  

Discussed with Alex this morning. We think the American spelling is fine here (it's shorter than catalogue) and that it's a slightly less lazy name than core_.

Follows: - https://github.com/simonw/datasette/issues/2157

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2163/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1805076818 I_kwDOBm6k_c5rl0lS 2102 API tokens with view-table but not view-database/view-instance cannot access the table simonw 9599 closed 0 simonw 9599   20 2023-07-14T15:34:27Z 2023-08-29T16:32:36Z 2023-08-29T16:32:35Z OWNER  

Spotted a problem while working on this: if you grant a token access to view table for a specific table but don't also grant view database and view instance permissions, that token is useless.

This was a deliberate design decision in Datasette - it's documented on https://docs.datasette.io/en/1.0a2/authentication.html#access-permissions-in-metadata

If a user cannot access a specific database, they will not be able to access tables, views or queries within that database. If a user cannot access the instance they will not be able to access any of the databases, tables, views or queries.

I'm now second-guessing if this was a good decision.

Originally posted by @simonw in https://github.com/simonw/datasette-auth-tokens/issues/7#issuecomment-1636031702

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2102/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1865281760 PR_kwDOBm6k_c5Ys3C5 2154 Cascade for restricted token view-table/view-database/view-instance operations simonw 9599 closed 0     8 2023-08-24T14:24:23Z 2023-08-29T16:32:35Z 2023-08-29T16:32:34Z OWNER simonw/datasette/pulls/2154

Refs: - #2102

Also includes a prototype implementation of --actor option which I'm using for testing this, from: - #2153


:books: Documentation preview :books:: https://datasette--2154.org.readthedocs.build/en/2154/

datasette 107914493 pull    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2154/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
0  
1870672704 PR_kwDOBm6k_c5Y-7Em 2162 Add new `--internal internal.db` option, deprecate legacy `_internal` database asg017 15178711 closed 0     4 2023-08-29T00:05:07Z 2023-08-29T03:24:23Z 2023-08-29T03:24:23Z CONTRIBUTOR simonw/datasette/pulls/2162

refs #2157

This PR adds a new --internal option to datasette serve. If provided, it is the path to a persistent internal database that Datasette core and Datasette plugins can use to store data, as discussed in the proposal issue.

This PR also removes and deprecates the previous in-memory _internal database. Those tables now appear in the internal database, with core_ prefixes (ex tables in _internal is now core_tables in internal).

A note on the new core_ tables

However, one important notes about those new core_ tables: If a --internal DB is passed in, that means those core_ tables will persist across multiple Datasette instances. This wasn't the case before, since _internal was always an in-memory database created from scratch.

I tried to put those core_ tables as TEMP tables - after all, there's always one 1 internal DB connection at a time, so I figured it would work. But, since we use the Database() wrapper for the internal DB, it has two separate connections: a default read-only connection and a write connection that is created when a write operation occurs. Which meant the TEMP tables would be created by the write connection, but not available in the read-only connection.

So I had a brillant idea: Attach an in-memory named database with cache=shared, and create those tables there!

sql ATTACH DATABASE 'file:datasette_internal_core?mode=memory&cache=shared' AS core;

We'd run this on both the read-only connection and the write-only connection. That way, those tables would stay in memory, they'd communicate with the cache=shared feature, and we'd be good to go.

However, I couldn't find an easy way to run a ATTACH DATABASE command on the read-only query.

Using Database() as a wrapper for the internal DB is pretty limiting - it's meant for Datasette "data" databases, where we want multiple readers and possibly 1 write connection at a time. But the internal database doesn't really require that kind of support - I think we could get away with a single read/write connection, but it seemed like too big of a rabbithole to go through now.


:books: Documentation preview :books:: https://datasette--2162.org.readthedocs.build/en/2162/

datasette 107914493 pull    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2162/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
0  
1869807874 PR_kwDOBm6k_c5Y8AN0 2160 Bump sphinx, furo, blacken-docs dependencies dependabot[bot] 49699333 closed 0     5 2023-08-28T13:49:31Z 2023-08-29T00:38:33Z 2023-08-29T00:38:32Z CONTRIBUTOR simonw/datasette/pulls/2160

Bumps the python-packages group with 3 updates: sphinx, furo and blacken-docs.

Updates sphinx from 7.1.2 to 7.2.4

Release notes

Sourced from sphinx's releases.

Sphinx 7.2.4

Changelog: https://www.sphinx-doc.org/en/master/changes.html

Sphinx 7.2.3

Changelog: https://www.sphinx-doc.org/en/master/changes.html

Sphinx 7.2.2

Changelog: https://www.sphinx-doc.org/en/master/changes.html

Sphinx 7.2.1

Changelog: https://www.sphinx-doc.org/en/master/changes.html

Sphinx 7.2.0

Changelog: https://www.sphinx-doc.org/en/master/changes.html

Changelog

Sourced from sphinx's changelog.

Release 7.2.4 (released Aug 28, 2023)

Bugs fixed

  • #11618: Fix a regression in the MoveModuleTargets transform, introduced in #10478 (#9662).
  • #11649: linkcheck: Resolve hanging tests for timezones west of London and incorrect conversion from UTC to offsets from the UNIX epoch. Patch by Dmitry Shachnev and Adam Turner.

Release 7.2.3 (released Aug 23, 2023)

Dependencies

  • #11576: Require sphinxcontrib-serializinghtml 1.1.9.

Bugs fixed

  • Fix regression in autodoc.Documenter.parse_name().
  • Fix regression in JSON serialisation.
  • #11543: autodoc: Support positional-only parameters in classmethod methods when autodoc_preserve_defaults is True.
  • Restore support string methods on path objects. This is deprecated and will be removed in Sphinx 8. Use :py:func:os.fspath to convert :py:class:~pathlib.Path objects to strings, or :py:class:~pathlib.Path's methods to work with path objects.

Release 7.2.2 (released Aug 17, 2023)

Bugs fixed

  • Fix the signature of the StateMachine.insert_input() patch, for when calling with keyword arguments.
  • Fixed membership testing (in) for the :py:class:str interface of the asset classes (_CascadingStyleSheet and _JavaScript), which several extensions relied upon.
  • Fixed a type error in SingleFileHTMLBuilder._get_local_toctree, includehidden may be passed as a string or a boolean.
  • Fix :noindex: for PyModule and JSModule.

Release 7.2.1 (released Aug 17, 2023)

... (truncated)

Commits
  • 3256f1f Bump to 7.2.4 final
  • 2f025a4 linkcheck: Fix conversion from UTC time to the UNIX epoch (#11649)
  • 1567281 autodoc: Fix UnboundLocalError in filter_members (#11651)
  • 5e88b9f Fix the MoveModuleTargets transform (#11647)
  • 694fcee Fix markup in CHANGES (#11639)
  • c503c90 Improve pathlib type annotations (#11646)
  • bf339b1 Bump version
  • 2f6ea14 Bump to 7.2.3 final
  • 511e407 Implement bool() for string paths
  • 494de73 Implement hash() for string paths
  • Additional commits viewable in compare view


Updates furo from 2023.7.26 to 2023.8.19

Changelog

Sourced from furo's changelog.

Changelog

2023.08.19 -- Xenolithic Xanadu

  • Fix missing search context with Sphinx 7.2, for dirhtml builds.
  • Drop support for Python 3.7.
  • Present configuration errors in a better format -- thanks @​AA-Turner!
  • Bump require_sphinx() to Sphinx 6.0, in line with dependency changes in Unassuming Ultramarine.

2023.08.17 -- Wonderous White

  • Fix compatiblity with Sphinx 7.2.0 and 7.2.1.

2023.07.26 -- Vigilant Volt

  • Fix compatiblity with Sphinx 7.1.
  • Improve how content overflow is handled.
  • Improve how literal blocks containing inline code are handled.

2023.05.20 -- Unassuming Ultramarine

  • ✨ Add support for Sphinx 7.
  • Drop support for Sphinx 5.
  • Improve the screen-reader label for sidebar collapse.
  • Make it easier to create derived themes from Furo.
  • Bump all JS dependencies (NodeJS and npm packages).

2023.03.27 -- Tasty Tangerine

  • Regenerate with newer version of sphinx-theme-builder, to fix RECORD hashes.
  • Add missing class to Font Awesome examples

2023.03.23 -- Sassy Saffron

  • Update Python version classifiers.
  • Increase the icon size in mobile header.
  • Increase admonition title bg opacity.
  • Change the default API background to transparent.
  • Transition the API background change.

... (truncated)

Commits
  • 0766bb2 Prepare release: 2023.08.19
  • 807d73c Update changelog
  • 364b261 Accomodate for the required data-content_root for search
  • 0d38bc6 Simplify retrieval of pygments_dark_style value (#699)
  • 3631ffc Use sphinx.errors.ConfigError (#697)
  • d2e2448 Remove duplicate HTML builder check (#698)
  • 7b4f130 Drop Python 3.7 (#701)
  • e322b71 Remove pointless assert (#702)
  • ee2097a Bump require_sphinx() to Sphinx 6.0 (#700)
  • c1ff10b Back to development
  • Additional commits viewable in compare view


Updates blacken-docs from 1.15.0 to 1.16.0

Changelog

Sourced from blacken-docs's changelog.

1.16.0 (2023-08-16)

  • Allow Markdown fence options.

    Thanks to initial work from Matthew Anderson in PR [#246](https://github.com/asottile/blacken-docs/issues/246) <https://github.com/adamchainz/blacken-docs/pull/246>__.

  • Expand Markdown detection to all Python language names from Pygments: py, sage, python3, py3, and numpy.

  • Preserve leading whitespace lines in reStructuredText code blocks.

    Thanks to Julianus Pfeuffer for the report in Issue [#217](https://github.com/asottile/blacken-docs/issues/217) <https://github.com/adamchainz/blacken-docs/issues/217>__.

  • Use exit code 2 to indicate errors from Black, whilst exit code 1 remains for “files have been formatted”.

    Thanks to Julianus Pfeuffer for the report in Issue [#218](https://github.com/asottile/blacken-docs/issues/218) <https://github.com/adamchainz/blacken-docs/issues/218>__.

  • Support passing the --preview option through to Black, to select the future style.

  • Remove language_version from .pre-commit-hooks.yaml. This change allows default_language_version in ``.pre-commit-config.yaml` to take precedence.

    Thanks to Aneesh Agrawal in PR [#258](https://github.com/asottile/blacken-docs/issues/258) <https://github.com/adamchainz/blacken-docs/pull/258>__.

Commits
  • 960ead2 Version 1.16.0
  • 8f0ed18 Support passing --preview through to Black (#273)
  • 4eb4e4c Tweak changelog note
  • 6c7450c Use exit code 2 to indicate errors (#272)
  • 99dfc8d Preserve leading whitespace lines in rST (#271)
  • 94465e8 Reformat markdown tests with dedent() (#270)
  • 7cd5f30 Use .md in glob example
  • f97e569 Document applying to many files (#269)
  • ae612b0 Expand Markdown detection to all Python language names (#268)
  • da9b455 Replace NamedTuple with plain class (#267)
  • Additional commits viewable in compare view


Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore <dependency name> major version` will close this group update PR and stop Dependabot creating any more for the specific dependency's major version (unless you unignore this specific dependency's major version or upgrade to it yourself) - `@dependabot ignore <dependency name> minor version` will close this group update PR and stop Dependabot creating any more for the specific dependency's minor version (unless you unignore this specific dependency's minor version or upgrade to it yourself) - `@dependabot ignore <dependency name>` will close this group update PR and stop Dependabot creating any more for the specific dependency (unless you unignore this specific dependency or upgrade to it yourself) - `@dependabot unignore <dependency name>` will remove all of the ignore conditions of the specified dependency - `@dependabot unignore <dependency name> <ignore condition>` will remove the ignore condition of the specified dependency and ignore conditions

:books: Documentation preview :books:: https://datasette--2160.org.readthedocs.build/en/2160/

datasette 107914493 pull    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2160/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
0  
1859415334 PR_kwDOBm6k_c5YY5Ea 2148 Bump sphinx, furo, blacken-docs dependencies dependabot[bot] 49699333 closed 0     9 2023-08-21T13:48:11Z 2023-08-29T00:15:31Z 2023-08-29T00:15:27Z CONTRIBUTOR simonw/datasette/pulls/2148

Bumps the python-packages group with 3 updates: sphinx, furo and blacken-docs.

Updates sphinx from 7.1.2 to 7.2.2

Release notes

Sourced from sphinx's releases.

Sphinx 7.2.2

Changelog: https://www.sphinx-doc.org/en/master/changes.html

Sphinx 7.2.1

Changelog: https://www.sphinx-doc.org/en/master/changes.html

Sphinx 7.2.0

Changelog: https://www.sphinx-doc.org/en/master/changes.html

Changelog

Sourced from sphinx's changelog.

Release 7.2.2 (released Aug 17, 2023)

Bugs fixed

  • Fix the signature of the StateMachine.insert_input() patch, for when calling with keyword arguments.
  • Fixed membership testing (in) for the :py:class:str interface of the asset classes (_CascadingStyleSheet and _JavaScript), which several extensions relied upon.
  • Fixed a type error in SingleFileHTMLBuilder._get_local_toctree, includehidden may be passed as a string or a boolean.
  • Fix :noindex: for PyModule and JSModule``.

Release 7.2.1 (released Aug 17, 2023)

Bugs fixed

  • Restored the the :py:class:str interface of the asset classes (_CascadingStyleSheet and _JavaScript), which several extensions relied upon. This will be removed in Sphinx 9.
  • Restored calls to Builder.add_{css,js}_file(), which several extensions relied upon.
  • Restored the private API TocTree.get_toctree_ancestors(), which several extensions relied upon.

Release 7.2.0 (released Aug 17, 2023)

Dependencies

  • #11511: Drop Python 3.8 support.
  • #11576: Require Pygments 2.14 or later.

Deprecated

  • #11512: Deprecate sphinx.util.md5 and sphinx.util.sha1. Use hashlib instead.
  • #11526: Deprecate sphinx.testing.path. Use os.path or pathlib instead.
  • #11528: Deprecate sphinx.util.split_index_msg and sphinx.util.split_into. Use sphinx.util.index_entries.split_index_msg instead.
  • Deprecate sphinx.builders.html.Stylesheet and sphinx.builders.html.Javascript. Use sphinx.application.Sphinx.add_css_file()

... (truncated)

Commits
  • ed84d63 Bump to 7.2.2 final
  • ea4a73e [bot]: Update message catalogues (#11612)
  • e47846a Fix :noindex: for PyModule and JSModule``
  • b2fc47f Add CHANGES entry for renaming the StateMachine.insert_input() parameter
  • 0835c3e Fix regression in SingleFileHTMLBuilder._get_local_toctree
  • 49dc0dd Fix asset class string interface membership testing
  • 8512855 Fix signature of docutils include_source monkeypatch (#11610)
  • e1d9068 Bump version
  • 441a9e4 Bump to 7.2.1 final
  • ec31853 Restore TocTree.get_toctree_ancestors()
  • Additional commits viewable in compare view


Updates furo from 2023.7.26 to 2023.8.19

Changelog

Sourced from furo's changelog.

Changelog

2023.08.19 -- Xenolithic Xanadu

  • Fix missing search context with Sphinx 7.2, for dirhtml builds.
  • Drop support for Python 3.7.
  • Present configuration errors in a better format -- thanks @​AA-Turner!
  • Bump require_sphinx() to Sphinx 6.0, in line with dependency changes in Unassuming Ultramarine.

2023.08.17 -- Wonderous White

  • Fix compatiblity with Sphinx 7.2.0 and 7.2.1.

2023.07.26 -- Vigilant Volt

  • Fix compatiblity with Sphinx 7.1.
  • Improve how content overflow is handled.
  • Improve how literal blocks containing inline code are handled.

2023.05.20 -- Unassuming Ultramarine

  • ✨ Add support for Sphinx 7.
  • Drop support for Sphinx 5.
  • Improve the screen-reader label for sidebar collapse.
  • Make it easier to create derived themes from Furo.
  • Bump all JS dependencies (NodeJS and npm packages).

2023.03.27 -- Tasty Tangerine

  • Regenerate with newer version of sphinx-theme-builder, to fix RECORD hashes.
  • Add missing class to Font Awesome examples

2023.03.23 -- Sassy Saffron

  • Update Python version classifiers.
  • Increase the icon size in mobile header.
  • Increase admonition title bg opacity.
  • Change the default API background to transparent.
  • Transition the API background change.

... (truncated)

Commits
  • 0766bb2 Prepare release: 2023.08.19
  • 807d73c Update changelog
  • 364b261 Accomodate for the required data-content_root for search
  • 0d38bc6 Simplify retrieval of pygments_dark_style value (#699)
  • 3631ffc Use sphinx.errors.ConfigError (#697)
  • d2e2448 Remove duplicate HTML builder check (#698)
  • 7b4f130 Drop Python 3.7 (#701)
  • e322b71 Remove pointless assert (#702)
  • ee2097a Bump require_sphinx() to Sphinx 6.0 (#700)
  • c1ff10b Back to development
  • Additional commits viewable in compare view


Updates blacken-docs from 1.15.0 to 1.16.0

Changelog

Sourced from blacken-docs's changelog.

1.16.0 (2023-08-16)

  • Allow Markdown fence options.

    Thanks to initial work from Matthew Anderson in PR [#246](https://github.com/asottile/blacken-docs/issues/246) <https://github.com/adamchainz/blacken-docs/pull/246>__.

  • Expand Markdown detection to all Python language names from Pygments: py, sage, python3, py3, and numpy.

  • Preserve leading whitespace lines in reStructuredText code blocks.

    Thanks to Julianus Pfeuffer for the report in Issue [#217](https://github.com/asottile/blacken-docs/issues/217) <https://github.com/adamchainz/blacken-docs/issues/217>__.

  • Use exit code 2 to indicate errors from Black, whilst exit code 1 remains for “files have been formatted”.

    Thanks to Julianus Pfeuffer for the report in Issue [#218](https://github.com/asottile/blacken-docs/issues/218) <https://github.com/adamchainz/blacken-docs/issues/218>__.

  • Support passing the --preview option through to Black, to select the future style.

  • Remove language_version from .pre-commit-hooks.yaml. This change allows default_language_version in ``.pre-commit-config.yaml` to take precedence.

    Thanks to Aneesh Agrawal in PR [#258](https://github.com/asottile/blacken-docs/issues/258) <https://github.com/adamchainz/blacken-docs/pull/258>__.

Commits
  • 960ead2 Version 1.16.0
  • 8f0ed18 Support passing --preview through to Black (#273)
  • 4eb4e4c Tweak changelog note
  • 6c7450c Use exit code 2 to indicate errors (#272)
  • 99dfc8d Preserve leading whitespace lines in rST (#271)
  • 94465e8 Reformat markdown tests with dedent() (#270)
  • 7cd5f30 Use .md in glob example
  • f97e569 Document applying to many files (#269)
  • ae612b0 Expand Markdown detection to all Python language names (#268)
  • da9b455 Replace NamedTuple with plain class (#267)
  • Additional commits viewable in compare view


Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore <dependency name> major version` will close this group update PR and stop Dependabot creating any more for the specific dependency's major version (unless you unignore this specific dependency's major version or upgrade to it yourself) - `@dependabot ignore <dependency name> minor version` will close this group update PR and stop Dependabot creating any more for the specific dependency's minor version (unless you unignore this specific dependency's minor version or upgrade to it yourself) - `@dependabot ignore <dependency name> dependency` will close this group update PR and stop Dependabot creating any more for the specific dependency (unless you unignore this specific dependency or upgrade to it yourself) - `@dependabot unignore <dependency name> dependency` will remove all of the ignore conditions of the specified dependency - `@dependabot unignore <dependency name> <ignore condition>` will remove the ignore condition of the specified dependency and ignore conditions

:books: Documentation preview :books:: https://datasette--2148.org.readthedocs.build/en/2148/

datasette 107914493 pull    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2148/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
0  
1865232341 I_kwDOBm6k_c5vLS_V 2153 Datasette --get --actor option simonw 9599 closed 0     5 2023-08-24T14:00:03Z 2023-08-28T20:19:15Z 2023-08-28T20:15:53Z OWNER  

I experimented with a prototype of this here: - https://github.com/simonw/datasette/issues/2102#issuecomment-1691037971_

Which lets me run requests as if they belonged to a specific actor like this:

bash datasette fixtures.db --get '/fixtures/facetable.json' --actor '{ "_r": { "r": { "fixtures": { "facetable": [ "vt" ] } } }, "a": "user" }'

Really useful for testing actors an _r options. Is this worth adding as a feature?

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2153/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1870345352 PR_kwDOBm6k_c5Y90K9 2161 -s/--setting x y gets merged into datasette.yml, refs #2143, #2156 simonw 9599 closed 0     1 2023-08-28T19:30:42Z 2023-08-28T20:06:15Z 2023-08-28T20:06:14Z OWNER simonw/datasette/pulls/2161

This change updates the -s/--setting option to datasette serve to allow it to be used to set arbitrarily complex nested settings in a way that is compatible with the new -c datasette.yml work happening in: - #2143

It will enable things like this: datasette data.db --setting plugins.datasette-ripgrep.path "/home/simon/code" For the moment though it just affects settings - so you can do this: datasette data.db --setting settings.sql_time_limit_ms 3500 I've also implemented a backwards compatibility mechanism, so if you use it this way (the old way): datasette data.db --setting sql_time_limit_ms 3500 It will notice that the setting you passed is one of Datasette's core settings, and will treat that as if you said settings.sql_time_limit_ms instead.


:books: Documentation preview :books:: https://datasette--2161.org.readthedocs.build/en/2161/

datasette 107914493 pull    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2161/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
0  
1865649347 I_kwDOBm6k_c5vM4zD 2156 datasette -s/--setting option for setting nested configuration options simonw 9599 open 0     4 2023-08-24T18:09:27Z 2023-08-28T19:33:05Z   OWNER  

I've been thinking about what it might look like to allow command-line arguments to be used to define any of the configuration options in datasette.yml, as alternative and more convenient syntax.

Here's what I've come up with: datasette \ -s settings.sql_time_limit_ms 1000 \ -s plugins.datasette-auth-tokens.manage_tokens true \ -s plugins.datasette-auth-tokens.manage_tokens_database tokens \ mydatabase.db tokens.db Which would be equivalent to datasette.yml containing this: yaml plugins: datasette-auth-tokens: manage_tokens: true manage_tokens_database: tokens settings: sql_time_limit_ms: 1000 More details in https://github.com/simonw/datasette/issues/2143#issuecomment-1690792514

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2156/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1865174661 PR_kwDOBm6k_c5YsfZ7 2152 Bump the python-packages group with 3 updates dependabot[bot] 49699333 closed 0     3 2023-08-24T13:34:44Z 2023-08-28T13:49:39Z 2023-08-28T13:49:37Z CONTRIBUTOR simonw/datasette/pulls/2152

Bumps the python-packages group with 3 updates: sphinx, furo and blacken-docs.

Updates sphinx from 7.1.2 to 7.2.3

Release notes

Sourced from sphinx's releases.

Sphinx 7.2.3

Changelog: https://www.sphinx-doc.org/en/master/changes.html

Sphinx 7.2.2

Changelog: https://www.sphinx-doc.org/en/master/changes.html

Sphinx 7.2.1

Changelog: https://www.sphinx-doc.org/en/master/changes.html

Sphinx 7.2.0

Changelog: https://www.sphinx-doc.org/en/master/changes.html

Changelog

Sourced from sphinx's changelog.

Release 7.2.3 (released Aug 23, 2023)

Dependencies

  • #11576: Require sphinxcontrib-serializinghtml 1.1.9.

Bugs fixed

  • Fix regression in autodoc.Documenter.parse_name().
  • Fix regression in JSON serialisation.
  • #11543: autodoc: Support positional-only parameters in classmethod methods when autodoc_preserve_defaults is True.
  • Restore support string methods on path objects. This is deprecated and will be removed in Sphinx 8. Use :py:funcos.fspath to convert :py:class:<del>pathlib.Path objects to strings, or :py:class:</del>pathlib.Path's methods to work with path objects.

Release 7.2.2 (released Aug 17, 2023)

Bugs fixed

  • Fix the signature of the StateMachine.insert_input() patch, for when calling with keyword arguments.
  • Fixed membership testing (in) for the :py:class:str interface of the asset classes (_CascadingStyleSheet and _JavaScript), which several extensions relied upon.
  • Fixed a type error in SingleFileHTMLBuilder._get_local_toctree, includehidden may be passed as a string or a boolean.
  • Fix :noindex: for PyModule and JSModule``.

Release 7.2.1 (released Aug 17, 2023)

Bugs fixed

  • Restored the the :py:class:str interface of the asset classes (_CascadingStyleSheet and _JavaScript), which several extensions relied upon. This will be removed in Sphinx 9.
  • Restored calls to Builder.add_{css,js}_file(), which several extensions relied upon.
  • Restored the private API TocTree.get_toctree_ancestors(), which several extensions relied upon.

Release 7.2.0 (released Aug 17, 2023)

... (truncated)

Commits
  • 2f6ea14 Bump to 7.2.3 final
  • 511e407 Implement bool() for string paths
  • 494de73 Implement hash() for string paths
  • 2986aa1 Override special methods for string paths
  • 07b87e9 Update CHANGES for 7.2.3
  • 6b17dd1 Support string methods on path objects (#11619)
  • a73fb59 Support positional-only parameters in classmethods (#11635)
  • 02cb02c Fix invocation of python -m sphinx build
  • 6183b6a Require sphinxcontrib-serializinghtml 1.1.9 or later
  • 1e16f21 Fix regression in autodoc.Documenter.parse_name (#11613)
  • Additional commits viewable in compare view


Updates furo from 2023.7.26 to 2023.8.19

Changelog

Sourced from furo's changelog.

Changelog

2023.08.19 -- Xenolithic Xanadu

  • Fix missing search context with Sphinx 7.2, for dirhtml builds.
  • Drop support for Python 3.7.
  • Present configuration errors in a better format -- thanks @​AA-Turner!
  • Bump require_sphinx() to Sphinx 6.0, in line with dependency changes in Unassuming Ultramarine.

2023.08.17 -- Wonderous White

  • Fix compatiblity with Sphinx 7.2.0 and 7.2.1.

2023.07.26 -- Vigilant Volt

  • Fix compatiblity with Sphinx 7.1.
  • Improve how content overflow is handled.
  • Improve how literal blocks containing inline code are handled.

2023.05.20 -- Unassuming Ultramarine

  • ✨ Add support for Sphinx 7.
  • Drop support for Sphinx 5.
  • Improve the screen-reader label for sidebar collapse.
  • Make it easier to create derived themes from Furo.
  • Bump all JS dependencies (NodeJS and npm packages).

2023.03.27 -- Tasty Tangerine

  • Regenerate with newer version of sphinx-theme-builder, to fix RECORD hashes.
  • Add missing class to Font Awesome examples

2023.03.23 -- Sassy Saffron

  • Update Python version classifiers.
  • Increase the icon size in mobile header.
  • Increase admonition title bg opacity.
  • Change the default API background to transparent.
  • Transition the API background change.

... (truncated)

Commits
  • 0766bb2 Prepare release: 2023.08.19
  • 807d73c Update changelog
  • 364b261 Accomodate for the required data-content_root for search
  • 0d38bc6 Simplify retrieval of pygments_dark_style value (#699)
  • 3631ffc Use sphinx.errors.ConfigError (#697)
  • d2e2448 Remove duplicate HTML builder check (#698)
  • 7b4f130 Drop Python 3.7 (#701)
  • e322b71 Remove pointless assert (#702)
  • ee2097a Bump require_sphinx() to Sphinx 6.0 (#700)
  • c1ff10b Back to development
  • Additional commits viewable in compare view


Updates blacken-docs from 1.15.0 to 1.16.0

Changelog

Sourced from blacken-docs's changelog.

1.16.0 (2023-08-16)

  • Allow Markdown fence options.

    Thanks to initial work from Matthew Anderson in PR [#246](https://github.com/asottile/blacken-docs/issues/246) <https://github.com/adamchainz/blacken-docs/pull/246>__.

  • Expand Markdown detection to all Python language names from Pygments: py, sage, python3, py3, and numpy.

  • Preserve leading whitespace lines in reStructuredText code blocks.

    Thanks to Julianus Pfeuffer for the report in Issue [#217](https://github.com/asottile/blacken-docs/issues/217) <https://github.com/adamchainz/blacken-docs/issues/217>__.

  • Use exit code 2 to indicate errors from Black, whilst exit code 1 remains for “files have been formatted”.

    Thanks to Julianus Pfeuffer for the report in Issue [#218](https://github.com/asottile/blacken-docs/issues/218) <https://github.com/adamchainz/blacken-docs/issues/218>__.

  • Support passing the --preview option through to Black, to select the future style.

  • Remove language_version from .pre-commit-hooks.yaml. This change allows default_language_version in ``.pre-commit-config.yaml` to take precedence.

    Thanks to Aneesh Agrawal in PR [#258](https://github.com/asottile/blacken-docs/issues/258) <https://github.com/adamchainz/blacken-docs/pull/258>__.

Commits
  • 960ead2 Version 1.16.0
  • 8f0ed18 Support passing --preview through to Black (#273)
  • 4eb4e4c Tweak changelog note
  • 6c7450c Use exit code 2 to indicate errors (#272)
  • 99dfc8d Preserve leading whitespace lines in rST (#271)
  • 94465e8 Reformat markdown tests with dedent() (#270)
  • 7cd5f30 Use .md in glob example
  • f97e569 Document applying to many files (#269)
  • ae612b0 Expand Markdown detection to all Python language names (#268)
  • da9b455 Replace NamedTuple with plain class (#267)
  • Additional commits viewable in compare view


Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore <dependency name> major version` will close this group update PR and stop Dependabot creating any more for the specific dependency's major version (unless you unignore this specific dependency's major version or upgrade to it yourself) - `@dependabot ignore <dependency name> minor version` will close this group update PR and stop Dependabot creating any more for the specific dependency's minor version (unless you unignore this specific dependency's minor version or upgrade to it yourself) - `@dependabot ignore <dependency name>` will close this group update PR and stop Dependabot creating any more for the specific dependency (unless you unignore this specific dependency or upgrade to it yourself) - `@dependabot unignore <dependency name>` will remove all of the ignore conditions of the specified dependency - `@dependabot unignore <dependency name> <ignore condition>` will remove the ignore condition of the specified dependency and ignore conditions

:books: Documentation preview :books:: https://datasette--2152.org.readthedocs.build/en/2152/

datasette 107914493 pull    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2152/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
0  
1866815458 PR_kwDOBm6k_c5YyF-C 2159 Implement Dark Mode colour scheme jamietanna 3315059 open 0     0 2023-08-25T10:46:23Z 2023-08-25T10:46:35Z   FIRST_TIME_CONTRIBUTOR simonw/datasette/pulls/2159

Closes #2095.


:books: Documentation preview :books:: https://datasette--2159.org.readthedocs.build/en/2159/

datasette 107914493 pull    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2159/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
1  
1865983069 PR_kwDOBm6k_c5YvQSi 2158 add brand option to metadata.json. publicmatt 52261150 open 0     0 2023-08-24T22:37:41Z 2023-08-24T22:37:57Z   FIRST_TIME_CONTRIBUTOR simonw/datasette/pulls/2158

This adds a brand link to the top navbar if 'brand' key is populated in metadata.json. The link will be either '#' or use the contents of 'brand_url' in metadata.json for href.

I was able to get this done on my own site by replacing templates/_crumbs.html with a custom version, but I thought it would be nice to incorporate this in the tool directly.


:books: Documentation preview :books:: https://datasette--2158.org.readthedocs.build/en/2158/

datasette 107914493 pull    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2158/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
0  
685806511 MDU6SXNzdWU2ODU4MDY1MTE= 950 Private/secret databases: database files that are only visible to plugins simonw 9599 closed 0     6 2020-08-25T20:46:17Z 2023-08-24T22:26:09Z 2023-08-24T22:26:08Z OWNER  

In thinking about the best way to implement https://github.com/simonw/datasette-auth-passwords/issues/6 (SQL-backed user accounts for datasette-auth-passwords) I realized that there are a few different use-cases where a plugin might want to store data that isn't visible to regular Datasette users:

  • Storing password hashes
  • Storing API tokens
  • Storing secrets that are used for data import integrations (secrets for talking to the Twitter API for example)

Idea: allow one or more private database files to be attached to Datasette, something like this:

datasette github.db linkedin.db -s secrets.db -m metadata.yml

The secrets.db file would not be visible using any of the Datasette's usual interface or API routes - but plugins would be able to run queries against it.

So datasette-auth-passwords might then be configured like this:

yaml plugins: datasette-auth-passwords: database: secrets sql: "select password_hash from passwords where username = :username" The plugin could even refuse to operate against a database that hadn't been loaded as a secret database.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/950/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
814595021 MDU6SXNzdWU4MTQ1OTUwMjE= 1241 Share button for copying current URL Kabouik 7107523 open 0     6 2021-02-23T15:55:40Z 2023-08-24T20:09:52Z   NONE  

I use datasette in an iframe inside another HTML file that contains other ways to represent my data (mostly leaflets maps built with R on summarized data), and the datasette iframe is a tab in that page.

This particular use prevents users to access the full URLs of their datasette views and queries, which is a shame because the way datasette handles URLs to make every view or query easy to share is awesome. I know how to get the URL from the context menu of my browser, but I don't think many visitors would do it or even notice that datasette uses permalinks for pretty much every action they do. Would it be possible to add a "Share link" button to the interface, either in datasette itself or in a plugin?

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1241/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1855885427 I_kwDOBm6k_c5unpBz 2143 De-tangling Metadata before Datasette 1.0 asg017 15178711 open 0     24 2023-08-18T00:51:50Z 2023-08-24T18:28:27Z   CONTRIBUTOR  

Metadata in Datasette is a really powerful feature, but is a bit difficult to work with. It was initially a way to add "metadata" about your "data" in Datasette instances, like descriptions for databases/tables/columns, titles, source URLs, licenses, etc. But it later became the go-to spot for other Datasette features that have nothing to do with metadata, like permissions/plugins/canned queries.

Specifically, I've found the following problems when working with Datasette metadata:

  1. Metadata cannot be updated without re-starting the entire Datasette instance.
  2. The metadata.json/metadata.yaml has become a kitchen sink of unrelated (imo) features like plugin config, authentication config, canned queries
  3. The Python APIs for defining extra metadata are a bit awkward (the datasette.metadata() class, get_metadata() hook, etc.)

Possible solutions

Here's a few ideas of Datasette core changes we can make to address these problems.

Re-vamp the Datasette Python metadata APIs

The Datasette object has a single datasette.metadata() method that's a bit difficult to work with. There's also no Python API for inserted new metadata, so plugins have to rely on the get_metadata() hook.

The get_metadata() hook can also be improved - it doesn't work with async functions yet, so you're quite limited to what you can do.

(I'm a bit fuzzy on what to actually do here, but I imagine it'll be very small breaking changes to a few Python methods)

Add an optional datasette_metadata table

Datasette should detect and use metadata stored in a new special table called datasette_metadata. This would be a regular table that a user can edit on their own, and would serve as a "live updating" source of metadata, than can be changed while the Datasette instance is running.

Not too sure what the schema would look like, but I'd imagine:

sql CREATE TABLE datasette_metadata( level text, target any, key text, value any, primary key (level, target) )

Every row in this table would map to a single metadata "entry".

  • level would be one of "datasette", "database", "table", "column", which is the "level" the entry describes. For example, level="table" means it is metadata about a specific table, level="database" for a specific database, or level="datasette" for the entire Datasette instance.
  • target would "point" to the specific object the entry metadata is about, and would depend on what level is specific.
  • level="database": target would be the string name of the database that the metadata entry is about. ex "fixtures"
  • level="table": target would be a JSON array of two strings. The first element would be the database name, and the second would be the table name. ex ["fixtures", "students"]
  • level="column": target would be a JSON array of 3 strings: The database name, table name, and column name. Ex ["fixtures", "students", "student_id"]
  • key would be the type of metadata entry the row has, similar to the current "keys" that exist in metadata.json. Ex "about_url", "source", "description", etc
  • value would be the text value of be metadata entry. The literal text value of a description, about_url, column_label, etc

A quick sample:

level | target | key | value -- | -- | -- | -- datasette | NULL | title | my datasette title... db | fixtures | source | <description of my database source> table | ["fixtures", "students"] | label_column | student_name column | ["fixtures", "students", "birthdate"] | description | <description of the fixtures.students.birthdate column>

This datasette_metadata would be configured with other tools, and hopefully not manually by end users. Datasette Core could also offer a UI for editing entries in datasette_metadata, to update descriptions/columns on the fly.

Re-vamp metadata.json and move non-metadata config to another place

The motivation behind this is that it's awkward that metadata.json contains config about things that are not strictly metadata, including:

  • Plugin configuration
  • Authentication/permissions (ex the allow key on datasettes/databases/tables
  • Canned queries. might be controversial, but in my mind, canned queries are application-specific code and configuration, and don't describe the data that exists in SQLite databases.

I think we should move these outside of metadata.json and into a different file. The datasette.json idea in #2093 may be a good solution here: plugin/permissions/canned queries can be defined in datasette.json, while metadata.json/datasette_metadata will strictly be about documenting databases/tables/columns.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2143/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1865572575 PR_kwDOBm6k_c5Yt2eO 2155 Fix hupper.start_reloader entry point cadeef 79087 open 0     0 2023-08-24T17:14:08Z 2023-08-24T17:14:20Z   FIRST_TIME_CONTRIBUTOR simonw/datasette/pulls/2155

Update hupper's entry point so that click commands are processed properly.

Fixes #2123


:books: Documentation preview :books:: https://datasette--2155.org.readthedocs.build/en/2155/

datasette 107914493 pull    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2155/reactions",
    "total_count": 1,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 1,
    "eyes": 0
}
0  
1858228057 I_kwDOBm6k_c5uwk9Z 2147 Plugin hook for database queries that are run jackowayed 18899 open 0     6 2023-08-20T18:43:50Z 2023-08-24T03:54:35Z   NONE  

I'm interested in making a plugin that saves every query that gets run to a table in the database. (I know about datasette-query-history but thought it would be good to have a server-side option.)

As far as I can tell reading the docs, there isn't really a hook setup to allow this.

Maybe I could hack it with some of the hooks that are passed requests, but that doesn't seem good.

I'm a little surprised this isn't possible, so I thought I would open an issue and see if that's a deeply considered decision or just "haven't needed it yet." I'm potentially interested in implementing the hook if the latter.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2147/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1864112887 PR_kwDOBm6k_c5Yo7bk 2151 Test Datasette on multiple SQLite versions asg017 15178711 open 0     1 2023-08-23T22:42:51Z 2023-08-23T22:58:13Z   CONTRIBUTOR simonw/datasette/pulls/2151

still testing, hope it works!


:books: Documentation preview :books:: https://datasette--2151.org.readthedocs.build/en/2151/

datasette 107914493 pull    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2151/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
1  
1863810783 I_kwDOBm6k_c5vF37f 2150 form label { width: 15% } is a bad default simonw 9599 closed 0     4 2023-08-23T18:22:27Z 2023-08-23T18:37:18Z 2023-08-23T18:35:48Z OWNER  

See: - https://github.com/simonw/datasette-configure-fts/issues/14 - https://github.com/simonw/datasette-auth-tokens/issues/12

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2150/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1781022369 I_kwDOBm6k_c5qKD6h 2091 Drop support for Python 3.7 simonw 9599 closed 0     3 2023-06-29T15:06:38Z 2023-08-23T18:18:18Z 2023-08-23T18:18:18Z OWNER  

It's EOL now, as of 2023-06-27 (two days ago): https://devguide.python.org/versions/

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2091/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1795051447 I_kwDOBm6k_c5q_k-3 2097 Drop Python 3.7 simonw 9599 closed 0     0 2023-07-08T18:39:44Z 2023-08-23T18:18:00Z 2023-08-23T18:18:00Z OWNER  

I'm going to drop Python 3.7.

Originally posted by @simonw in https://github.com/simonw/datasette/issues/1153#issuecomment-1627455892

It's not supported any more: https://devguide.python.org/versions/

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2097/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
459509126 MDU6SXNzdWU0NTk1MDkxMjY= 516 Enforce import sort order with isort simonw 9599 open 0     8 2019-06-22T20:35:50Z 2023-08-23T02:15:36Z   OWNER  

I want to use isort to order imports. A few steps here:

  • [x] Add a .isort.cfg file (see below)
  • [x] Use isort -rc to reformat existing code
  • [ ] Commit this change
  • [x] Add a unit test that ensures future changes remain isort compatible
datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/516/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
459689615 MDExOlB1bGxSZXF1ZXN0MjkwOTcxMjk1 524 Sort commits using isort, refs #516 simonw 9599 closed 0     1 2019-06-24T05:04:48Z 2023-08-23T01:31:08Z 2023-08-23T01:31:08Z OWNER simonw/datasette/pulls/524

Also added a lint unit test to ensure they stay sorted. #516

datasette 107914493 pull    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/524/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
0  
449886319 MDU6SXNzdWU0NDk4ODYzMTk= 493 Rename metadata.json to config.json simonw 9599 closed 0   Datasette 1.0 3268330 7 2019-05-29T15:48:03Z 2023-08-23T01:29:21Z 2023-08-23T01:29:20Z OWNER  

It is increasingly being useful configuration options, when it started out as purely metadata.

Could cause confusion with the --config mechanism though - maybe that should be called "settings" instead?

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/493/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
324720095 MDU6SXNzdWUzMjQ3MjAwOTU= 275 "config" section in metadata.json (root, database and table level) simonw 9599 closed 0     3 2018-05-20T16:02:28Z 2023-08-23T01:28:37Z 2023-08-23T01:28:37Z OWNER  

Split off from #274

Metadata should an optional "config" section at root, table or database level.

The TableView and RowView and DatabaseView and BaseView classes could all have a .config("key") method which knows how to resolve the hierarchy of configs.

This will allow individual tables (or databases) to set their own config settings for things like sql_time_limit_ms

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/275/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1861812208 PR_kwDOBm6k_c5YhH-W 2149 Start a new `datasette.yaml` configuration file, with settings support asg017 15178711 closed 0     2 2023-08-22T16:24:16Z 2023-08-23T01:26:11Z 2023-08-23T01:26:11Z CONTRIBUTOR simonw/datasette/pulls/2149

refs #2093 #2143

This is the first step to implementing the new datasette.yaml/datasette.json configuration file.

  • The old --config argument is now back, and is the path to a datasette.yaml file. Acts like the --metadata flag.
  • The old settings.json behavior has been removed.
  • The "settings" key inside datasette.yaml defines the same --settings flags
  • Values passed in --settings will over-write values in datasette.yaml

Docs for the Config file is pretty light, not much to add until we add more config to the file.


:books: Documentation preview :books:: https://datasette--2149.org.readthedocs.build/en/2149/

datasette 107914493 pull    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2149/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
0  
1857234285 I_kwDOBm6k_c5usyVt 2145 If a row has a primary key of `null` various things break simonw 9599 open 0     23 2023-08-18T20:06:28Z 2023-08-21T17:30:01Z   OWNER  

Stumbled across this while experimenting with datasette-write-ui. The error I got was a 500 on the /db page:

'NoneType' object has no attribute 'encode'

Tracked it down to this code, which assembles the URL for a row page:

https://github.com/simonw/datasette/blob/943df09dcca93c3b9861b8c96277a01320db8662/datasette/utils/init.py#L120-L134

That's because tilde_encode can't handle None: https://github.com/simonw/datasette/blob/943df09dcca93c3b9861b8c96277a01320db8662/datasette/utils/init.py#L1175-L1178

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2145/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1856760386 PR_kwDOBm6k_c5YQGcc 2144 Bump the python-packages group with 3 updates dependabot[bot] 49699333 closed 0     2 2023-08-18T13:49:37Z 2023-08-21T13:48:18Z 2023-08-21T13:48:16Z CONTRIBUTOR simonw/datasette/pulls/2144

Bumps the python-packages group with 3 updates: sphinx, furo and blacken-docs.

Updates sphinx from 7.1.2 to 7.2.2

Release notes

Sourced from sphinx's releases.

Sphinx 7.2.2

Changelog: https://www.sphinx-doc.org/en/master/changes.html

Sphinx 7.2.1

Changelog: https://www.sphinx-doc.org/en/master/changes.html

Sphinx 7.2.0

Changelog: https://www.sphinx-doc.org/en/master/changes.html

Changelog

Sourced from sphinx's changelog.

Release 7.2.2 (released Aug 17, 2023)

Bugs fixed

  • Fix the signature of the StateMachine.insert_input() patch, for when calling with keyword arguments.
  • Fixed membership testing (in) for the :py:class:str interface of the asset classes (_CascadingStyleSheet and _JavaScript), which several extensions relied upon.
  • Fixed a type error in SingleFileHTMLBuilder._get_local_toctree, includehidden may be passed as a string or a boolean.
  • Fix :noindex: for PyModule and JSModule``.

Release 7.2.1 (released Aug 17, 2023)

Bugs fixed

  • Restored the the :py:class:str interface of the asset classes (_CascadingStyleSheet and _JavaScript), which several extensions relied upon. This will be removed in Sphinx 9.
  • Restored calls to Builder.add_{css,js}_file(), which several extensions relied upon.
  • Restored the private API TocTree.get_toctree_ancestors(), which several extensions relied upon.

Release 7.2.0 (released Aug 17, 2023)

Dependencies

  • #11511: Drop Python 3.8 support.
  • #11576: Require Pygments 2.14 or later.

Deprecated

  • #11512: Deprecate sphinx.util.md5 and sphinx.util.sha1. Use hashlib instead.
  • #11526: Deprecate sphinx.testing.path. Use os.path or pathlib instead.
  • #11528: Deprecate sphinx.util.split_index_msg and sphinx.util.split_into. Use sphinx.util.index_entries.split_index_msg instead.
  • Deprecate sphinx.builders.html.Stylesheet and sphinx.builders.html.Javascript. Use sphinx.application.Sphinx.add_css_file()

... (truncated)

Commits
  • ed84d63 Bump to 7.2.2 final
  • ea4a73e [bot]: Update message catalogues (#11612)
  • e47846a Fix :noindex: for PyModule and JSModule``
  • b2fc47f Add CHANGES entry for renaming the StateMachine.insert_input() parameter
  • 0835c3e Fix regression in SingleFileHTMLBuilder._get_local_toctree
  • 49dc0dd Fix asset class string interface membership testing
  • 8512855 Fix signature of docutils include_source monkeypatch (#11610)
  • e1d9068 Bump version
  • 441a9e4 Bump to 7.2.1 final
  • ec31853 Restore TocTree.get_toctree_ancestors()
  • Additional commits viewable in compare view


Updates furo from 2023.7.26 to 2023.8.17

Changelog

Sourced from furo's changelog.

Changelog

2023.08.17 -- Wonderous White

  • Fix compatiblity with Sphinx 7.2.0 and 7.2.1.

2023.07.26 -- Vigilant Volt

  • Fix compatiblity with Sphinx 7.1.
  • Improve how content overflow is handled.
  • Improve how literal blocks containing inline code are handled.

2023.05.20 -- Unassuming Ultramarine

  • ✨ Add support for Sphinx 7.
  • Drop support for Sphinx 5.
  • Improve the screen-reader label for sidebar collapse.
  • Make it easier to create derived themes from Furo.
  • Bump all JS dependencies (NodeJS and npm packages).

2023.03.27 -- Tasty Tangerine

  • Regenerate with newer version of sphinx-theme-builder, to fix RECORD hashes.
  • Add missing class to Font Awesome examples

2023.03.23 -- Sassy Saffron

  • Update Python version classifiers.
  • Increase the icon size in mobile header.
  • Increase admonition title bg opacity.
  • Change the default API background to transparent.
  • Transition the API background change.
  • Remove the "indent" of API entries which have a background.
  • Break long inline code literals.

2022.12.07 -- Reverent Raspberry

  • ✨ Add support for Sphinx 6.
  • ✨ Improve footnote presentation with docutils 0.18+.

... (truncated)

Commits
  • 99f4e6d Prepare release: 2023.08.17
  • 59ad807 Update changelog
  • 9b94420 Accomodate for Sphinx 7.2 behavior change
  • 7aeb6d2 [pre-commit.ci] pre-commit autoupdate (#675)
  • 0c9be3f Back to development
  • See full diff in compare view


Updates blacken-docs from 1.15.0 to 1.16.0

Changelog

Sourced from blacken-docs's changelog.

1.16.0 (2023-08-16)

  • Allow Markdown fence options.

    Thanks to initial work from Matthew Anderson in PR [#246](https://github.com/asottile/blacken-docs/issues/246) <https://github.com/adamchainz/blacken-docs/pull/246>__.

  • Expand Markdown detection to all Python language names from Pygments: py, sage, python3, py3, and numpy.

  • Preserve leading whitespace lines in reStructuredText code blocks.

    Thanks to Julianus Pfeuffer for the report in Issue [#217](https://github.com/asottile/blacken-docs/issues/217) <https://github.com/adamchainz/blacken-docs/issues/217>__.

  • Use exit code 2 to indicate errors from Black, whilst exit code 1 remains for “files have been formatted”.

    Thanks to Julianus Pfeuffer for the report in Issue [#218](https://github.com/asottile/blacken-docs/issues/218) <https://github.com/adamchainz/blacken-docs/issues/218>__.

  • Support passing the --preview option through to Black, to select the future style.

  • Remove language_version from .pre-commit-hooks.yaml. This change allows default_language_version in ``.pre-commit-config.yaml` to take precedence.

    Thanks to Aneesh Agrawal in PR [#258](https://github.com/asottile/blacken-docs/issues/258) <https://github.com/adamchainz/blacken-docs/pull/258>__.

Commits
  • 960ead2 Version 1.16.0
  • 8f0ed18 Support passing --preview through to Black (#273)
  • 4eb4e4c Tweak changelog note
  • 6c7450c Use exit code 2 to indicate errors (#272)
  • 99dfc8d Preserve leading whitespace lines in rST (#271)
  • 94465e8 Reformat markdown tests with dedent() (#270)
  • 7cd5f30 Use .md in glob example
  • f97e569 Document applying to many files (#269)
  • ae612b0 Expand Markdown detection to all Python language names (#268)
  • da9b455 Replace NamedTuple with plain class (#267)
  • Additional commits viewable in compare view


Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore <dependency name> major version` will close this group update PR and stop Dependabot creating any more for the specific dependency's major version (unless you unignore this specific dependency's major version or upgrade to it yourself) - `@dependabot ignore <dependency name> minor version` will close this group update PR and stop Dependabot creating any more for the specific dependency's minor version (unless you unignore this specific dependency's minor version or upgrade to it yourself) - `@dependabot ignore <dependency name> dependency` will close this group update PR and stop Dependabot creating any more for the specific dependency (unless you unignore this specific dependency or upgrade to it yourself) - `@dependabot unignore <dependency name> dependency` will remove all of the ignore conditions of the specified dependency - `@dependabot unignore <dependency name> <ignore condition>` will remove the ignore condition of the specified dependency and ignore conditions

:books: Documentation preview :books:: https://datasette--2144.org.readthedocs.build/en/2144/

datasette 107914493 pull    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2144/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
0  
1854970601 PR_kwDOBm6k_c5YKAZ4 2142 Bump the python-packages group with 2 updates dependabot[bot] 49699333 closed 0     2 2023-08-17T13:07:53Z 2023-08-18T13:49:29Z 2023-08-18T13:49:26Z CONTRIBUTOR simonw/datasette/pulls/2142

Bumps the python-packages group with 2 updates: sphinx and blacken-docs.

Updates sphinx from 7.1.2 to 7.2.0

Release notes

Sourced from sphinx's releases.

Sphinx 7.2.0

Changelog: https://www.sphinx-doc.org/en/master/changes.html

Changelog

Sourced from sphinx's changelog.

Release 7.2.0 (released Aug 17, 2023)

Dependencies

  • #11511: Drop Python 3.8 support.
  • #11576: Require Pygments 2.14 or later.

Deprecated

  • #11512: Deprecate sphinx.util.md5 and sphinx.util.sha1. Use hashlib instead.
  • #11526: Deprecate sphinx.testing.path. Use os.path or pathlib instead.
  • #11528: Deprecate sphinx.util.split_index_msg and sphinx.util.split_into. Use sphinx.util.index_entries.split_index_msg instead.
  • Deprecate sphinx.builders.html.Stylesheet and sphinx.builders.html.Javascript. Use sphinx.application.Sphinx.add_css_file() and sphinx.application.Sphinx.add_js_file() instead.
  • #11582: Deprecate sphinx.builders.html.StandaloneHTMLBuilder.css_files and sphinx.builders.html.StandaloneHTMLBuilder.script_files. Use sphinx.application.Sphinx.add_css_file() and sphinx.application.Sphinx.add_js_file() instead.
  • #11459: Deprecate sphinx.ext.autodoc.preserve_defaults.get_function_def(). Patch by Bénédikt Tran.

Features added

  • #11526: Support os.PathLike types and pathlib.Path objects in many more places.
  • #5474: coverage: Print summary statistics tables. Patch by Jorge Leitao.
  • #6319: viewcode: Add :confval:viewcode_line_numbers to control whether line numbers are added to rendered source code. Patch by Ben Krikler.
  • #9662: Add the :no-typesetting: option to suppress textual output and only create a linkable anchor. Patch by Latosha Maltba.
  • #11221: C++: Support domain objects in the table of contents. Patch by Rouslan Korneychuk.
  • #10938: doctest: Add :confval:doctest_show_successes option. Patch by Trey Hunner.
  • #11533: Add :no-index:, :no-index-entry:, and :no-contents-entry:.
  • #11572: Improve debug logging of reasons why files are detected as out of date. Patch by Eric Larson.

... (truncated)

Commits
  • da9f8a5 Bump to 7.2.0 final
  • 794324a Declare support for Python 3.13
  • 03bceac [bot]: Update message catalogues (#11538)
  • aecebce Make _resolve_toctree() keyword-only
  • 76658c4 Fix sphinx.ext.autodoc.preserve_defaults (#11550)
  • 4dee162 Fix layout issues in the agogo theme for smaller viewports (#11594)
  • 4ddbee4 Fix console_scripts entry points
  • 4add50a Remove unneeded type annotation
  • 9d8ef83 Fix test_inspect_main_noargs
  • ddf8a8e Add CHANGES entry for GH-11533
  • Additional commits viewable in compare view


Updates blacken-docs from 1.15.0 to 1.16.0

Changelog

Sourced from blacken-docs's changelog.

1.16.0 (2023-08-16)

  • Allow Markdown fence options.

    Thanks to initial work from Matthew Anderson in PR [#246](https://github.com/asottile/blacken-docs/issues/246) <https://github.com/adamchainz/blacken-docs/pull/246>__.

  • Expand Markdown detection to all Python language names from Pygments: py, sage, python3, py3, and numpy.

  • Preserve leading whitespace lines in reStructuredText code blocks.

    Thanks to Julianus Pfeuffer for the report in Issue [#217](https://github.com/asottile/blacken-docs/issues/217) <https://github.com/adamchainz/blacken-docs/issues/217>__.

  • Use exit code 2 to indicate errors from Black, whilst exit code 1 remains for “files have been formatted”.

    Thanks to Julianus Pfeuffer for the report in Issue [#218](https://github.com/asottile/blacken-docs/issues/218) <https://github.com/adamchainz/blacken-docs/issues/218>__.

  • Support passing the --preview option through to Black, to select the future style.

  • Remove language_version from .pre-commit-hooks.yaml. This change allows default_language_version in ``.pre-commit-config.yaml` to take precedence.

    Thanks to Aneesh Agrawal in PR [#258](https://github.com/asottile/blacken-docs/issues/258) <https://github.com/adamchainz/blacken-docs/pull/258>__.

Commits
  • 960ead2 Version 1.16.0
  • 8f0ed18 Support passing --preview through to Black (#273)
  • 4eb4e4c Tweak changelog note
  • 6c7450c Use exit code 2 to indicate errors (#272)
  • 99dfc8d Preserve leading whitespace lines in rST (#271)
  • 94465e8 Reformat markdown tests with dedent() (#270)
  • 7cd5f30 Use .md in glob example
  • f97e569 Document applying to many files (#269)
  • ae612b0 Expand Markdown detection to all Python language names (#268)
  • da9b455 Replace NamedTuple with plain class (#267)
  • Additional commits viewable in compare view


Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore <dependency name> major version` will close this group update PR and stop Dependabot creating any more for the specific dependency's major version (unless you unignore this specific dependency's major version or upgrade to it yourself) - `@dependabot ignore <dependency name> minor version` will close this group update PR and stop Dependabot creating any more for the specific dependency's minor version (unless you unignore this specific dependency's minor version or upgrade to it yourself) - `@dependabot ignore <dependency name> dependency` will close this group update PR and stop Dependabot creating any more for the specific dependency (unless you unignore this specific dependency or upgrade to it yourself) - `@dependabot unignore <dependency name> dependency` will remove all of the ignore conditions of the specified dependency - `@dependabot unignore <dependency name> <ignore condition>` will remove the ignore condition of the specified dependency and ignore conditions

:books: Documentation preview :books:: https://datasette--2142.org.readthedocs.build/en/2142/

datasette 107914493 pull    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2142/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
0  
1853289039 PR_kwDOBm6k_c5YEUBK 2141 Bump the python-packages group with 1 update dependabot[bot] 49699333 closed 0     2 2023-08-16T13:47:35Z 2023-08-17T13:07:48Z 2023-08-17T13:07:45Z CONTRIBUTOR simonw/datasette/pulls/2141

Bumps the python-packages group with 1 update: blacken-docs.

Changelog

Sourced from blacken-docs's changelog.

1.16.0 (2023-08-16)

  • Allow Markdown fence options.

    Thanks to initial work from Matthew Anderson in PR [#246](https://github.com/asottile/blacken-docs/issues/246) <https://github.com/adamchainz/blacken-docs/pull/246>__.

  • Expand Markdown detection to all Python language names from Pygments: py, sage, python3, py3, and numpy.

  • Preserve leading whitespace lines in reStructuredText code blocks.

    Thanks to Julianus Pfeuffer for the report in Issue [#217](https://github.com/asottile/blacken-docs/issues/217) <https://github.com/adamchainz/blacken-docs/issues/217>__.

  • Use exit code 2 to indicate errors from Black, whilst exit code 1 remains for “files have been formatted”.

    Thanks to Julianus Pfeuffer for the report in Issue [#218](https://github.com/asottile/blacken-docs/issues/218) <https://github.com/adamchainz/blacken-docs/issues/218>__.

  • Support passing the --preview option through to Black, to select the future style.

  • Remove language_version from .pre-commit-hooks.yaml. This change allows default_language_version in ``.pre-commit-config.yaml` to take precedence.

    Thanks to Aneesh Agrawal in PR [#258](https://github.com/asottile/blacken-docs/issues/258) <https://github.com/adamchainz/blacken-docs/pull/258>__.

Commits
  • 960ead2 Version 1.16.0
  • 8f0ed18 Support passing --preview through to Black (#273)
  • 4eb4e4c Tweak changelog note
  • 6c7450c Use exit code 2 to indicate errors (#272)
  • 99dfc8d Preserve leading whitespace lines in rST (#271)
  • 94465e8 Reformat markdown tests with dedent() (#270)
  • 7cd5f30 Use .md in glob example
  • f97e569 Document applying to many files (#269)
  • ae612b0 Expand Markdown detection to all Python language names (#268)
  • da9b455 Replace NamedTuple with plain class (#267)
  • Additional commits viewable in compare view


Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore <dependency name> major version` will close this group update PR and stop Dependabot creating any more for the specific dependency's major version (unless you unignore this specific dependency's major version or upgrade to it yourself) - `@dependabot ignore <dependency name> minor version` will close this group update PR and stop Dependabot creating any more for the specific dependency's minor version (unless you unignore this specific dependency's minor version or upgrade to it yourself) - `@dependabot ignore <dependency name> dependency` will close this group update PR and stop Dependabot creating any more for the specific dependency (unless you unignore this specific dependency or upgrade to it yourself) - `@dependabot unignore <dependency name> dependency` will remove all of the ignore conditions of the specified dependency - `@dependabot unignore <dependency name> <ignore condition>` will remove the ignore condition of the specified dependency and ignore conditions

:books: Documentation preview :books:: https://datasette--2141.org.readthedocs.build/en/2141/

datasette 107914493 pull    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2141/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
0  
1847201263 I_kwDOBm6k_c5uGg3v 2140 Remove all remaining documentation instances of '$ ' simonw 9599 closed 0     1 2023-08-11T17:42:13Z 2023-08-11T17:52:25Z 2023-08-11T17:45:00Z OWNER  

For example this: https://github.com/simonw/datasette/blob/4535568f2ce907af646304d0ebce2500ebd55677/docs/authentication.rst?plain=1#L33-L35

The problem with that $ prefix is that it prevents users from copying and pasting the raw command.

https://docs.datasette.io/en/stable/authentication.html#using-the-root-actor

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2140/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1838266862 I_kwDOBm6k_c5tkbnu 2126 Permissions in metadata.yml / metadata.json ctsrc 36199671 closed 0     3 2023-08-06T16:24:10Z 2023-08-11T05:52:30Z 2023-08-11T05:52:29Z NONE  

https://docs.datasette.io/en/latest/authentication.html#other-permissions-in-metadata says the following:

For all other permissions, you can use one or more "permissions" blocks in your metadata.

To grant access to the permissions debug tool to all signed in users you can grant permissions-debug to any actor with an id matching the wildcard * by adding this a the root of your metadata:

yaml permissions: debug-menu: id: '*'

I tried this.

My metadata.yml file looks like:

yaml permissions: debug-menu: id: '*' permissions-debug: id: '*' plugins: datasette-auth-passwords: myuser_password_hash: $env: "PASSWORD_HASH_MYUSER"

And then I run

zsh datasette -m metadata.yml tiddlywiki.db --root

And I open a session for the "root" user of datasette with the link given.

I open a private browser session and log in as "myuser" from http://127.0.0.1:8001/-/login

Then I check http://127.0.0.1:8001/-/actor which confirms that I am logged in as the "myuser" actor

json { "actor": { "id": "myuser" } }

In the session where I am logged in as "myuser" I then try to go to http://127.0.0.1:8001/-/permissions

But all I get there as the logged in user "myuser" is

Forbidden

Permission denied

And then if I check the http://127.0.0.1:8001/-/permissions as the datasette "root" user from another browser session, I see:

permissions-debug checked at 2023-08-06T16:22:58.997841 ✗ (used default)

Actor: {"id": "myuser"}

It seems that in spite of having tried to give the permissions-debug permission to the "myuser" user in my metadata.yml file, datasette does not agree that "myuser" has permission permissions-debug..

What do I need to do differently so that my "myuser" user is able to access http://127.0.0.1:8001/-/permissions ?

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2126/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1823393475 I_kwDOBm6k_c5srsbD 2119 database color shows only on index page, not other pages simonw 9599 closed 0   Datasette 1.0 3268330 3 2023-07-27T00:19:39Z 2023-08-11T05:25:45Z 2023-08-11T05:16:24Z OWNER  

I think this has been a bug for a long time.

https://latest.datasette.io/ currently shows:

Those colors are based on a hash of the database name. But when you click through to https://latest.datasette.io/fixtures

It's red on all sub-pages too.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2119/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
476852861 MDU6SXNzdWU0NzY4NTI4NjE= 568 Add database_color as a configurable option LBHELewis 50906992 open 0     1 2019-08-05T13:14:45Z 2023-08-11T05:19:42Z   NONE  

This would be really useful as it would allow us to tie in with colour schemes.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/568/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1846076261 I_kwDOBm6k_c5uCONl 2139 border-color: ##ff0000 bug - two hashes simonw 9599 closed 0   Datasette 1.0a4 8755003 2 2023-08-11T01:22:58Z 2023-08-11T05:16:24Z 2023-08-11T05:16:24Z OWNER  

Spotted this on https://latest.datasette.io/extra_database

```html

```
datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2139/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1802613340 PR_kwDOBm6k_c5VZhfw 2100 Make primary key view accessible to render_cell hook meowcat 1563881 open 0     0 2023-07-13T09:30:36Z 2023-08-10T13:15:41Z   FIRST_TIME_CONTRIBUTOR simonw/datasette/pulls/2100

:books: Documentation preview :books:: https://datasette--2100.org.readthedocs.build/en/2100/

datasette 107914493 pull    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2100/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
0  
1838469176 I_kwDOBm6k_c5tlNA4 2127 Context base class to support documenting the context simonw 9599 open 0   Datasette 1.0 3268330 3 2023-08-07T00:01:02Z 2023-08-10T01:30:25Z   OWNER  

This idea first came up here: - https://github.com/simonw/datasette/issues/2112#issuecomment-1652751140

If datasette.render_template(...) takes an optional Context subclass as an alternative to a context dictionary, I could then use dataclasses to define the context made available to specific templates - which then gives me something I can use to help document what they are.

Also refs: - https://github.com/simonw/datasette/issues/1510

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2127/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1843391585 I_kwDOBm6k_c5t3-xh 2134 Add writable canned query demo to latest.datasette.io simonw 9599 closed 0     5 2023-08-09T14:31:30Z 2023-08-10T01:22:46Z 2023-08-10T01:05:56Z OWNER  

This would be useful while working on: - #2114

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2134/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1844213115 I_kwDOBm6k_c5t7HV7 2138 on_success_message_sql option for writable canned queries simonw 9599 closed 0   Datasette 1.0a4 8755003 2 2023-08-10T00:20:14Z 2023-08-10T00:39:40Z 2023-08-10T00:34:26Z OWNER  

Or... how about if the on_success_message option could define a SQL query to be executed to generate that message? Maybe on_success_message_sql.

  • https://github.com/simonw/datasette/issues/2134
datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2138/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1841501975 I_kwDOBm6k_c5twxcX 2133 [feature request]`datasette install plugins.json` options HaveF 54462 closed 0     9 2023-08-08T15:06:50Z 2023-08-10T00:31:24Z 2023-08-09T22:04:46Z NONE  

Hi, simon ❤️

datasette plugins --all > plugins.json could generate all plugins info. On another machine, it would be great to install all plugins just by datasette install plugins.json

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2133/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
627794879 MDU6SXNzdWU2Mjc3OTQ4Nzk= 782 Redesign default .json format simonw 9599 closed 0   Datasette 1.0a4 8755003 55 2020-05-30T18:47:07Z 2023-08-10T00:07:17Z 2023-08-10T00:07:17Z OWNER  

The default JSON just isn't right. I find myself using ?_shape=array for almost everything I build against the API.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/782/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1843600087 I_kwDOBm6k_c5t4xrX 2135 Release notes for 1.0a3 simonw 9599 closed 0   Datasette 1.0a3 9700784 3 2023-08-09T16:09:26Z 2023-08-09T19:17:07Z 2023-08-09T19:17:06Z OWNER  

118 commits! https://github.com/simonw/datasette/compare/1.0a2...26be9f0445b753fb84c802c356b0791a72269f25

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2135/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1843710170 I_kwDOBm6k_c5t5Mja 2136 Query view shouldn't return `columns` simonw 9599 closed 0   Datasette 1.0a3 9700784 4 2023-08-09T17:23:57Z 2023-08-09T19:03:04Z 2023-08-09T19:03:04Z OWNER  

I just noticed that https://latest.datasette.io/fixtures/roadside_attraction_characteristics.json?_labels=on&_size=1 returns: json { "ok": true, "next": "1", "rows": [ { "rowid": 1, "attraction_id": { "value": 1, "label": "The Mystery Spot" }, "characteristic_id": { "value": 2, "label": "Paranormal" } } ], "truncated": false } But https://latest.datasette.io/fixtures.json?sql=select+rowid%2C+attraction_id%2C+characteristic_id+from+roadside_attraction_characteristics+order+by+rowid+limit+1 returns: json { "rows": [ { "rowid": 1, "attraction_id": 1, "characteristic_id": 2 } ], "columns": [ "rowid", "attraction_id", "characteristic_id" ], "ok": true, "truncated": false } The columns key in the query response is inconsistent with the table response.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2136/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1843821954 I_kwDOBm6k_c5t5n2C 2137 Redesign row default JSON simonw 9599 open 0   Datasette 1.0a4 8755003 1 2023-08-09T18:49:11Z 2023-08-09T19:02:47Z   OWNER  

This URL here:

https://latest.datasette.io/fixtures/simple_primary_key/1.json?_extras=foreign_key_tables

json { "database": "fixtures", "table": "simple_primary_key", "rows": [ { "id": "1", "content": "hello" } ], "columns": [ "id", "content" ], "primary_keys": [ "id" ], "primary_key_values": [ "1" ], "units": {}, "foreign_key_tables": [ { "other_table": "foreign_key_references", "column": "id", "other_column": "foreign_key_with_blank_label", "count": 0, "link": "/fixtures/foreign_key_references?foreign_key_with_blank_label=1" }, { "other_table": "foreign_key_references", "column": "id", "other_column": "foreign_key_with_label", "count": 1, "link": "/fixtures/foreign_key_references?foreign_key_with_label=1" }, { "other_table": "complex_foreign_keys", "column": "id", "other_column": "f3", "count": 1, "link": "/fixtures/complex_foreign_keys?f3=1" }, { "other_table": "complex_foreign_keys", "column": "id", "other_column": "f2", "count": 0, "link": "/fixtures/complex_foreign_keys?f2=1" }, { "other_table": "complex_foreign_keys", "column": "id", "other_column": "f1", "count": 1, "link": "/fixtures/complex_foreign_keys?f1=1" } ], "query_ms": 4.226590999678592, "source": "tests/fixtures.py", "source_url": "https://github.com/simonw/datasette/blob/main/tests/fixtures.py", "license": "Apache License 2.0", "license_url": "https://github.com/simonw/datasette/blob/main/LICENSE", "ok": true, "truncated": false }

That ?_extras= should be ?_extra= - plus the row JSON should be redesigned to fit the new default JSON representation.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2137/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1822939274 I_kwDOBm6k_c5sp9iK 2113 Implement and document extras for the new query view page simonw 9599 open 0   Datasette 1.0a4 8755003 3 2023-07-26T18:24:01Z 2023-08-09T17:35:22Z   OWNER  
  • 2109

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2113/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1560662739 I_kwDOBm6k_c5dBdLT 2007 `render_cell()` hook should take an optional `request` argument simonw 9599 closed 0     1 2023-01-28T03:13:00Z 2023-08-09T17:15:03Z 2023-01-28T03:34:26Z OWNER  

From Discord: https://discordapp.com/channels/823971286308356157/996877076982415491/1068227071156965486

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2007/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1840417903 I_kwDOBm6k_c5tsoxv 2131 Refactor code that supports templates_considered comment simonw 9599 open 0   Datasette 1.0 3268330 1 2023-08-08T01:28:36Z 2023-08-09T15:27:41Z   OWNER  

I ended up duplicating it here: https://github.com/simonw/datasette/blob/7532feb424b1dce614351e21b2265c04f9669fe2/datasette/views/database.py#L164-L167

I think it should move to datasette.render_template() - and maybe have a renamed template variable too.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2131/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1822940263 I_kwDOBm6k_c5sp9xn 2114 Implement canned queries against new query JSON work simonw 9599 closed 0   Datasette 1.0a3 9700784 3 2023-07-26T18:24:50Z 2023-08-09T15:26:58Z 2023-08-09T15:26:57Z OWNER  
  • 2109

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2114/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1841343173 I_kwDOBm6k_c5twKrF 2132 Get form fields on query page working again simonw 9599 closed 0   Datasette 1.0a3 9700784 1 2023-08-08T13:39:05Z 2023-08-08T13:45:10Z 2023-08-08T13:45:09Z OWNER  

Caused by: - #2112

https://latest.datasette.io/fixtures?sql=select+pk1%2C+pk2%2C+pk3%2C+content+from+compound_three_primary_keys+where+%22pk1%22+%3D+%3Ap0+order+by+pk1%2C+pk2%2C+pk3+limit+101&p0=b

The :p0 form field is missing. Submitting the form results in this error:

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2132/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1840324765 I_kwDOBm6k_c5tsSCd 2129 CSV ?sql= should indicate errors simonw 9599 open 0   Datasette 1.0 3268330 1 2023-08-07T23:13:04Z 2023-08-08T02:02:21Z   OWNER  

https://latest.datasette.io/_memory.csv?sql=select+blah is a blank page right now:

bash curl -I 'https://latest.datasette.io/_memory.csv?sql=select+blah' HTTP/2 200 access-control-allow-origin: * access-control-allow-headers: Authorization, Content-Type access-control-expose-headers: Link access-control-allow-methods: GET, POST, HEAD, OPTIONS access-control-max-age: 3600 content-type: text/plain; charset=utf-8 x-databases: _memory, _internal, fixtures, fixtures2, extra_database, ephemeral date: Mon, 07 Aug 2023 23:12:15 GMT server: Google Frontend

Originally posted by @simonw in https://github.com/simonw/datasette/issues/2118#issuecomment-1668688947

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2129/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1822982933 I_kwDOBm6k_c5sqIMV 2117 Figure out what to do about `DatabaseView.name` simonw 9599 closed 0   Datasette 1.0a3 9700784 1 2023-07-26T18:58:06Z 2023-08-08T02:02:07Z 2023-08-08T02:02:07Z OWNER  

In the old code:

https://github.com/simonw/datasette/blob/08181823990a71ffa5a1b57b37259198eaa43e06/datasette/views/database.py#L34-L35

This name class attribute was later used by some of the plugin hooks, passed as view_name: https://github.com/simonw/datasette/blob/18dd88ee4d78fe9d760e9da96028ae06d938a85c/datasette/hookspecs.py#L50-L54

Figure out how that should work once I've refactored those classes to view functions instead.

Refs: - #2109

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2117/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1822940964 I_kwDOBm6k_c5sp98k 2115 Ensure all tests pass against new query view JSON simonw 9599 closed 0   Datasette 1.0a3 9700784 0 2023-07-26T18:25:20Z 2023-08-08T02:01:39Z 2023-08-08T02:01:38Z OWNER  
  • 2109

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2115/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1822938661 I_kwDOBm6k_c5sp9Yl 2112 Build HTML version of /content?sql=... simonw 9599 closed 0   Datasette 1.0a3 9700784 5 2023-07-26T18:23:34Z 2023-08-08T02:01:09Z 2023-08-08T02:01:01Z OWNER  

This will help make the hook as robust as possible. - #2109

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2112/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1822937426 I_kwDOBm6k_c5sp9FS 2111 Implement new /content.json?sql=... simonw 9599 closed 0   Datasette 1.0a3 9700784 4 2023-07-26T18:22:39Z 2023-08-08T02:00:37Z 2023-08-08T02:00:22Z OWNER  

This will be the base that the remaining work builds on top of. Refs: - #2109

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2111/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1840329615 I_kwDOBm6k_c5tsTOP 2130 Render plugin mechanism needs `error` and `truncated` fields simonw 9599 closed 0   Datasette 1.0a3 9700784 2 2023-08-07T23:19:19Z 2023-08-08T01:51:54Z 2023-08-08T01:47:42Z OWNER  

While working on: - https://github.com/simonw/datasette/pull/2118

It became clear that the render callback function documented here: https://docs.datasette.io/en/0.64.3/plugin_hooks.html#register-output-renderer-datasette

Needs to grow the ability to be told if an error occurred (an error string) and if the results were truncated (a truncated boolean).

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2130/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1823352380 PR_kwDOBm6k_c5Wfgd9 2118 New JSON design for query views simonw 9599 closed 0   Datasette 1.0a3 9700784 11 2023-07-26T23:29:21Z 2023-08-08T01:47:40Z 2023-08-08T01:47:39Z OWNER simonw/datasette/pulls/2118

WIP. Refs: - #2109


:books: Documentation preview :books:: https://datasette--2118.org.readthedocs.build/en/2118/

datasette 107914493 pull    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2118/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
0  
1833193570 PR_kwDOBm6k_c5XArm3 2125 Bump sphinx from 6.1.3 to 7.1.2 dependabot[bot] 49699333 closed 0     2 2023-08-02T13:28:39Z 2023-08-07T16:20:30Z 2023-08-07T16:20:27Z CONTRIBUTOR simonw/datasette/pulls/2125

Bumps sphinx from 6.1.3 to 7.1.2.

Release notes

Sourced from sphinx's releases.

Sphinx 7.1.2

Changelog: https://www.sphinx-doc.org/en/master/changes.html

Sphinx 7.1.1

Changelog: https://www.sphinx-doc.org/en/master/changes.html

Sphinx 7.1.0

Changelog: https://www.sphinx-doc.org/en/master/changes.html

v7.0.1

Changelog: https://www.sphinx-doc.org/en/master/changes.html

v7.0.0

Changelog: https://www.sphinx-doc.org/en/master/changes.html

v7.0.0rc1

Changelog: https://www.sphinx-doc.org/en/master/changes.html

v6.2.1

Changelog: https://www.sphinx-doc.org/en/master/changes.html

v6.2.0

Changelog: https://www.sphinx-doc.org/en/master/changes.html

Changelog

Sourced from sphinx's changelog.

Release 7.1.2 (released Aug 02, 2023)

Bugs fixed

  • #11542: linkcheck: Properly respect :confval:linkcheck_anchors and do not spuriously report failures to validate anchors. Patch by James Addison.

Release 7.1.1 (released Jul 27, 2023)

Bugs fixed

  • #11514: Fix SOURCE_DATE_EPOCH in multi-line copyright footer. Patch by Bénédikt Tran.

Release 7.1.0 (released Jul 24, 2023)

Incompatible changes

  • Releases are no longer signed, given the change in PyPI policy_.

    .. _change in PyPI policy: https://blog.pypi.org/posts/2023-05-23-removing-pgp/

Deprecated

  • #11412: Emit warnings on using a deprecated Python-specific index entry type (namely, module, keyword, operator, object, exception, statement, and builtin) in the :rst:dir:index directive, and set the removal version to Sphinx 9. Patch by Adam Turner.

Features added

  • #11415: Add a checksum to JavaScript and CSS asset URIs included within generated HTML, using the CRC32 algorithm.
  • :meth:~sphinx.application.Sphinx.require_sphinx now allows the version requirement to be specified as (major, minor).
  • #11011: Allow configuring a line-length limit for object signatures, via :confval:maximum_signature_line_length and the domain-specific variants. If the length of the signature (in characters) is greater than the configured limit, each parameter in the signature will be split to its own logical line. This behaviour may also be controlled by options on object description directives, for example :rst:dir:py:function:single-line-parameter-list.

... (truncated)

Commits
  • 320d0b6 Bump to 7.1.2 final
  • 06c1339 Ensure that linkcheck_anchors is properly respected (#11544)
  • 89808c6 Emend attribution for GH-11431 in CHANGES (#11547)
  • 4958d2a Fix flake8 E231 reported linting failure
  • 4862da8 Bump version
  • d8d73c1 Bump to 7.1.1 final
  • a54e630 [internationalisation] Update message catalogues
  • 8452300 Fix multi-line copyright when SOURCE_DATE_EPOCH is set (#11524)
  • fe08cec bump version
  • e560f63 Bump to 7.1.0 final
  • Additional commits viewable in compare view


Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

:books: Documentation preview :books:: https://datasette--2125.org.readthedocs.build/en/2125/

datasette 107914493 pull    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2125/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
0  
1824399610 PR_kwDOBm6k_c5WjCS8 2121 Bump furo from 2023.3.27 to 2023.7.26 dependabot[bot] 49699333 closed 0     2 2023-07-27T13:40:48Z 2023-08-07T16:20:23Z 2023-08-07T16:20:20Z CONTRIBUTOR simonw/datasette/pulls/2121

Bumps furo from 2023.3.27 to 2023.7.26.

Changelog

Sourced from furo's changelog.

Changelog

2023.07.26 -- Vigilant Volt

  • Fix compatiblity with Sphinx 7.1.
  • Improve how content overflow is handled.
  • Improve how literal blocks containing inline code are handled.

2023.05.20 -- Unassuming Ultramarine

  • ✨ Add support for Sphinx 7.
  • Drop support for Sphinx 5.
  • Improve the screen-reader label for sidebar collapse.
  • Make it easier to create derived themes from Furo.
  • Bump all JS dependencies (NodeJS and npm packages).

2023.03.27 -- Tasty Tangerine

  • Regenerate with newer version of sphinx-theme-builder, to fix RECORD hashes.
  • Add missing class to Font Awesome examples

2023.03.23 -- Sassy Saffron

  • Update Python version classifiers.
  • Increase the icon size in mobile header.
  • Increase admonition title bg opacity.
  • Change the default API background to transparent.
  • Transition the API background change.
  • Remove the "indent" of API entries which have a background.
  • Break long inline code literals.

2022.12.07 -- Reverent Raspberry

  • ✨ Add support for Sphinx 6.
  • ✨ Improve footnote presentation with docutils 0.18+.
  • Drop support for Sphinx 4.
  • Improve documentation about what the edit button does.
  • Improve handling of empty-flexboxes for better print experience on Chrome.
  • Improve styling for inline signatures.

... (truncated)

Commits
  • 35f5307 Prepare release: 2023.07.26
  • 0a8bedc Update changelog
  • a92dd0c Make _add_asset_hashes a no-op with Sphinx 7.1
  • f8db95b Improve literals with inline code are handled
  • 1680dbe Document the use of figclass with figure directive
  • beebd7e Increase the specificity of the admonition title selector
  • 834e951 Setup uploads to Percy
  • 27bf2c0 [pre-commit.ci] pre-commit autoupdate (#672)
  • c8b51d0 Fix how content overflow is handled
  • 80afa27 [pre-commit.ci] pre-commit autoupdate (#652)
  • Additional commits viewable in compare view


Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

:books: Documentation preview :books:: https://datasette--2121.org.readthedocs.build/en/2121/

datasette 107914493 pull    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2121/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
0  
1796830110 PR_kwDOBm6k_c5VFw3j 2098 Bump blacken-docs from 1.14.0 to 1.15.0 dependabot[bot] 49699333 closed 0     2 2023-07-10T13:49:12Z 2023-08-07T16:20:22Z 2023-08-07T16:20:20Z CONTRIBUTOR simonw/datasette/pulls/2098

Bumps blacken-docs from 1.14.0 to 1.15.0.

Changelog

Sourced from blacken-docs's changelog.

1.15.0 (2023-07-09)

  • Drop Python 3.7 support.
Commits
  • 2e74dc4 Version 1.15.0
  • 8ddb2f1 [pre-commit.ci] pre-commit autoupdate (#257)
  • 64fbca6 Drop Python 3.7 support (#256)
  • f6a0817 [pre-commit.ci] pre-commit autoupdate (#255)
  • cbdcb50 [pre-commit.ci] pre-commit autoupdate (#254)
  • 37fc70e Upgrade requirements (#253)
  • 7bbe434 [pre-commit.ci] pre-commit autoupdate (#250)
  • See full diff in compare view


Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

:books: Documentation preview :books:: https://datasette--2098.org.readthedocs.build/en/2098/

datasette 107914493 pull    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2098/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
0  

Next page

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options:

CREATE TABLE [issues] (
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [number] INTEGER,
   [title] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [state] TEXT,
   [locked] INTEGER,
   [assignee] INTEGER REFERENCES [users]([id]),
   [milestone] INTEGER REFERENCES [milestones]([id]),
   [comments] INTEGER,
   [created_at] TEXT,
   [updated_at] TEXT,
   [closed_at] TEXT,
   [author_association] TEXT,
   [pull_request] TEXT,
   [body] TEXT,
   [repo] INTEGER REFERENCES [repos]([id]),
   [type] TEXT
, [active_lock_reason] TEXT, [performed_via_github_app] TEXT, [reactions] TEXT, [draft] INTEGER, [state_reason] TEXT);
CREATE INDEX [idx_issues_repo]
                ON [issues] ([repo]);
CREATE INDEX [idx_issues_milestone]
                ON [issues] ([milestone]);
CREATE INDEX [idx_issues_assignee]
                ON [issues] ([assignee]);
CREATE INDEX [idx_issues_user]
                ON [issues] ([user]);
Powered by Datasette · Queries took 584.772ms · About: github-to-sqlite