This goes beyond the transform() method - the curious methods that create new SQL tables could benefit from the ability to add CHECK constraints too.

I haven't used these myself, do you have any CREATE TABLE examples that use them that you can share?

I'm using them myself for the first time as well, this is a tutorial of how to use (and change) them in sqlite: https://www.sqlitetutorial.net/sqlite-check-constraint/

These changes so far are now in the 0.60a0 alpha: https://github.com/simonw/datasette/releases/tag/0.60a0

This is now available to try out in Datasette 0.60a0: https://github.com/simonw/datasette/releases/tag/0.60a0

Oh that makes sense: In Python 3.6 this happens: Collecting janus<1.1,>=0.6.2 Using cached janus-0.7.0-py3-none-any.whl (6.9 kB) While in Python 3.7 or higher this happens: Collecting janus<1.1,>=0.6.2 Downloading janus-1.0.0-py3-none-any.whl (6.9 kB) So this is safe to apply because pip is smart enough to pick the version of Janus that works for that Python version.

From this diff between 0.7.0 and 1.0: https://github.com/aio-libs/janus/compare/v0.7.0...v1.0.0

It looks like the only change relevant to compatibility is loop = asyncio.get_running_loop() directly instead of falling back to asyncio.get_event_loop() if get_running_loop isn't available.

They say they've dropped 3.6 support, but Datasette's tests against 3.6 are still passing.

The release notes for the 0.60a0 alpha will be useful here: https://github.com/simonw/datasette/releases/tag/0.60a0

The table page should expose the query both with and without the limit clause. The above gave me back:

sql select id, ACCESS_TYP, UNIT_ID, UNIT_NAME, SUID_NMA, AGNCY_ID, AGNCY_NAME, AGNCY_LEV, AGNCY_TYP, AGNCY_WEB, LAYER, MNG_AG_ID, MNG_AGENCY, MNG_AG_LEV, MNG_AG_TYP, PARK_URL, COUNTY, ACRES, LABEL_NAME, YR_EST, DES_TP, GAP_STS, geometry from CPAD_2020a_Units where "AGNCY_LEV" = :p0 order by id limit 101 But I actually wanted to run a fetch() against a version of that without the order by id limit 101 bit (I wanted to figure out the Extent() of the geometry column) - so I need something like datasette.table_sql_no_order_no_limit.

This should aim to be as consistent as possible with the various arguments to hooks on https://docs.datasette.io/en/stable/plugin_hooks.html

I'm going to merge this to main now. I can continue the refactoring there, but having it in main means I can put out an alpha release with the new hook which will unblock me from running tests against it in this repo: https://github.com/simonw/datasette-leaflet-freedraw/pull/8

I've convinced myself that this plugin hook design is good through this datasette-leaflet-freedraw prototype: https://github.com/simonw/datasette-leaflet-freedraw/blob/e8a16a0fe90656b8d655c02881d23a2b9833281d/datasette_leaflet_freedraw/init.py

I'm happy with how the prototype that used this plugin in datasette-leaflet-freedraw turned out: https://github.com/simonw/datasette-leaflet-freedraw/blob/e8a16a0fe90656b8d655c02881d23a2b9833281d/datasette_leaflet_freedraw/init.py

I could use this hook to add table filtering on a map to the existing datasette-leaflet-freedraw plugin.

Codecov Report

Merging #1559 (71af58d) into main (0663d55) will increase coverage by 0.09%. The diff coverage is 97.97%.

```diff @@ Coverage Diff @@

main #1559 +/-

========================================== + Coverage 91.96% 92.05% +0.09%
========================================== Files 34 34
Lines 4442 4493 +51
========================================== + Hits 4085 4136 +51
Misses 357 357
```

| Impacted Files | Coverage Δ | | |---|---|---| | datasette/plugins.py | 82.35% <ø> (ø) | | | datasette/filters.py | 95.69% <97.67%> (+1.33%) | :arrow_up: | | datasette/hookspecs.py | 100.00% <100.00%> (ø) | | | datasette/views/table.py | 96.21% <100.00%> (+0.13%) | :arrow_up: |

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 0663d55...71af58d. Read the comment docs.

Before I land this I'm going to build one prototype plugin against it to confirm that the new hook is useful in its current shape.

I'll add support for filtering a table by drawing on a map to https://datasette.io/plugins/datasette-leaflet-freedraw

Codecov Report

Merging #1562 (2f008e8) into main (0663d55) will not change coverage. The diff coverage is n/a.

```diff @@ Coverage Diff @@

main #1562 +/-

======================================= Coverage 91.96% 91.96%
======================================= Files 34 34
Lines 4442 4442
======================================= Hits 4085 4085
Misses 357 357
```

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 0663d55...2f008e8. Read the comment docs.

I think this is missing the _macro.html template file but I have that in my Dropbox.

The one slightly weird thing about this hook is how it adds extra_context without an obvious way for plugins to add extra HTML to the templates based on that context.

Maybe I need the proposed mechanism from - #1191

Which has an in-progress PR: - #1204

This already has tests and documentation, and I've used it to refactor out the logic for ?_where= and ?_search= and ?_through=.

Do I like this enough to land it on main?

Also, I think I can still use it to refactor out the Filters code that implements ?col=x and ?col__lt=5 and suchlike.

Documentation for that hook in the PR branch: https://github.com/simonw/datasette/blob/54e9b3972f277431a001e685f78e5dd6403a6d8d/docs/plugin_hooks.rst#filters_from_requestrequest-database-table-datasette

Documentation of the new hook in the PR: https://github.com/simonw/datasette/blob/54e9b3972f277431a001e685f78e5dd6403a6d8d/docs/plugin_hooks.rst#filters_from_requestrequest-database-table-datasette