html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,issue,performed_via_github_app
https://github.com/simonw/datasette/issues/1696#issuecomment-1407767434,https://api.github.com/repos/simonw/datasette/issues/1696,1407767434,IC_kwDOBm6k_c5T6NOK,193185,2023-01-29T20:56:20Z,2023-01-29T20:56:20Z,CONTRIBUTOR,"I did some horrible things in https://github.com/cldellow/datasette-ui-extras/issues/2 to enable this in my plugin -- example here: https://dux-demo.fly.dev/cooking/posts?_facet=owner_user_id&owner_user_id=67

The implementation relies on two things:

- a `filters_from_request` hook that adds a good human description (unfortunately, without the benefit of the CSS styling you mention)
- doing something evil to hijack the `exact` and `not` operators in the `Filters` class. We can't leave them as is, or we'll get 2 human descriptions -- the built-in Datasette one and the one from my plugin. We can't remove them, or the filters UI will stop supporting the `=` and `!=` operators

This got me thinking: it'd be neat if the list of operators that the filters UI supported wasn't a closed set.

A motivating example: adding a geospatial `NEAR` operator. Ideally it'd take two arguments - a target point and a radius, so you could express a filter like `find me all rows whose lat/lng are within 10km of 43.4516° N, 80.4925° W`. (Optionally, the UI could be enhanced if the geonames database was loaded and queried, so a user could say `find me all rows whose lat/lng are within 10km of Kitchener, ON`, and the city gets translated to a lat/lng for them)","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1186696202,
https://github.com/simonw/datasette/pull/2008#issuecomment-1407733793,https://api.github.com/repos/simonw/datasette/issues/2008,1407733793,IC_kwDOBm6k_c5T6FAh,9599,2023-01-29T18:17:40Z,2023-01-29T18:17:40Z,OWNER,"> We don't have any performance tests yet - would be a useful thing to add, I've not built anything like that before (at least not in CI, I've always done as-hoc performance testing using something like Locust) so I don't have a great feel for how it could work.

Had an interesting conversation about this just now: https://fedi.simonwillison.net/@simon/109773800944614366

There's a risk that different runs will return different results due to the shared resource nature of GitHub Actions runners, but a good fix for that is to run comparative tests where you run the benchmark against e.g. both `main` and the incoming PR branch and report back on any differences.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1560982210,
https://github.com/simonw/datasette/pull/2008#issuecomment-1407716963,https://api.github.com/repos/simonw/datasette/issues/2008,1407716963,IC_kwDOBm6k_c5T6A5j,193185,2023-01-29T17:04:03Z,2023-01-29T17:04:03Z,CONTRIBUTOR,"Performance tests - I think most places don't have them as a formal gate enforced by CI. TypeScript and scalac seem to have tests that run to capture timings. The timings are included by a bot as a comment or build check, and also stored in a database so you can graph changes over time to spot regressions. Probably overkill for Datasette!

Window functions - oh, good point. Looks like Ubuntu shipped JSON1 support as far back as sqlite 3.11. I'll let this PR linger until there's a way to run against different SQLite versions. For now, I'm shipping this with `datasette-ui-extras`, since I think it's OK for a plugin to enforce a higher minimum requirement.

Tests - there actually did end up being test changes to capture the undercount bug of the current implementation, so the current implementation would fail against the new tests.

Perhaps a non-window function version could be written that uses `random()` instead of `row_number() over ()` in order to get a unique key. It's technically not unique, but in practice, I imagine it'll work well.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1560982210,
https://github.com/simonw/datasette/pull/2008#issuecomment-1407568923,https://api.github.com/repos/simonw/datasette/issues/2008,1407568923,IC_kwDOBm6k_c5T5cwb,9599,2023-01-29T05:47:36Z,2023-01-29T05:47:36Z,OWNER,"> I don't know how/if you do automated tests for performance, so I haven't changed any of the tests.

We don't have any performance tests yet - would be a useful thing to add, I've not built anything like that before (at least not in CI, I've always done as-hoc performance testing using something like Locust) so I don't have a great feel for how it could work.

I see not having to change the tests at all for this change as a really positive sign. If you find any behaviour differences between this and the previous that's a sign we should add a mother test or two specifying the behaviour we want.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1560982210,
https://github.com/simonw/datasette/pull/2008#issuecomment-1407567753,https://api.github.com/repos/simonw/datasette/issues/2008,1407567753,IC_kwDOBm6k_c5T5ceJ,9599,2023-01-29T05:39:54Z,2023-01-29T05:40:34Z,OWNER,"I absolutely _love_ this performance boost - really nice find.

One concern: this will be the first time Datasette ships a core feature that uses window functions.

Window functions were added to SQLite in [version 3.25.0](https://www.sqlite.org/releaselog/3_25_0.html) on 2018-09-15 - which means it's still very common for Datasette to run on versions that don't yet support them.

So I see two options:
- Detect window function support and switch between the old implementation and this better, new one
- Detect window functions and disable the facet-by-JSON feature entirely if they are missing

I like the first option a bit better.

This also leads to a tricky CI challenge: Datasette needs to be able to run its test suite against more than one SQLite version to confidently test this feature going forward.

I don't yet have a good GitHub Actions recipe for this, but I _really_ need one - for `sqlite-utils` too.

Might be able to use this trick for that: https://til.simonwillison.net/sqlite/ld-preload","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1560982210,
https://github.com/simonw/datasette/pull/2008#issuecomment-1407561308,https://api.github.com/repos/simonw/datasette/issues/2008,1407561308,IC_kwDOBm6k_c5T5a5c,193185,2023-01-29T04:50:50Z,2023-01-29T04:50:50Z,CONTRIBUTOR,I pushed a revised version which ends up being faster -- the example which currently takes 4 seconds now runs in 500ms.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1560982210,
https://github.com/simonw/datasette/pull/2008#issuecomment-1407558284,https://api.github.com/repos/simonw/datasette/issues/2008,1407558284,IC_kwDOBm6k_c5T5aKM,193185,2023-01-29T04:23:58Z,2023-01-29T04:24:27Z,CONTRIBUTOR,"Ack, this PR is broken. I see now that the `inner.*` is necessary for ensuring the correct count in the face of rows having duplicate values in views.

That fixes the overcounting, but I think can undercount when the rows have the same data, eg a view like:

```sql
SELECT '[""bar""]' tags UNION ALL SELECT '[""bar""]'
```

will produce a count of `{""bar"": 1 }`, when it should be `{""bar"": 2}`. In fact, this could apply in tables without primary keys, too.

If `inner` came from a base table that had a primary key or a rowid, we could use those column(s) to solve that case.

I guess a general solution would be to compute a window function so we have a distinct ID for each row. Will fiddle to see if I can get that working.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1560982210,
https://github.com/simonw/datasette/issues/1973#issuecomment-1407523547,https://api.github.com/repos/simonw/datasette/issues/1973,1407523547,IC_kwDOBm6k_c5T5Rrb,193185,2023-01-29T00:40:31Z,2023-01-29T00:40:31Z,CONTRIBUTOR,"A +1 for switching to `CustomRow`: I think you currently only get a `CustomRow` if the result set had a column that was an fkey ([this code](https://github.com/simonw/datasette/blob/3c352b7132ef09b829abb69a0da0ad00be5edef9/datasette/views/table.py#L667-L682))

Otherwise you get vanilla `sqlite3.Row`s, which will fail if you try to access `.columns` or lookup the cell by name, which surprised me recently","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1515815014,