.status_code
```
This passes most of the time, but very occasionally fails - in this case in Python 3.7
It seems to only fail for the `article,k` compound primary key test.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1919/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1469796454,I_kwDOBm6k_c5Xm1Bm,1920,Document Datasette.metadata() method,25778,open,0,,,0,2022-11-30T15:10:36Z,2022-11-30T15:10:36Z,,CONTRIBUTOR,,"Code is here: https://github.com/simonw/datasette/blob/main/datasette/app.py#L503
This will be the official way to access metadata from plugins.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1920/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1469821027,I_kwDOBm6k_c5Xm7Bj,1921,Document methods to get canned queries,25778,open,0,,,0,2022-11-30T15:26:33Z,2022-11-30T23:34:21Z,,CONTRIBUTOR,,"Two methods will get canned queries for a Datasette instance:
[`Datasette.get_canned_queries`](https://github.com/simonw/datasette/blob/main/datasette/app.py#L575) will return all canned queries for a database that an `actor` can see.
[`Datasette.get_canned_query`](https://github.com/simonw/datasette/blob/main/datasette/app.py#L592) will return a single canned query by name.
",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1921/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1490576818,I_kwDOBm6k_c5Y2GWy,1943,`/-/permissions` should list available permissions,9599,open,0,,8755003,1,2022-12-11T23:38:03Z,2022-12-15T00:41:37Z,,OWNER,,"> Idea: a `/-/permissions` introspection endpoint for listing registered permissions
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1939#issuecomment-1345691103_
",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1943/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1479920517,I_kwDOBm6k_c5YNcuF,1934,Return number of ignored/replaced items from /-/insert,9599,open,0,,3268330,0,2022-12-06T19:01:58Z,2022-12-06T19:02:03Z,,OWNER,,"Idea from here:
- https://github.com/simonw/sqlite-utils/issues/516",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1934/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1483250004,I_kwDOBm6k_c5YaJlU,1936,Fix /db/table/-/upsert in the API explorer,9599,open,0,,3268330,2,2022-12-08T00:59:34Z,2022-12-08T01:36:02Z,,OWNER,,"Split from:
- #1931
- #1878
This is a bit tricky because the code needs to figure out what the primary keys are for an item, and whether or not `rowid` should be included.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1936/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1486036269,I_kwDOBm6k_c5Ykx0t,1941,Mechanism for supporting key rotation for DATASETTE_SECRET,9599,open,0,,,1,2022-12-09T05:24:53Z,2022-12-09T05:25:20Z,,OWNER,,"Currently if you change `DATASETTE_SECRET` all existing signed tokens - both cookies and API tokens and potentially other things too - will instantly expire.
Adding support for key rotation would allow keys to be rotated on a semi-regular basis without logging everyone out / invalidating every API token instantly.
Can model this on how Django does it: https://github.com/django/django/commit/0dcd549bbe36c060f536ec270d34d9e7d4b8e6c7",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1941/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1487738738,I_kwDOBm6k_c5YrRdy,1942,Option for plugins to request that JSON be served on the page,9599,open,0,,3268330,1,2022-12-10T01:08:53Z,2022-12-10T01:11:30Z,,OWNER,,"Idea came from a conversation with @hydrosquall - what if a Datasette plugin could say ""I'd like the JSON for a page to be included in a variable on the HTML page""?
`datasette-cluster-map` already needs this - the first thing it does when the page loads is `fetch()` a JSON representation of that same data.
This idea fits with my overall goals to unify the JSON and HTML context too.
Refs:
- #1711",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1942/reactions"", ""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 1, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1493404423,I_kwDOBm6k_c5ZA4sH,1948,500 error on permission debug page when testing actors with _r,9599,open,0,,,1,2022-12-13T05:22:03Z,2022-12-13T05:22:19Z,,OWNER,,"
The 500 error is silent unless you are looking at the DevTools network pane.
",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1948/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1493471221,I_kwDOBm6k_c5ZBI_1,1949,`.json` errors should be returned as JSON,9599,open,0,,8755003,10,2022-12-13T06:14:12Z,2022-12-15T00:46:27Z,,OWNER,,"Eg the error in this issue:
- #1945 ",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1949/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1495716243,I_kwDOBm6k_c5ZJtGT,1952,Improvements to /-/create-token restrictions interface,9599,open,0,,8755003,1,2022-12-14T05:22:39Z,2022-12-14T05:23:13Z,,OWNER,,"> It would be neat not to show write permissions against immutable databases too - and not hard from a performance perspective since it doesn't involve hundreds more permission checks.
>
> That will need permissions to grow a flag for if they need a mutable database though, which is a bigger job.
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1947#issuecomment-1350414402_
Also, DO show the `_memory` database there if Datasette was started in `--crossdb` mode.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1952/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1497577017,I_kwDOBm6k_c5ZQzY5,1957,Reconsider row value truncation on query page,9599,open,0,,,1,2022-12-14T23:49:47Z,2022-12-14T23:50:50Z,,OWNER,,"Consider this example: https://ripgrep.datasette.io/repos?sql=select+json_group_array%28full_name%29+from+repos
```sql
select json_group_array(full_name) from repos
```
![CleanShot 2022-12-14 at 15 48 32@2x](https://user-images.githubusercontent.com/9599/207739709-8177f683-f938-49a1-8225-42791fad88fe.png)
My intention here was to get a string of JSON I can copy and paste elsewhere - see: https://til.simonwillison.net/sqlite/compare-before-after-json
The truncation isn't helping here.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1957/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1509783085,I_kwDOBm6k_c5Z_XYt,1969,sql-formatter javascript is not now working with CloudFlare rocketloader,536941,open,0,,,0,2022-12-23T21:14:06Z,2023-01-10T01:56:33Z,,CONTRIBUTOR,,"This is probably not a bug with datasette, but I thought you might want to know, @simonw.
I noticed today that my CloudFlare proxied datasette instance lost the ""Format SQL"" option. I'm pretty sure it was there last week.
In the CloudFlare settings, if I turn off [Rocket Loader](https://developers.cloudflare.com/fundamentals/speed/rocket-loader/), I get the ""Format SQL"" option back.
Rocket Loader works by asynchronously loading the javascript, so maybe there was a recent change that doesn't play well with the asynch loading?
I'm up to date with https://github.com/simonw/datasette/commit/e03aed00026cc2e59c09ca41f69a247e1a85cc89",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1969/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1500636982,I_kwDOBm6k_c5Zcec2,1962,"Alternative, async-friendly pattern for `make_app_client()` and similar - fully retire `TestClient`",9599,open,0,,,1,2022-12-16T17:56:51Z,2022-12-16T21:55:29Z,,OWNER,,"In this issue I replaced a whole bunch of places that used the non-async `app_client` fixture with an async `ds_client` fixture instead:
- #1959
But I didn't get everything, and a lot of tests are still using the old `TestClient` mechanism as a result.
The main work here is replacing all of the `app_client_...` fixtures which use variants on the default client - and changing the tests that call `make_app_client()` to do something else instead.
This requires some careful thought. I need to come up with a really nice pattern for creating variants on the `ds_client` default fixture - and do so in a way that minimizes the number of open files, refs:
- #1843",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1962/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1501778647,I_kwDOBm6k_c5Zg1LX,1964,Cog menu is not keyboard accessible (also no ARIA),9599,open,0,,,1,2022-12-18T06:36:28Z,2022-12-18T06:37:28Z,,OWNER,,"This menu here: https://latest.datasette.io/fixtures/attraction_characteristic
You can tab to it (see the outline) and hit space or enter to open it, but you can't then navigate the items in the open menu using the keyboard.
![cog-menu](https://user-images.githubusercontent.com/9599/208284973-2a04cdab-ed95-4316-979c-67fe5f7787db.gif)
",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1964/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1504352503,I_kwDOBm6k_c5Zqpj3,1968,Allow to hide some queries in metadata.yml,562352,open,0,,,0,2022-12-20T10:45:41Z,2022-12-20T10:45:41Z,,NONE,,"By default all queries are displayed.
But there are many cases where it would be interesting to hide the queries by default:
* the website is targeting non-tech people
* the query is veeeeeery long ([eg.](https://mirabelle.openfoodfacts.org/products/energy_calculator))
* reading the query is not important for the users, they only want to see the result
Of course, the user still could have the option to see the query.
It could be an option in the metadata file:
```yml
databases:
awesome_db:
tables:
products:
hide_sql: true
queries:
great_query:
hide_sql: true
sql: select * from products where code = :barcode
```
The priority could be:
* no option in the metadata and nothing in the URL: query displayed
* hide_sql in the metadata and nothing in the URL: query displayed as asked in the metadata
* hide_sql in the metadata and &_hide_sql= in the URL: query as asked in the URL
See also: #1824
",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1968/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1524867951,I_kwDOBm6k_c5a46Nv,1980,"""Cannot sort table by id"" when sortable_columns is used",9599,open,0,,,2,2023-01-09T03:21:33Z,2023-01-09T03:23:53Z,,OWNER,,"I had an instance with this in `metadata.yml`:
```yaml
databases:
timezones:
tables:
timezones:
sortable_columns:
- tzid
```
When I clicked on the ""Apply"" button here:
It sent me to `/timezones/timezones?_sort=id&id__exact=133` with the error message:
> 500: Cannot sort table by id",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1980/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1524983536,I_kwDOBm6k_c5a5Wbw,1981,Canned query field labels truncated,9599,open,0,,,1,2023-01-09T06:04:24Z,2023-01-09T06:05:44Z,,OWNER,,"Eg here on mobile: https://timezones.datasette.io/timezones/by_point?longitude=-0.1406632&latitude=50.8246776
![107A1894-D1DA-4158-9EA3-40C840DD10E3](https://user-images.githubusercontent.com/9599/211248895-c922ce61-95d3-47ca-9314-dcff7c86afab.jpeg)
",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1981/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1525815985,I_kwDOBm6k_c5a8hqx,1983,Make CustomJSONEncoder a documented public API,9599,open,0,,,3,2023-01-09T15:27:05Z,2023-01-09T15:35:58Z,,OWNER,,It's used by `datasette-geojson` here: https://github.com/eyeseast/datasette-geojson/commit/902bf135a5a33a0dc8264673d00a59a67cb05152,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1983/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1515815014,I_kwDOBm6k_c5aWYBm,1973,render_cell plugin hook's row object is not a sqlite.Row,193185,open,0,,,4,2023-01-01T20:27:46Z,2023-01-29T00:40:31Z,,CONTRIBUTOR,,"From https://docs.datasette.io/en/stable/plugin_hooks.html#render-cell-row-value-column-table-database-datasette:
> row - sqlite.Row
> The SQLite row object that the value being rendered is part of
This appears to actually be a [CustomRow](https://github.com/simonw/datasette/blob/f0fadc28ddb9f82e5cc1ecaa51e8a342eb6dc528/datasette/utils/__init__.py#L773-L789), but I think that's unrelated to my issue.
I have a table:
```sql
CREATE TABLE IF NOT EXISTS ""dss_job_stats""(
job_id integer not null references dss_job(id) on delete cascade,
host text not null,
// other columns elided as irrelevant
primary key (job_id, host)
);
```
On datasette 0.63.2, the `render_cell` hook receives a `row` value that looks like:
```
CustomRow([('job_id', {'value': 2, 'label': '2'}), ('host', 'cldellow.com')])
```
I expected the `job_id` value to be `2`, but it's actually `{'value': 2, 'label': '2'}`.
I can work around this, but was wondering if this was intended behaviour?",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1973/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1516815571,I_kwDOBm6k_c5aaMTT,1975,_col=id can cause id column to export twice in CSV export,9599,open,0,,,0,2023-01-03T00:25:15Z,2023-01-03T00:25:21Z,,OWNER,,"https://datasette.simonwillison.net/simonwillisonblog/blog_entry.csv?_col=id&_col=title&_col=body&_labels=on&_size=1
```csv
id,id,title,body
1,1,WaSP Phase II,""The Web Standards project has launched Phase II.
""
```
That should not have two `id` columns.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1975/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1529707837,I_kwDOBm6k_c5bLX09,1988,Reconsider pattern where plugins could break existing template context,9599,open,0,,3268330,4,2023-01-11T21:13:43Z,2023-01-11T21:25:05Z,,OWNER,,"> I hadn't run into an issue with plugins like `datasette-template-sql` interfering with the existing context for other features before! Definitely not a good thing.
_Originally posted by @simonw in https://github.com/simonw/datasette-write/issues/6#issuecomment-1379490596_
",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1988/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1531991339,I_kwDOBm6k_c5bUFUr,1989,Suggestion: Hiding columns,116795,open,0,,,3,2023-01-13T09:33:32Z,2023-03-31T06:18:05Z,,NONE,,As there's the possibility of [hiding tables](https://docs.datasette.io/en/stable/metadata.html#hiding-tables) - I've run into the **need of hiding specific columns** - data that's either not relevant for public or can't be shown due to privacy reasons. ,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1989/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1532000914,I_kwDOBm6k_c5bUHqS,1990,Suggestion: Highlight error messages ('These facets timed out'),116795,open,0,,,0,2023-01-13T09:40:58Z,2023-01-13T09:40:58Z,,NONE,,"I had trouble figuring out why faceting didn't work in some instances, it took a while before I noticed the _These facets timed out_ notice.
It might help if that would be highlighted, or fading out highlight - if one might think it would be too visually disturbing.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1990/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1533673397,I_kwDOBm6k_c5baf-1,1991,fts5 tables are not auto-detected and hidden,83819,open,0,,,0,2023-01-15T06:00:42Z,2023-01-20T04:54:24Z,,NONE,,"I set up a [Datasette instance](https://huggingface.co/spaces/Sygil/INE-dataset-explorer/tree/main) and was following the docs on full-text search.
When I used fts4, datasette automatically hid the FTS tables and added the FTS search box where appropriate, but when I changed to fts5 it no longer does either.
If I [manually set](https://huggingface.co/spaces/keturn/INED-datasette/blob/main/metadata.json#L9) `fts_table` for a view, then search does work as expected.
My table and view creation code looks like this:
```py
connection.execute(""""""CREATE TABLE IF NOT EXISTS
captions(image_key text PRIMARY KEY, caption text NOT NULL)
"""""")
connection.execute(""""""CREATE VIRTUAL TABLE
captions_fts USING
fts5(caption, image_key UNINDEXED, content=captions)
"""""")
```",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1991/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1536851861,I_kwDOBm6k_c5bmn-V,1994,Stuck on loading screen,10913053,open,0,,,1,2023-01-17T18:33:49Z,2023-01-23T08:21:08Z,,NONE,,"Can’t actually open it!
Downloaded today from the releases tab
Running macOS13.1
```
bin/python3.9 --version
Python 3.9.6
Took 83ms
bin/python3.9 --version
Python 3.9.6
Took 113ms
bin/pip install datasette>=0.59 datasette-app-support>=0.11.6 datasette-vega>=0.6.2 datasette-cluster-map>=0.17.1 datasette-pretty-json>=0.2.1 datasette-edit-schema>=0.4 datasette-configure-fts>=1.1 datasette-leaflet>=0.2.2 --disable-pip-version-check
Requirement already satisfied: datasette>=0.59 in lib/python3.9/site-packages (0.63)
Requirement already satisfied: datasette-app-support>=0.11.6 in lib/python3.9/site-packages (0.11.6)
Requirement already satisfied: datasette-vega>=0.6.2 in lib/python3.9/site-packages (0.6.2)
Requirement already satisfied: datasette-cluster-map>=0.17.1 in lib/python3.9/site-packages (0.17.2)
Requirement already satisfied: datasette-pretty-json>=0.2.1 in lib/python3.9/site-packages (0.2.2)
Requirement already satisfied: datasette-edit-schema>=0.4 in lib/python3.9/site-packages (0.5.1)
Requirement already satisfied: datasette-configure-fts>=1.1 in lib/python3.9/site-packages (1.1)
Requirement already satisfied: datasette-leaflet>=0.2.2 in lib/python3.9/site-packages (0.2.2)
Requirement already satisfied: click>=7.1.1 in lib/python3.9/site-packages (from datasette>=0.59) (8.1.3)
Requirement already satisfied: hupper>=1.9 in lib/python3.9/site-packages (from datasette>=0.59) (1.10.3)
Requirement already satisfied: pint>=0.9 in lib/python3.9/site-packages (from datasette>=0.59) (0.20.1)
Requirement already satisfied: PyYAML>=5.3 in lib/python3.9/site-packages (from datasette>=0.59) (6.0)
Requirement already satisfied: httpx>=0.20 in lib/python3.9/site-packages (from datasette>=0.59) (0.23.0)
Requirement already satisfied: aiofiles>=0.4 in lib/python3.9/site-packages (from datasette>=0.59) (22.1.0)
Requirement already satisfied: asgi-csrf>=0.9 in lib/python3.9/site-packages (from datasette>=0.59) (0.9)
Requirement already satisfied: asgiref>=3.2.10 in lib/python3.9/site-packages (from datasette>=0.59) (3.5.2)
Requirement already satisfied: uvicorn>=0.11 in lib/python3.9/site-packages (from datasette>=0.59) (0.19.0)
Requirement already satisfied: itsdangerous>=1.1 in lib/python3.9/site-packages (from datasette>=0.59) (2.1.2)
Requirement already satisfied: click-default-group-wheel>=1.2.2 in lib/python3.9/site-packages (from datasette>=0.59) (1.2.2)
Requirement already satisfied: janus>=0.6.2 in lib/python3.9/site-packages (from datasette>=0.59) (1.0.0)
Requirement already satisfied: pluggy>=1.0 in lib/python3.9/site-packages (from datasette>=0.59) (1.0.0)
Requirement already satisfied: Jinja2>=2.10.3 in lib/python3.9/site-packages (from datasette>=0.59) (3.1.2)
Requirement already satisfied: mergedeep>=1.1.1 in lib/python3.9/site-packages (from datasette>=0.59) (1.3.4)
Requirement already satisfied: sqlite-utils in lib/python3.9/site-packages (from datasette-app-support>=0.11.6) (3.30)
Requirement already satisfied: packaging in lib/python3.9/site-packages (from datasette-app-support>=0.11.6) (21.3)
Requirement already satisfied: python-multipart in lib/python3.9/site-packages (from asgi-csrf>=0.9->datasette>=0.59) (0.0.5)
Requirement already satisfied: httpcore<0.16.0,>=0.15.0 in lib/python3.9/site-packages (from httpx>=0.20->datasette>=0.59) (0.15.0)
Requirement already satisfied: certifi in lib/python3.9/site-packages (from httpx>=0.20->datasette>=0.59) (2022.9.24)
Requirement already satisfied: rfc3986[idna2008]<2,>=1.3 in lib/python3.9/site-packages (from httpx>=0.20->datasette>=0.59) (1.5.0)
Requirement already satisfied: sniffio in lib/python3.9/site-packages (from httpx>=0.20->datasette>=0.59) (1.3.0)
Requirement already satisfied: h11<0.13,>=0.11 in lib/python3.9/site-packages (from httpcore<0.16.0,>=0.15.0->httpx>=0.20->datasette>=0.59) (0.12.0)
Requirement already satisfied: anyio==3.* in lib/python3.9/site-packages (from httpcore<0.16.0,>=0.15.0->httpx>=0.20->datasette>=0.59) (3.6.2)
Requirement already satisfied: idna>=2.8 in lib/python3.9/site-packages (from anyio==3.*->httpcore<0.16.0,>=0.15.0->httpx>=0.20->datasette>=0.59) (3.4)
Requirement already satisfied: typing-extensions>=3.7.4.3 in lib/python3.9/site-packages (from janus>=0.6.2->datasette>=0.59) (4.4.0)
Requirement already satisfied: MarkupSafe>=2.0 in lib/python3.9/site-packages (from Jinja2>=2.10.3->datasette>=0.59) (2.1.1)
Requirement already satisfied: tabulate in lib/python3.9/site-packages (from sqlite-utils->datasette-app-support>=0.11.6) (0.9.0)
Requirement already satisfied: python-dateutil in lib/python3.9/site-packages (from sqlite-utils->datasette-app-support>=0.11.6) (2.8.2)
Requirement already satisfied: sqlite-fts4 in lib/python3.9/site-packages (from sqlite-utils->datasette-app-support>=0.11.6) (1.0.3)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in lib/python3.9/site-packages (from packaging->datasette-app-support>=0.11.6) (3.0.9)
Requirement already satisfied: six>=1.5 in lib/python3.9/site-packages (from python-dateutil->sqlite-utils->datasette-app-support>=0.11.6) (1.16.0)
Took 784ms
```
STUCK",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1994/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1538197093,I_kwDOBm6k_c5brwZl,1995,foreign_keys error 500,137183,open,0,,,0,2023-01-18T15:27:36Z,2023-01-18T16:44:01Z,,NONE,,"**Error 500 expected string or bytes-like object**
[espial-new.sqlite3.zip](https://github.com/simonw/datasette/files/10447965/espial-new.sqlite3.zip)
run `datasette espial-new.sqlite3` & click on any table other than `User`
```
/home/jon/.local/lib/python3.10/site-packages/datasette/app.py:814 in │
│ expand_foreign_keys │
│ │
│ 811 │ │ │ from {other_table} │
│ 812 │ │ │ where {other_column} in ({placeholders}) │
│ 813 │ │ """""".format( │
│ ❱ 814 │ │ │ other_column=escape_sqlite(fk[""other_column""]), │
│ 815 │ │ │ label_column=escape_sqlite(label_column), │
│ 816 │ │ │ other_table=escape_sqlite(fk[""other_table""]), │
│ 817 │ │ │ placeholders="", "".join([""?""] * len(set(values))), │
│ │
│ ╭───────────────────────────── locals ──────────────────────────────╮ │
│ │ column = 'user_id' │ │
│ │ database = 'espial-new' │ │
│ │ db = │ │
│ │ fk = { │ │
│ │ │ 'column': 'user_id', │ │
│ │ │ 'other_table': 'user', │ │
│ │ │ 'other_column': None │ │
│ │ } │ │
│ │ foreign_keys = [ │ │
│ │ │ { │ │
│ │ │ │ 'column': 'user_id', │ │
│ │ │ │ 'other_table': 'user', │ │
│ │ │ │ 'other_column': None │ │
│ │ │ } │ │
│ │ ] │ │
│ │ label_column = 'name' │ │
│ │ labeled_fks = {} │ │
│ │ self = │ │
│ │ table = 'bookmark' │ │
│ │ values = [] │ │
│ ╰───────────────────────────────────────────────────────────────────╯ │
│ │
│ /home/jon/.local/lib/python3.10/site-packages/datasette/utils/__init__.py:346 │
│ in escape_sqlite │
│ │
│ 343 │
│ 344 │
│ 345 def escape_sqlite(s): │
│ ❱ 346 │ if _boring_keyword_re.match(s) and (s.lower() not in reserved_words) │
│ 347 │ │ return s │
│ 348 │ else: │
│ 349 │ │ return f""[{s}]"" │
│ │
│ ╭─ locals ─╮ │
│ │ s = None │ │
│ ╰──────────╯ │
╰─────────────────────────────────────────────────────────────────────────────────╯
TypeError: expected string or bytes-like object
Traceback (most recent call last):
File ""/home/jon/.local/lib/python3.10/site-packages/datasette/app.py"", line 1354, in route_path
response = await view(request, send)
File ""/home/jon/.local/lib/python3.10/site-packages/datasette/views/base.py"", line 134, in view
return await self.dispatch_request(request)
File ""/home/jon/.local/lib/python3.10/site-packages/datasette/views/base.py"", line 91, in dispatch_request
return await handler(request)
File ""/home/jon/.local/lib/python3.10/site-packages/datasette/views/base.py"", line 361, in get
response_or_template_contexts = await self.data(request, **data_kwargs)
File ""/home/jon/.local/lib/python3.10/site-packages/datasette/views/table.py"", line 158, in data
return await self._data_traced(request, default_labels, _next, _size)
File ""/home/jon/.local/lib/python3.10/site-packages/datasette/views/table.py"", line 603, in _data_traced
await self.ds.expand_foreign_keys(
File ""/home/jon/.local/lib/python3.10/site-packages/datasette/app.py"", line 814, in expand_foreign_keys
other_column=escape_sqlite(fk[""other_column""]),
File ""/home/jon/.local/lib/python3.10/site-packages/datasette/utils/__init__.py"", line 346, in escape_sqlite
if _boring_keyword_re.match(s) and (s.lower() not in reserved_words):
TypeError: expected string or bytes-like object
INFO: 127.0.0.1:38574 - ""GET /espial-new/bookmark HTTP/1.1"" 500 Internal Server Error
INFO: 127.0.0.1:38574 - ""GET /-/static/app.css?d59929 HTTP/1.1"" 200 OK
```
Schema:
```
CREATE TABLE IF NOT EXISTS ""user""
(
""id"" INTEGER PRIMARY KEY,
""name"" VARCHAR NOT NULL,
""password_hash"" VARCHAR NOT NULL,
""api_token"" VARCHAR NULL,
""private_default"" BOOLEAN NOT NULL,
""archive_default"" BOOLEAN NOT NULL,
""privacy_lock"" BOOLEAN NOT NULL,
CONSTRAINT ""unique_user_name"" UNIQUE (""name"")
);
CREATE TABLE IF NOT EXISTS ""bookmark""
(
""id"" INTEGER PRIMARY KEY,
""user_id"" INTEGER NOT NULL REFERENCES ""user"" ON DELETE RESTRICT ON UPDATE RESTRICT,
""slug"" VARCHAR NOT NULL DEFAULT (Lower(Hex(Randomblob(6)))),
""href"" VARCHAR NOT NULL,
""description"" VARCHAR NOT NULL,
""extended"" VARCHAR NOT NULL,
""time"" TIMESTAMP NOT NULL,
""shared"" BOOLEAN NOT NULL,
""to_read"" BOOLEAN NOT NULL,
""selected"" BOOLEAN NOT NULL,
""archive_href"" VARCHAR NULL,
CONSTRAINT ""unique_user_href"" UNIQUE (""user_id"", ""href""),
CONSTRAINT ""unique_user_slug"" UNIQUE (""user_id"", ""slug"")
);
CREATE TABLE IF NOT EXISTS ""bookmark_tag""
(
""id"" INTEGER PRIMARY KEY,
""user_id"" INTEGER NOT NULL REFERENCES ""user"" ON DELETE RESTRICT ON UPDATE RESTRICT,
""tag"" VARCHAR NOT NULL,
""bookmark_id"" INTEGER NOT NULL REFERENCES ""bookmark"" ON DELETE RESTRICT ON UPDATE RESTRICT,
""seq"" INTEGER NOT NULL,
CONSTRAINT ""unique_user_tag_bookmark_id"" UNIQUE (""user_id"", ""tag"", ""bookmark_id""),
CONSTRAINT ""unique_user_bookmark_id_tag_seq"" UNIQUE (""user_id"", ""bookmark_id"", ""tag"", ""seq"")
);
CREATE TABLE IF NOT EXISTS ""note""
(
""id"" INTEGER PRIMARY KEY,
""user_id"" INTEGER NOT NULL REFERENCES ""user"" ON DELETE RESTRICT ON UPDATE RESTRICT,
""slug"" VARCHAR NOT NULL DEFAULT (Lower(Hex(Randomblob(10)))),
""length"" INTEGER NOT NULL,
""title"" VARCHAR NOT NULL,
""text"" VARCHAR NOT NULL,
""is_markdown"" BOOLEAN NOT NULL,
""shared"" BOOLEAN NOT NULL DEFAULT false,
""created"" TIMESTAMP NOT NULL,
""updated"" TIMESTAMP NOT NULL
);
CREATE INDEX idx_bookmark_time ON bookmark (user_id, time DESC);
CREATE INDEX idx_bookmark_tag_bookmark_id ON bookmark_tag (bookmark_id, id, tag, seq);
CREATE INDEX idx_note_user_created ON note (user_id, created DESC);
```
",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1995/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1557507274,I_kwDOBm6k_c5c1azK,2005,`extra_template_vars` should be OK to return `None`,9599,open,0,,,1,2023-01-26T01:40:45Z,2023-01-26T01:41:50Z,,OWNER,,"Got this exception and had to make sure it always returned `{}`:
```
File "".../python3.11/site-packages/datasette/app.py"", line 1049, in render_template
assert isinstance(extra_vars, dict), ""extra_vars is of type {}"".format(
AssertionError: extra_vars is of type
```",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2005/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1558644003,I_kwDOBm6k_c5c5wUj,2006,Teach `datasette publish` to pin to `datasette<1.0` in a 0.x release,9599,open,0,,3268330,2,2023-01-26T19:17:40Z,2023-01-26T19:20:53Z,,OWNER,,"I just realized that when I ship Datasette 1.0 there may be automated deployments out there which could deploy the 1.0 version by accident, potentially breaking any customizations that aren't compatible with the 1.0 changes.
I can hopefully help avoid that by shipping one last entry in the `0.x` series that ensures `datasette publish` pins to `<1.0` when it installs Datasette itself.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2006/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1551113681,I_kwDOBm6k_c5cdB3R,1998,`datasette --version` should also show the SQLite version,9599,open,0,,,2,2023-01-20T16:11:30Z,2023-01-20T18:19:06Z,,OWNER,,Idea came up here: https://discord.com/channels/823971286308356157/823971286941302908/1066026473003159783,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1998/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1552368054,I_kwDOBm6k_c5ch0G2,2000,rewrite_sql hook,193185,open,0,,,1,2023-01-23T01:02:52Z,2023-01-23T06:08:01Z,,CONTRIBUTOR,,"I'm not sold that this is a good idea, but thought it'd be worth writing up a ticket. Proposal: add a hook like
```python
def rewrite_sql(datasette, database, request, fn, sql, params)
```
It would be called from Database.execute, Database.execute_write, Database.execute_write_script, Database.execute_write_many before running the user's SQL. `fn` would indicate which method was being used, in case that's relevant for the SQL inspection -- for example `execute` only permits a single statement.
The hook could return a SQL statement to be executed instead, or an async function to be awaited on that returned the SQL to be executed.
Plugins that could be written with this hook:
- https://github.com/cldellow/datasette-ersatz-table-valued-functions would use this to avoid monkey-patching
- a plugin to inspect and reject unsafe Spatialite function calls (reported by [Simon in Discord](https://discord.com/channels/823971286308356157/823971286941302908/1066438832293159004))
- a plugin to do more general rewrites of queries to enforce table or row-level security, for example, based on the currently logged in actor's ID
- a plugin to maintain audit tables when users write to a table
- a plugin to cache expensive queries (eg the queries that drive facets) - these could allow stale reads if previously cached, then refresh them in an offline queue
Flaws with this idea:
`execute_fn` and `execute_write_fn` would not go through this hook, which limits the guarantees you can make about it for security purposes.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2000/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1553615704,I_kwDOBm6k_c5cmktY,2001,Datasette is not compatible with SQLite's strict quoting compilation option,406380,open,0,,,4,2023-01-23T19:10:07Z,2023-01-25T04:59:58Z,,NONE,,"I have linked Python3.11 on macOS against recent SQLite that was compiled using `-DSQLITE_DQS=0`. This option disables interpretation of double-quoted identifiers as string literals, described in the SQLite docs as a ""MySQL 3.x misfeature"". See https://www.sqlite.org/quirks.html#dblquote for background.
Datasette uses the double-quote syntax in a number of key places, and is thus completely broken in this environment.
My experience was to `pip install datasette`, then run `datasette serve -I my-data.db`. When I visit `http://127.0.0.1:8001` I get a 500 response.
The error: `sqlite3.OperationalError: no such column: geometry_columns`
The responsible SQL: `'select 1 from sqlite_master where tbl_name = ""geometry_columns""'`
I then installed datasette from GitHub master in development mode and changed the offending SQL to use correct quotes: `""select 1 from sqlite_master where tbl_name = 'geometry_columns'""`.
With this change, I get a little further, but have the same problem with the first table name in my database (in my case, ""Meta""):
```
OperationalError: no such column: Meta
Traceback (most recent call last):
File ""/Users/gwk/external/datasette/datasette/app.py"", line 1522, in route_path
response = await view(request, send)
^^^^^^^^^^^^^^^^^^^^^^^^^
File ""/Users/gwk/external/datasette/datasette/views/base.py"", line 151, in view
return await self.dispatch_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ""/Users/gwk/external/datasette/datasette/views/base.py"", line 105, in dispatch_request
response = await handler(request)
^^^^^^^^^^^^^^^^^^^^^^
File ""/Users/gwk/external/datasette/datasette/views/index.py"", line 70, in get
""fts_table"": await db.fts_table(table),
^^^^^^^^^^^^^^^^^^^^^^^^^
File ""/Users/gwk/external/datasette/datasette/database.py"", line 363, in fts_table
return await self.execute_fn(lambda conn: detect_fts(conn, table))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ""/Users/gwk/external/datasette/datasette/database.py"", line 213, in execute_fn
return await asyncio.get_event_loop().run_in_executor(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ""/usr/local/py/Python.framework/Versions/3.11/lib/python3.11/concurrent/futures/thread.py"", line 58, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ""/Users/gwk/external/datasette/datasette/database.py"", line 211, in in_thread
return fn(conn)
^^^^^^^^
File ""/Users/gwk/external/datasette/datasette/database.py"", line 363, in
return await self.execute_fn(lambda conn: detect_fts(conn, table))
^^^^^^^^^^^^^^^^^^^^^^^
File ""/Users/gwk/external/datasette/datasette/utils/__init__.py"", line 588, in detect_fts
rows = conn.execute(detect_fts_sql(table)).fetchall()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
sqlite3.OperationalError: no such column: Meta
INFO: 127.0.0.1:50258 - ""GET / HTTP/1.1"" 500 Internal Server Error
```
I will try to continue playing with this, but I also hope that the datasette developers will enable this mode in a test environment as I am unlikely to be able to exercise all of the SQL in the codebase, or make a pull request very soon.
Note that the DQS setting compile-time option can be overridden at runtime with calls to the C API:
```
sqlite3_db_config(db, SQLITE_DBCONFIG_DQS_DDL, 0, (void*)0);
sqlite3_db_config(db, SQLITE_DBCONFIG_DQS_DML, 0, (void*)0);
```
As far as I can tell, `sqlite3_db_config` is not exposed in Python, but perhaps we could figure out how to invoke it using `ctypes`.
",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2001/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1554032168,I_kwDOBm6k_c5coKYo,2002,Document how actors are displayed,9599,open,0,,,0,2023-01-24T00:08:49Z,2023-01-24T00:08:49Z,,OWNER,,"https://github.com/simonw/datasette/blob/e4ebef082de90db4e1b8527abc0d582b7ae0bc9d/datasette/utils/__init__.py#L1052-L1056
This logic should be reflected in the documentation on https://docs.datasette.io/en/stable/authentication.html#actors",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2002/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1575880841,I_kwDOBm6k_c5d7giJ,2020,"Documentation refers to ""off"" setting; doesn't seem to work, ""false"" does",1350673,open,0,,,0,2023-02-08T10:38:10Z,2023-02-08T10:38:10Z,,NONE,,"https://docs.datasette.io/en/stable/settings.html#suggest-facets, among others, suggests using ""off"" to disable the setting; however, this doesn't appear to work in the JSON config files, where it apparently needs to be a ""JSON boolean"" and have the values ""true"" or ""false"". Perhaps the Python code is more flexible?...but either way, the documentation probably should mention it.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2020/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1563264257,I_kwDOBm6k_c5dLYUB,2010,Row page should default to card view,9599,open,0,,3268330,1,2023-01-30T21:49:37Z,2023-01-30T21:52:06Z,,OWNER,,"Datasette currently uses the same table layout on the row pages as it does on the table pages:
https://datasette.io/content/pypi_packages?_sort=name&name__exact=datasette-column-inspect
https://datasette.io/content/pypi_packages/datasette-column-inspect
If you shrink down to mobile width you get this instead, on both of those pages:
I think that view, which I think of as the ""card view"", is plain better if you're looking at just a single row - and it (or a variant of it) should be the default presentation on the row page.
",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2010/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1564769997,I_kwDOBm6k_c5dRH7N,2011,"Applied facet did not result in an ""x"" icon to dismiss it",9599,open,0,,,1,2023-01-31T17:57:44Z,2023-01-31T17:58:54Z,,OWNER,,"![CleanShot 2023-01-31 at 09 55 56@2x](https://user-images.githubusercontent.com/9599/215843684-1761a230-d490-4f87-be6d-186319366794.png)
That's against this data https://data.sfgov.org/City-Management-and-Ethics/Supplier-Contracts/cqi5-hm2d imported using https://datasette.io/plugins/datasette-socrata
It's for `Contract Type` of `Non-Purchasing Contract (Rents, etc.)` - so possible that some of the spaces or punctuation in either the name of the value tripped up the code that decides if the X icon should be displayed.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2011/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1564774831,I_kwDOBm6k_c5dRJGv,2012,Missing space in database summary,9599,open,0,,,0,2023-01-31T18:01:13Z,2023-01-31T18:01:13Z,,OWNER,,"Spotted this on an instance index page:
",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2012/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1565179870,I_kwDOBm6k_c5dSr_e,2013,Datasette uses non-standard quoting for identifiers,193185,open,0,,,0,2023-02-01T00:05:39Z,2023-02-01T00:06:30Z,,CONTRIBUTOR,,"Related to #2001, but where #2001 was about literals, this is about identifiers
From https://www.sqlite.org/lang_keywords.html:
> ""keyword"" A keyword in double-quotes is an identifier.
> [keyword] A keyword enclosed in square brackets is an identifier. This is not standard SQL. This quoting mechanism is used by MS Access and SQL Server and is included in SQLite for compatibility.
Datasette uses this quoting here -- https://github.com/simonw/datasette/blob/0b4a28691468b5c758df74fa1d72a823813c96bf/datasette/utils/__init__.py#L345-L349, in some of the other DB access code, and in some of the test fixtures.
Migrating to standard double quote identifiers would make it easier to get Datasette working with alternative backends",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2013/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1571207083,I_kwDOBm6k_c5dprer,2016,Database metadata fields like description are not available in the index page template's context,9993,open,0,,3268330,1,2023-02-05T02:25:53Z,2023-02-05T22:56:43Z,,NONE,,"When looping through `databases` in the index.html template, I'd like to print the description of each database alongside its name. But it appears that isn't passed in from the view, unless I'm missing it. It would be great to have that.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2016/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1571711808,I_kwDOBm6k_c5drmtA,2018,`check_visibility` gives confusing (wrong?) results if permission is `None`,193185,open,0,,,0,2023-02-06T01:03:08Z,2023-02-06T01:03:46Z,,CONTRIBUTOR,,"I'm trying to gate access to an edit UI on the user having `update-row` on the underlying view or table.
I expected [datasette.check_visibility](https://docs.datasette.io/en/latest/internals.html#await-check-visibility-actor-action-none-resource-none-permissions-none) to be a good way to do this:
```python
visible, private = await datasette.check_visibility(
request.actor,
permissions=[
(""update-row"", (database, table)),
],
)
if not visible:
return None
```
But `visible` is returning true, even when there is no explicit `update-row` permission. (In this case, `request.actor` is `None`.)
Based on [the update-row permissions docs](https://docs.datasette.io/en/latest/authentication.html#update-row), I expected this to be default deny, and so no explicit permission would result in false.
I think the root cause is that `check_visibility` calls `ensure_permissions` and expects it to throw if the permission is not available.
But `ensure_permissions` does not throw when `permission_allowed` returns None: https://github.com/simonw/datasette/blob/1.0a2/datasette/app.py#L825-L829",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2018/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1573424830,I_kwDOBm6k_c5dyI6-,2019,Refactor out the keyset pagination code,9599,open,0,,,14,2023-02-06T23:04:00Z,2023-02-08T01:40:46Z,,OWNER,,"While working on:
- #1999
I noticed that some of the most complex code in the existing table view is the code that implements keyset pagination:
https://github.com/simonw/datasette/blob/0b4a28691468b5c758df74fa1d72a823813c96bf/datasette/views/table.py#L417-L493
Extracting that into a utility function would simplify that code a lot.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2019/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1592327343,I_kwDOBm6k_c5e6Pyv,2029,"Sorry Simon, didn't know how else to contact you",5804626,open,0,,,0,2023-02-20T19:02:53Z,2023-02-20T19:02:53Z,,NONE,,"Hi Simon,
Would you be willing to chat with me about Datasette? I have some questions. I am working on a project to evaluate data ingestion tools for a research organization and I ran across Datasette. I have looked through a lot of your documentation, but still have some questions, which are very specific. If you would be willing to write me back about this, my email is laura@renci.org.
Thanks,
Laura",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2029/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1577548579,I_kwDOBm6k_c5eB3sj,2021,Docker images for 1.0 alphas?,1563881,open,0,,,0,2023-02-09T09:35:52Z,2023-02-09T09:35:52Z,,NONE,,"Hi,
would you consider putting 1.0alpha images on Dockerhub?
(Also, how usable are the alphas?)",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2021/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1579973223,I_kwDOBm6k_c5eLHpn,2024,Mention WAL mode in documentation,9599,open,0,,,1,2023-02-10T16:11:10Z,2023-02-10T16:11:53Z,,OWNER,,It's not currently obvious from the docs how you can ensure that Datasette runs well in situations where other processes may update the underlying SQLite files.,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2024/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1590183272,I_kwDOBm6k_c5eyEVo,2027,"How to redirect from ""/"" to a specific db/table",1350673,open,0,,,4,2023-02-18T03:14:01Z,2023-03-08T04:42:22Z,,NONE,,"Using nginx to redirect public IP to the local uvicorn server as 'normal'. I can't figure out how to redirect such that '/' results in accessing the one db/table I want to serve; redirecting / to /db/table breaks some of the CSS; fooling with base_url doesn't seem to help. Can someone explain this, if it's possible?",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2027/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1605959201,I_kwDOBm6k_c5fuP4h,2032,datasette errors when foreign key integrity is enabled,193185,open,0,,,0,2023-03-02T01:27:51Z,2023-03-02T01:31:58Z,,CONTRIBUTOR,,"By default, [SQLite does not enforce foreign key constraints](https://www.sqlite.org/foreignkeys.html#fk_enable). I typically enable these checks by running:
```sql
PRAGMA foreign_keys = ON;
```
inside of a `prepare_connection` hook.
If a plugin causes the schema to change (eg datasette-scraper creating a new table, or datasette-edit-schema changing a column), then https://github.com/simonw/datasette/blob/0b4a28691468b5c758df74fa1d72a823813c96bf/datasette/utils/internal_db.py#L71-L77 will fail with:
```
FOREIGN KEY constraint failed
```
This could be resolved by either:
- deleting from the `tables` column last
- changing the schema so that the foreign keys have [ON DELETE CASCADE](https://www.sqlite.org/foreignkeys.html#fk_actions)
Let me know if you'd be open to a PR that addresses this -- since foreign key constraints aren't enabled by default, I guess it's questionable whether this is a bug. I think I can workaround this by inspecting the database parameter in `prepare_connection` and trying not to enable fkey checks on the `_internal` database.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2032/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1615692818,I_kwDOBm6k_c5gTYQS,2035,Potential feature: special support for `?a=1&a=2` on the query page,9599,open,0,,3268330,14,2023-03-08T18:05:03Z,2023-03-31T16:09:08Z,,OWNER,,"From a discussion on Discord: https://discord.com/channels/823971286308356157/996877076982415491/1082789517062320138
The key idea is to make it easier for people to implement `where id in (...)` that's populated from query string arguments.
What if you could add `?id=11&id=32&id=62` to the URL and have that made available as a list that can be used in the query?",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2035/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1618249044,I_kwDOBm6k_c5gdIVU,2038,Consider a `strict_templates` setting,9599,open,0,,,2,2023-03-10T02:09:13Z,2023-03-10T02:11:06Z,,OWNER,,"A setting which turns on Jinja strict mode, so any templates that access undefined variables raise a hard error.
Prototype here:
```diff
diff --git a/datasette/app.py b/datasette/app.py
index 40416713..1428a3f0 100644
--- a/datasette/app.py
+++ b/datasette/app.py
@@ -200,6 +200,7 @@ SETTINGS = (
""Allow display of SQL trace debug information with ?_trace=1"",
),
Setting(""base_url"", ""/"", ""Datasette URLs should use this base path""),
+ Setting(""strict_templates"", False, ""Raise errors for undefined template variables""),
)
_HASH_URLS_REMOVED = ""The hash_urls setting has been removed, try the datasette-hashed-urls plugin instead""
OBSOLETE_SETTINGS = {
@@ -399,11 +400,14 @@ class Datasette:
),
]
)
+ env_extras = {}
+ if self.setting(""strict_templates""):
+ env_extras[""undefined""] = StrictUndefined
self.jinja_env = Environment(
loader=template_loader,
autoescape=True,
enable_async=True,
- undefined=StrictUndefined,
+ **env_extras,
)
self.jinja_env.filters[""escape_css_string""] = escape_css_string
self.jinja_env.filters[""quote_plus""] = urllib.parse.quote_plus
```
Explored this idea a bit in:
- #1999",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2038/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1620515757,I_kwDOBm6k_c5glxut,2039,Subtle bug with `--load-extension` and `--static` flags with absolute Windows paths with`C:\`,15178711,open,0,,,0,2023-03-12T21:18:52Z,2023-03-12T21:18:52Z,,CONTRIBUTOR,,"From the Datasette discord: A user tried running the following command on windows:
```
datasette --load-extension=""C:\spatialite\mod_spatialite-5.0.1-win-x86\mod_spatialite.dll""
```
This failed with `""The specified module could not be found""`, because the entrypoint option introduced in #1789 splits the input differently. Instead of loading the extension found at `""C:\spatialite\mod_spatialite-5.0.1-win-x86\mod_spatialite.dll""`, it instead tried to load the extension at `""C""` with entrypoint `""\spatialite\mod_spatialite-5.0.1-win-x86\mod_spatialite.dll"".
This is hard because most absolute windows paths have a colon in them, like `C:\foo.txt` or `D:\bar.txt`. I'd image the `--static` flag is also vulnerable to this type of bug.
The ""solution"" is to use a relative path instead, but that doesn't feel that great. ",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2039/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1636616315,I_kwDOBm6k_c5hjMh7,2042,Gather feedback on new ?_extra= design,9599,open,0,,,0,2023-03-22T23:07:43Z,2023-03-22T23:08:19Z,,OWNER,,"Now that I've landed:
- #1999
See also:
- #262
I want to get some feedback from people on the design of the new `?_extra=` feature, before freezing it into Datasette 1.0.
The big change is that the default JSON representation is now MUCH slimmer - it only gives you keys for `""next""` and `""rows""`, where rows is a list of JSON objects (not a list of arrays as was previously the default) - for example https://latest.datasette.io/fixtures/sortable.json
If you want extra stuff you can ask for it with the new `?_extra=` parameter - e.g. https://latest.datasette.io/fixtures/sortable.json?_extra=columns&_extra=suggested_facets
You can use `?_extra=extras` to see a list of available extras: https://latest.datasette.io/fixtures/sortable.json?_extra=extras
",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2042/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1641013220,I_kwDOBm6k_c5hz9_k,2045,First column on a view page has no facet option in cog menu,9599,open,0,,3268330,0,2023-03-26T18:02:47Z,2023-03-26T18:02:48Z,,OWNER,,"e.g. first column on this page - cog menu has no option to facet.
https://datasette.io/content/tools
",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2045/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1657861026,I_kwDOBm6k_c5i0POi,2054,"Make detailed notes on how table, query and row views work right now",9599,open,0,,,13,2023-04-06T18:21:09Z,2023-04-07T20:14:38Z,,OWNER,,"Research to help influence the following:
- #2049
- #2053
- #2050
- #262 ",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2054/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,