html_url,issue_url,id,node_id,user,user_label,created_at,updated_at,author_association,body,reactions,issue,issue_label,performed_via_github_app
https://github.com/simonw/datasette/issues/1939#issuecomment-1343734812,https://api.github.com/repos/simonw/datasette/issues/1939,1343734812,IC_kwDOBm6k_c5QF8Qc,9599,simonw,2022-12-09T01:57:07Z,2022-12-09T01:57:07Z,OWNER,"This search is better:
datasette permission_allowed -user:simonw -path:datasette/** -path:docs/** -path:tests/** language:python
That returns 11 results: https://cs.github.com/?scopeName=All+repos&scope=&q=datasette+permission_allowed+-user%3Asimonw+-path%3Adatasette%2F**+-path%3Adocs%2F**+-path%3Atests%2F**+language%3Apython
3 are forks of my repos. The rest are all by four users:
- [20after4/ddd](https://github.com/20after4/ddd)
- [emg110/datasette-graphql](https://github.com/emg110/datasette-graphql)
- [next-LI/datasette-csv-importer](https://github.com/next-LI/datasette-csv-importer)
- [next-LI/datasette-demo](https://github.com/next-LI/datasette-demo)
- [next-LI/datasette-live-config](https://github.com/next-LI/datasette-live-config)
- [next-LI/datasette-live-permissions](https://github.com/next-LI/datasette-live-permissions)
- [next-LI/datasette-search-all](https://github.com/next-LI/datasette-search-all)
- [next-LI/datasette-surveys](https://github.com/next-LI/datasette-surveys)
- [next-LI/datasette-write](https://github.com/next-LI/datasette-write)
- [rclement/datasette-dashboards](https://github.com/rclement/datasette-dashboards)
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1485757511,register_permissions(datasette) plugin hook,
https://github.com/simonw/datasette/issues/1939#issuecomment-1343728929,https://api.github.com/repos/simonw/datasette/issues/1939,1343728929,IC_kwDOBm6k_c5QF60h,9599,simonw,2022-12-09T01:48:11Z,2022-12-09T01:52:33Z,OWNER,"This code search shows a bunch of repos I don't know about that would be affected by this change: https://cs.github.com/?scopeName=All+repos&scope=&q=datasette+permission_allowed+-user%3Asimonw#
These (and likely more):
Repositories
- [20after4/ddd](https://github.com/20after4/ddd)
- [next-LI/datasette-csv-importer](https://github.com/next-LI/datasette-csv-importer)
- [digital-land/datasette](https://github.com/digital-land/datasette)
- [mroswell/datasette](https://github.com/mroswell/datasette)
- [next-LI/datasette-live-config](https://github.com/next-LI/datasette-live-config)
- [keladhruv/datasette](https://github.com/keladhruv/datasette)
- [RhetTbull/datasette](https://github.com/RhetTbull/datasette)
- [chriswedgwood/datasette](https://github.com/chriswedgwood/datasette)
- [boan-anbo/datasette](https://github.com/boan-anbo/datasette)
- [MattTriano/datasette](https://github.com/MattTriano/datasette)
- [incadenza/datasette](https://github.com/incadenza/datasette)
- [robdyke/datasette](https://github.com/robdyke/datasette)
- [ctb/datasette](https://github.com/ctb/datasette)
- [eyeseast/datasette](https://github.com/eyeseast/datasette)
- [symbol-management/api-match-audit](https://github.com/symbol-management/api-match-audit)
Actually a lot of those are forks of Datasette itself - so maybe this is manageable?
Would be nice if I could come up with a GitHub search that excluded any repos with ""datasette"" as their exact name.
https://docs.github.com/en/search-github/github-code-search/understanding-github-code-search-syntax#using-qualifiers says:
> **Note:** The new code search beta does not currently support regular expressions or partial matching for repository names, so you will have to type the entire repository name (including the user prefix) for the `repo:` qualifier to work.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1485757511,register_permissions(datasette) plugin hook,
https://github.com/simonw/datasette/issues/1939#issuecomment-1343727184,https://api.github.com/repos/simonw/datasette/issues/1939,1343727184,IC_kwDOBm6k_c5QF6ZQ,9599,simonw,2022-12-09T01:45:15Z,2022-12-09T01:45:15Z,OWNER,"Moving the concept of the default for the permission into this registry warrants a redesign of this method anyway:
https://github.com/simonw/datasette/blob/e539c1c024bc62d88df91d9107cbe37e7f0fe55f/datasette/app.py#L706","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1485757511,register_permissions(datasette) plugin hook,
https://github.com/simonw/datasette/issues/1939#issuecomment-1343724732,https://api.github.com/repos/simonw/datasette/issues/1939,1343724732,IC_kwDOBm6k_c5QF5y8,9599,simonw,2022-12-09T01:40:44Z,2022-12-09T01:43:25Z,OWNER,"```python
Permission = collections.namedtuple(
""Permission"", (""name"", ""abbr"", ""takes_database"", ""takes_table"", ""default"")
)
```
I don`t think that design is quite right.
- Elsewhere in the code the concept is called an ""action"" rather than a ""permission"" - I think I can stick with the `Permission` name here though, it's pretty clear
- `takes_database` - is `takes_` the right verb here?
- `takes_table` can also refer to a SQL view or a canned named query
A question that was raised by the work in #1938 is whether you should be able to grant a permission like `insert-row` at the instance or database level - and if so, what does that look like? I think you should be able to do that, it doesn't make sense to have to grant it explicitly for every single table.
So maybe `takes_table` and `takes_database` are the right names here? But `table` is still bad because it doesn't reflect views and canned queries.
One thought is to use `resource` - but that will require a bunch of breaking changes to the existing APIs which treat resource as a tuple. Now's the best time to do that though before Datasette 1.0.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1485757511,register_permissions(datasette) plugin hook,
https://github.com/simonw/datasette/issues/1881#issuecomment-1301645921,https://api.github.com/repos/simonw/datasette/issues/1881,1301645921,IC_kwDOBm6k_c5NlYph,9599,simonw,2022-11-03T05:10:05Z,2022-12-09T01:38:21Z,OWNER,"I'd love to come up with a good short name for the second part of the resource two-tuple, the thing which is usually the name of a table but could also be the name of a SQL view or the name of a canned query.
Idea 8th December: why not call it resource? A resource could be a thing that lives inside a database.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1434094365,Tool for simulating permission checks against actors,
https://github.com/simonw/datasette/issues/1939#issuecomment-1343722020,https://api.github.com/repos/simonw/datasette/issues/1939,1343722020,IC_kwDOBm6k_c5QF5Ik,9599,simonw,2022-12-09T01:36:05Z,2022-12-09T01:36:16Z,OWNER,"I originally added `permissions.py` for the permission debug tool in https://github.com/simonw/datasette/commit/c51d9246b996a2831c9bd6a1e205f6cb48b9a5f3 - I don't think anything else uses it yet.
- #1881","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1485757511,register_permissions(datasette) plugin hook,
https://github.com/simonw/datasette/issues/1939#issuecomment-1343721522,https://api.github.com/repos/simonw/datasette/issues/1939,1343721522,IC_kwDOBm6k_c5QF5Ay,9599,simonw,2022-12-09T01:35:15Z,2022-12-09T01:35:15Z,OWNER,"One concern I have about this: there are a bunch of existing plugins that do stuff with permissions that won't currently be using this hook.
Do I break those plugins, forcing new releases of them for compatibility with Datasette 1.0?
Or maybe I keep them working, but until they've upgraded to register their permissions there are things about them that won't work - e.g. you won't be able to configure their permissions in `metadata.yml` until they release something that does this hook.
Best thing is probably for me to get this working in core first and then evaluate the impact it would have on existing plugins once I have some running code.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1485757511,register_permissions(datasette) plugin hook,
https://github.com/simonw/datasette/issues/1636#issuecomment-1343715746,https://api.github.com/repos/simonw/datasette/issues/1636,1343715746,IC_kwDOBm6k_c5QF3mi,9599,simonw,2022-12-09T01:27:41Z,2022-12-09T01:27:58Z,OWNER,"I may need to consult this file to figure out if the permission that is being checked can act at the database/table/instance level:
https://github.com/simonw/datasette/blob/e539c1c024bc62d88df91d9107cbe37e7f0fe55f/datasette/permissions.py#L1-L19","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1138008042,"""permissions"" propery in metadata for configuring arbitrary permissions",
https://github.com/simonw/datasette/issues/1636#issuecomment-1343446071,https://api.github.com/repos/simonw/datasette/issues/1636,1343446071,IC_kwDOBm6k_c5QE1w3,9599,simonw,2022-12-08T22:16:17Z,2022-12-08T22:16:17Z,OWNER,First draft of documentation: https://datasette--1938.org.readthedocs.build/en/1938/authentication.html#other-permissions-in-metadata,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1138008042,"""permissions"" propery in metadata for configuring arbitrary permissions",
https://github.com/simonw/datasette/pull/1938#issuecomment-1343445885,https://api.github.com/repos/simonw/datasette/issues/1938,1343445885,IC_kwDOBm6k_c5QE1t9,9599,simonw,2022-12-08T22:16:03Z,2022-12-08T22:16:03Z,OWNER,Docs: https://datasette--1938.org.readthedocs.build/en/1938/authentication.html#other-permissions-in-metadata,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1485488236,"""permissions"" blocks in metadata.json/yaml",
https://github.com/simonw/datasette/issues/1636#issuecomment-1343440504,https://api.github.com/repos/simonw/datasette/issues/1636,1343440504,IC_kwDOBm6k_c5QE0Z4,9599,simonw,2022-12-08T22:10:28Z,2022-12-08T22:10:48Z,OWNER,"What if you want to grant `insert-row` to a user for ALL tables in a database, or even for all tables in all databases?
You should be able to do that by putting that in the root `permissions:` block. Need to figure out how the implementation will handle that.
Also: there are some permissions like `view-instance` or `debug-menu` for which putting them at the `database` or `table` or `query` level doesn't actually make any sense.
Ideally the implementation would spot those on startup and refuse to start the server, with a helpful error message.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1138008042,"""permissions"" propery in metadata for configuring arbitrary permissions",
https://github.com/simonw/datasette/pull/1930#issuecomment-1343360006,https://api.github.com/repos/simonw/datasette/issues/1930,1343360006,IC_kwDOBm6k_c5QEgwG,9599,simonw,2022-12-08T21:12:28Z,2022-12-08T21:12:28Z,OWNER,Thanks!,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1473664029,Typo in JSON API `Updating a row` documentation,
https://github.com/simonw/datasette/issues/1867#issuecomment-1341874378,https://api.github.com/repos/simonw/datasette/issues/1867,1341874378,IC_kwDOBm6k_c5P-2DK,9599,simonw,2022-12-08T02:07:06Z,2022-12-08T02:28:02Z,OWNER,"Basic version of this allows you to rename a table:
```
POST /db/table/-/rename
{
""name"": ""new_table_name""
}
```
If a table with that new name already exists you will get an error - unless you pass `""replace"": true` in which case that table will be dropped and replaced by the new one.
```
POST /db/table/-/rename
{
""name"": ""new_table_name"",
""replace"": true
}
```
This is useful because it allows for that atomic replacement operation: upload brand new data into a `_tmp_name` table, then atomic rename and replace after the upload has completed.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1426080014,/db/table/-/rename API (also allows atomic replace),
https://github.com/simonw/datasette/issues/1636#issuecomment-1341854373,https://api.github.com/repos/simonw/datasette/issues/1636,1341854373,IC_kwDOBm6k_c5P-xKl,9599,simonw,2022-12-08T01:43:35Z,2022-12-08T01:43:35Z,OWNER,I'm going to write the documentation for this first.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1138008042,"""permissions"" propery in metadata for configuring arbitrary permissions",
https://github.com/simonw/datasette/issues/1936#issuecomment-1341849735,https://api.github.com/repos/simonw/datasette/issues/1936,1341849735,IC_kwDOBm6k_c5P-wCH,9599,simonw,2022-12-08T01:35:54Z,2022-12-08T01:35:54Z,OWNER,"Running that twice produced this:
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1483250004,Fix /db/table/-/upsert in the API explorer,
https://github.com/simonw/datasette/issues/1936#issuecomment-1341849496,https://api.github.com/repos/simonw/datasette/issues/1936,1341849496,IC_kwDOBm6k_c5P-v-Y,9599,simonw,2022-12-08T01:35:35Z,2022-12-08T01:35:35Z,OWNER,"Related bug: you can send `""id"": null` and it works (it should throw an error):
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1483250004,Fix /db/table/-/upsert in the API explorer,
https://github.com/simonw/datasette/issues/1937#issuecomment-1341848525,https://api.github.com/repos/simonw/datasette/issues/1937,1341848525,IC_kwDOBm6k_c5P-vvN,9599,simonw,2022-12-08T01:34:03Z,2022-12-08T01:34:03Z,OWNER,Check should go somewhere around here: https://github.com/simonw/datasette/blob/dee18ed8ce7af2ab8699bcb5a51a99f48301bc42/datasette/views/database.py#L625-L631,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1483320357,/db/-/create API should require insert-rows permission to use row: or rows: option,
https://github.com/simonw/datasette/pull/1931#issuecomment-1341825314,https://api.github.com/repos/simonw/datasette/issues/1931,1341825314,IC_kwDOBm6k_c5P-qEi,9599,simonw,2022-12-08T01:03:18Z,2022-12-08T01:03:18Z,OWNER,"I broke this test:
```
ds_write =
@pytest.mark.asyncio
async def test_insert_row(ds_write):
token = write_token(ds_write)
response = await ds_write.client.post(
""/data/docs/-/insert"",
json={""row"": {""title"": ""Test"", ""score"": 1.2, ""age"": 5}},
headers={
""Authorization"": ""***"".format(token),
""Content-Type"": ""application/json"",
},
)
expected_row = {""id"": 1, ""title"": ""Test"", ""score"": 1.2, ""age"": 5}
> assert response.status_code == 201
E assert 500 == 201
E + where 500 = .status_code
/home/runner/work/datasette/datasette/tests/test_api_write.py:43: AssertionError
----------------------------- Captured stderr call -----------------------------
Traceback (most recent call last):
File ""/home/runner/work/datasette/datasette/datasette/app.py"", line 1447, in route_path
response = await view(request, send)
File ""/home/runner/work/datasette/datasette/datasette/views/base.py"", line 151, in view
return await self.dispatch_request(request)
File ""/home/runner/work/datasette/datasette/datasette/views/base.py"", line 105, in dispatch_request
response = await handler(request)
File ""/home/runner/work/datasette/datasette/datasette/views/table.py"", line 1228, in post
row_pk_values_for_later = [tuple(row[pk] for pk in pks) for row in rows]
File ""/home/runner/work/datasette/datasette/datasette/views/table.py"", line 1228, in
row_pk_values_for_later = [tuple(row[pk] for pk in pks) for row in rows]
File ""/home/runner/work/datasette/datasette/datasette/views/table.py"", line 1228, in
row_pk_values_for_later = [tuple(row[pk] for pk in pks) for row in rows]
KeyError: 'id'
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1473814539,/db/table/-/upsert,
https://github.com/simonw/datasette/pull/1931#issuecomment-1341821213,https://api.github.com/repos/simonw/datasette/issues/1931,1341821213,IC_kwDOBm6k_c5P-pEd,9599,simonw,2022-12-08T00:58:21Z,2022-12-08T00:58:21Z,OWNER,"In the interests of shipping, I'm going to punt the API explorer to a later issue.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1473814539,/db/table/-/upsert,
https://github.com/simonw/datasette/pull/1931#issuecomment-1339968514,https://api.github.com/repos/simonw/datasette/issues/1931,1339968514,IC_kwDOBm6k_c5P3kwC,9599,simonw,2022-12-06T20:28:47Z,2022-12-06T20:28:47Z,OWNER,"Should the `""return"": true` mode reflect the order in which the rows were provided when the API was called?
I think it should. Since this is small enough to happily fit in Python memory (thanks to the `max_insert_rows` setting) I can load the fresh data from the database and then sort it in Python space before returning it.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1473814539,/db/table/-/upsert,
https://github.com/simonw/datasette/issues/1855#issuecomment-1339952692,https://api.github.com/repos/simonw/datasette/issues/1855,1339952692,IC_kwDOBm6k_c5P3g40,9599,simonw,2022-12-06T20:15:50Z,2022-12-06T20:16:00Z,OWNER,"That commit there https://github.com/simonw/datasette/commit/6da17d5529773dfe41b53ed4ce5a6ecb46ed2457 (which will be squash-merged in a PR later on) made it so that `_r` was correctly copied across from the token to the created actor, and fixed a bug in the code that checks permissions against it for resources.
I needed that mechanism to write a test that exercised different API permissions.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1423336089,`datasette create-token` ability to create tokens with a reduced set of permissions,
https://github.com/simonw/datasette/pull/1931#issuecomment-1339911152,https://api.github.com/repos/simonw/datasette/issues/1931,1339911152,IC_kwDOBm6k_c5P3Wvw,9599,simonw,2022-12-06T19:38:12Z,2022-12-06T19:38:12Z,OWNER,Documentation: https://datasette--1931.org.readthedocs.build/en/1931/json_api.html#upserting-rows,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1473814539,/db/table/-/upsert,
https://github.com/simonw/datasette/issues/1927#issuecomment-1339910494,https://api.github.com/repos/simonw/datasette/issues/1927,1339910494,IC_kwDOBm6k_c5P3Wle,9599,simonw,2022-12-06T19:37:39Z,2022-12-06T19:37:39Z,OWNER,"I'll finish this after I land:
- #1931 ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1473411197,ignore:true/replace:true options for /db/-/create API,
https://github.com/simonw/datasette/issues/1929#issuecomment-1339909159,https://api.github.com/repos/simonw/datasette/issues/1929,1339909159,IC_kwDOBm6k_c5P3WQn,9599,simonw,2022-12-06T19:36:23Z,2022-12-06T19:36:23Z,OWNER,https://docs.datasette.io/en/1.0a1/json_api.html 👍 ,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1473659191,Incorrect link from the API explorer to the JSON API documentation,
https://github.com/simonw/datasette/issues/1929#issuecomment-1339871933,https://api.github.com/repos/simonw/datasette/issues/1929,1339871933,IC_kwDOBm6k_c5P3NK9,9599,simonw,2022-12-06T19:23:48Z,2022-12-06T19:24:17Z,OWNER,"I can do that on this page: https://readthedocs.org/projects/datasette/versions/?version_filter=1.0
I'm making them both active and hidden:
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1473659191,Incorrect link from the API explorer to the JSON API documentation,
https://github.com/simonw/datasette/issues/1929#issuecomment-1339867570,https://api.github.com/repos/simonw/datasette/issues/1929,1339867570,IC_kwDOBm6k_c5P3MGy,9599,simonw,2022-12-06T19:22:47Z,2022-12-06T19:22:47Z,OWNER,"Yeah I should deploy the docs for the alpha versions (the intention is that the docs exactly match the release you are using), thanks for spotting that.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1473659191,Incorrect link from the API explorer to the JSON API documentation,
https://github.com/simonw/sqlite-utils/issues/516#issuecomment-1339834918,https://api.github.com/repos/simonw/sqlite-utils/issues/516,1339834918,IC_kwDOCGYnMM5P3EIm,9599,simonw,2022-12-06T19:00:18Z,2022-12-06T19:00:35Z,OWNER,"Right now the command produces no output at all.
Maybe a `--verbose` mode that writes these numbers to standard error (or even standard output since it's an option)?
Is there a better name than `--verbose` for this? `--summary` perhaps?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1479914599,Feature request: output number of ignored/replaced rows for insert command,
https://github.com/simonw/datasette/pull/1931#issuecomment-1339784569,https://api.github.com/repos/simonw/datasette/issues/1931,1339784569,IC_kwDOBm6k_c5P2315,9599,simonw,2022-12-06T18:16:15Z,2022-12-06T18:17:56Z,OWNER,"Just noticed the insert API returns `{}` when it should return `{""ok"": true}` - will fix that here too.
UPDATE: no it did that already, it was just the documentation that was wrong.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1473814539,/db/table/-/upsert,
https://github.com/simonw/datasette/pull/1931#issuecomment-1339768422,https://api.github.com/repos/simonw/datasette/issues/1931,1339768422,IC_kwDOBm6k_c5P2z5m,9599,simonw,2022-12-06T18:04:59Z,2022-12-06T18:04:59Z,OWNER,"I realized this API should require both the `insert-row` AND the `update-row` permissions, since calls to it could do either one.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1473814539,/db/table/-/upsert,
https://github.com/simonw/datasette/issues/1855#issuecomment-1302815929,https://api.github.com/repos/simonw/datasette/issues/1855,1302815929,IC_kwDOBm6k_c5Np2S5,9599,simonw,2022-11-04T00:19:10Z,2022-12-03T07:05:51Z,OWNER,Added the tests.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1423336089,`datasette create-token` ability to create tokens with a reduced set of permissions,
https://github.com/simonw/datasette/issues/1878#issuecomment-1336100218,https://api.github.com/repos/simonw/datasette/issues/1878,1336100218,IC_kwDOBm6k_c5Po0V6,9599,simonw,2022-12-03T07:02:15Z,2022-12-03T07:02:15Z,OWNER,"Moved this work to a PR:
- #1931","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1432013704,/db/table/-/upsert API,
https://github.com/simonw/datasette/issues/1927#issuecomment-1336099588,https://api.github.com/repos/simonw/datasette/issues/1927,1336099588,IC_kwDOBm6k_c5Po0ME,9599,simonw,2022-12-03T06:58:14Z,2022-12-03T06:58:14Z,OWNER,I have not yet documented the new `insert` and `replace` options.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1473411197,ignore:true/replace:true options for /db/-/create API,
https://github.com/simonw/datasette/issues/1927#issuecomment-1336099533,https://api.github.com/repos/simonw/datasette/issues/1927,1336099533,IC_kwDOBm6k_c5Po0LN,9599,simonw,2022-12-03T06:57:52Z,2022-12-03T06:57:52Z,OWNER,I'm going to push what I have anyway. I'll keep this issue open while I think through the above comment.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1473411197,ignore:true/replace:true options for /db/-/create API,
https://github.com/simonw/datasette/issues/1927#issuecomment-1336099368,https://api.github.com/repos/simonw/datasette/issues/1927,1336099368,IC_kwDOBm6k_c5Po0Io,9599,simonw,2022-12-03T06:56:36Z,2022-12-03T06:56:36Z,OWNER,"Neither of these options make sense if you didn't pass a `""pk""`.
My initial implementation spotted if the `pk` was missing and looked it up from the table, but actually I don't think that makes sense - if you know the table exists and hence don't pass the `pk` you should be using `/-/insert` or `/-/upsert` instead.
So maybe this work should expanded to include validation that checks if the table exists already - and if it does, confirms that the primary key (and maybe even the columns) are the same as for that existing table.
Of course if you only send `row` or `rows` then checking `columns` doesn't completely make sense - but we could check that the rows you have sent are equal to or a subset of the columns in the table. We could even check the column types as well, as seen in:
- #1910 ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1473411197,ignore:true/replace:true options for /db/-/create API,
https://github.com/simonw/datasette/issues/1878#issuecomment-1336094562,https://api.github.com/repos/simonw/datasette/issues/1878,1336094562,IC_kwDOBm6k_c5Poy9i,9599,simonw,2022-12-03T06:27:50Z,2022-12-03T06:29:06Z,OWNER,"This adds it to the API explorer:
```diff
diff --git a/datasette/views/special.py b/datasette/views/special.py
index 1f84b094..1b4a9d3c 100644
--- a/datasette/views/special.py
+++ b/datasette/views/special.py
@@ -316,21 +316,37 @@ class ApiExplorerView(BaseView):
request.actor, ""insert-row"", (name, table)
):
pks = await db.primary_keys(table)
- table_links.append(
- {
- ""path"": self.ds.urls.table(name, table) + ""/-/insert"",
- ""method"": ""POST"",
- ""label"": ""Insert rows into {}"".format(table),
- ""json"": {
- ""rows"": [
- {
- column: None
- for column in await db.table_columns(table)
- if column not in pks
- }
- ]
+ table_links.extend(
+ [
+ {
+ ""path"": self.ds.urls.table(name, table) + ""/-/insert"",
+ ""method"": ""POST"",
+ ""label"": ""Insert rows into {}"".format(table),
+ ""json"": {
+ ""rows"": [
+ {
+ column: None
+ for column in await db.table_columns(table)
+ if column not in pks
+ }
+ ]
+ },
},
- }
+ {
+ ""path"": self.ds.urls.table(name, table) + ""/-/upsert"",
+ ""method"": ""POST"",
+ ""label"": ""Upsert rows into {}"".format(table),
+ ""json"": {
+ ""rows"": [
+ {
+ column: None
+ for column in await db.table_columns(table)
+ if column not in pks
+ }
+ ]
+ },
+ },
+ ]
)
if await self.ds.permission_allowed(
request.actor, ""drop-table"", (name, table)
```
Except it doesn't quite, because the example JSON this generates is invalid as it does not include the primary key column.
(Made me notice that the way example columns are created for `/-/insert` will fail for tables that don't have an auto-incrementing primary key)","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1432013704,/db/table/-/upsert API,
https://github.com/simonw/datasette/issues/1878#issuecomment-1336094470,https://api.github.com/repos/simonw/datasette/issues/1878,1336094470,IC_kwDOBm6k_c5Poy8G,9599,simonw,2022-12-03T06:27:13Z,2022-12-03T06:27:13Z,OWNER,"Tests are going to need to cover both rowid-only and compound primary key tables, including all of the error states.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1432013704,/db/table/-/upsert API,
https://github.com/simonw/datasette/issues/1878#issuecomment-1336094381,https://api.github.com/repos/simonw/datasette/issues/1878,1336094381,IC_kwDOBm6k_c5Poy6t,9599,simonw,2022-12-03T06:26:25Z,2022-12-03T06:26:25Z,OWNER,"Initial prototype:
```diff
diff --git a/datasette/app.py b/datasette/app.py
index 125b4969..282c0984 100644
--- a/datasette/app.py
+++ b/datasette/app.py
@@ -40,7 +40,7 @@ from .views.special import (
PermissionsDebugView,
MessagesDebugView,
)
-from .views.table import TableView, TableInsertView, TableDropView
+from .views.table import TableView, TableInsertView, TableUpsertView, TableDropView
from .views.row import RowView, RowDeleteView, RowUpdateView
from .renderer import json_renderer
from .url_builder import Urls
@@ -1292,6 +1292,10 @@ class Datasette:
TableInsertView.as_view(self),
r""/(?P[^\/\.]+)/(?P
[^\/\.]+)/-/drop$"",
diff --git a/datasette/views/table.py b/datasette/views/table.py
index 7ba78c11..ae0d6366 100644
--- a/datasette/views/table.py
+++ b/datasette/views/table.py
@@ -1074,9 +1074,15 @@ class TableInsertView(BaseView):
def __init__(self, datasette):
self.ds = datasette
- async def _validate_data(self, request, db, table_name):
+ async def _validate_data(self, request, db, table_name, pks, upsert):
errors = []
+ pks_list = []
+ if isinstance(pks, str):
+ pks_list = [pks]
+ else:
+ pks_list = list(pks)
+
def _errors(errors):
return None, errors, {}
@@ -1135,6 +1141,15 @@ class TableInsertView(BaseView):
# Validate columns of each row
columns = set(await db.table_columns(table_name))
for i, row in enumerate(rows):
+ if upsert:
+ # It MUST have the primary key
+ missing_pks = [pk for pk in pks_list if pk not in row]
+ if missing_pks:
+ errors.append(
+ 'Row {} is missing primary key column(s): ""{}""'.format(
+ i, '"", ""'.join(missing_pks)
+ )
+ )
invalid_columns = set(row.keys()) - columns
if invalid_columns:
errors.append(
@@ -1146,7 +1161,7 @@ class TableInsertView(BaseView):
return _errors(errors)
return rows, errors, extras
- async def post(self, request):
+ async def post(self, request, upsert=False):
try:
resolved = await self.ds.resolve_table(request)
except NotFound as e:
@@ -1164,7 +1179,12 @@ class TableInsertView(BaseView):
request.actor, ""insert-row"", resource=(database_name, table_name)
):
return _error([""Permission denied""], 403)
- rows, errors, extras = await self._validate_data(request, db, table_name)
+
+ pks = await db.primary_keys(table_name)
+
+ rows, errors, extras = await self._validate_data(
+ request, db, table_name, pks, upsert
+ )
if errors:
return _error(errors, 400)
@@ -1172,15 +1192,19 @@ class TableInsertView(BaseView):
replace = extras.get(""replace"")
should_return = bool(extras.get(""return"", False))
- # Insert rows
- def insert_rows(conn):
+
+ def insert_or_upsert_rows(conn):
table = sqlite_utils.Database(conn)[table_name]
+ kwargs = {}
+ if upsert:
+ kwargs[""pk""] = pks[0] if len(pks) == 1 else pks
+ else:
+ kwargs = {""ignore"": ignore, ""replace"": replace}
if should_return:
rowids = []
+ method = table.upsert if upsert else table.insert
for row in rows:
- rowids.append(
- table.insert(row, ignore=ignore, replace=replace).last_rowid
- )
+ rowids.append(method(row, **kwargs).last_rowid)
return list(
table.rows_where(
""rowid in ({})"".format("","".join(""?"" for _ in rowids)),
@@ -1188,10 +1212,11 @@ class TableInsertView(BaseView):
)
)
else:
- table.insert_all(rows, ignore=ignore, replace=replace)
+ method_all = table.upsert_all if upsert else table.insert_all
+ method_all(rows, **kwargs)
try:
- rows = await db.execute_write_fn(insert_rows)
+ rows = await db.execute_write_fn(insert_or_upsert_rows)
except Exception as e:
return _error([str(e)])
result = {""ok"": True}
@@ -1200,6 +1225,13 @@ class TableInsertView(BaseView):
return Response.json(result, status=201)
+class TableUpsertView(TableInsertView):
+ name = ""table-upsert""
+
+ async def post(self, request):
+ return await super().post(request, upsert=True)
+
+
class TableDropView(BaseView):
name = ""table-drop""
```
Manual testing reveals that this mostly works... but it's not doing the right thing for `""return"": true` - it always returns an empty list.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1432013704,/db/table/-/upsert API,
https://github.com/simonw/datasette/issues/1878#issuecomment-1336073212,https://api.github.com/repos/simonw/datasette/issues/1878,1336073212,IC_kwDOBm6k_c5Potv8,9599,simonw,2022-12-03T05:38:49Z,2022-12-03T05:38:49Z,OWNER,And on Discord today: https://discord.com/channels/823971286308356157/823971286941302908/1048426072066236536,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1432013704,/db/table/-/upsert API,
https://github.com/simonw/datasette/issues/1878#issuecomment-1336070843,https://api.github.com/repos/simonw/datasette/issues/1878,1336070843,IC_kwDOBm6k_c5PotK7,9599,simonw,2022-12-03T05:37:53Z,2022-12-03T05:37:53Z,OWNER,Also requested here: https://news.ycombinator.com/item?id=33839894,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1432013704,/db/table/-/upsert API,
https://github.com/simonw/datasette/issues/1927#issuecomment-1335984268,https://api.github.com/repos/simonw/datasette/issues/1927,1335984268,IC_kwDOBm6k_c5PoYCM,9599,simonw,2022-12-03T00:26:26Z,2022-12-03T00:26:26Z,OWNER,"Also: the documentation should clarify that you can call this API multiple times when using the `rows` option.
(It will probably grow `""alter"": true` soon too).","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1473411197,ignore:true/replace:true options for /db/-/create API,
https://github.com/simonw/datasette/issues/1928#issuecomment-1335966329,https://api.github.com/repos/simonw/datasette/issues/1928,1335966329,IC_kwDOBm6k_c5PoTp5,9599,simonw,2022-12-02T23:47:11Z,2022-12-02T23:47:11Z,OWNER,Wrote about this extensively here: https://simonwillison.net/2022/Dec/2/datasette-write-api/,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1473481262,Hacker News Datasette write demo,
https://github.com/simonw/datasette/issues/1928#issuecomment-1335870889,https://api.github.com/repos/simonw/datasette/issues/1928,1335870889,IC_kwDOBm6k_c5Pn8Wp,9599,simonw,2022-12-02T21:41:09Z,2022-12-02T21:41:09Z,OWNER,"Got it working!
https://simon.datasette.cloud/data/hacker_news_posts
https://github.com/simonw/scrape-hacker-news-by-domain/blob/main/submit-to-datasette-cloud.sh","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1473481262,Hacker News Datasette write demo,
https://github.com/simonw/datasette/issues/1928#issuecomment-1335870887,https://api.github.com/repos/simonw/datasette/issues/1928,1335870887,IC_kwDOBm6k_c5Pn8Wn,9599,simonw,2022-12-02T21:25:16Z,2022-12-02T21:25:16Z,OWNER,"Here's the change that should submit data to Datasette Cloud: https://github.com/simonw/scrape-hacker-news-by-domain/commit/848bb7e835a9fb87cd656362591835179cd1dc1b
I ran `delete from hacker_news_posts` against my instance so https://simon.datasette.cloud/data/hacker_news_posts is now empty.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1473481262,Hacker News Datasette write demo,
https://github.com/simonw/datasette/issues/1928#issuecomment-1335870884,https://api.github.com/repos/simonw/datasette/issues/1928,1335870884,IC_kwDOBm6k_c5Pn8Wk,9599,simonw,2022-12-02T21:19:58Z,2022-12-02T21:19:58Z,OWNER,"But until I fix this issue:
- https://github.com/simonw/datasette/issues/1927
I need to insert freshly scraped data like this:
```bash
export ROWS=$(
jq -n --argjson rows ""$(cat simonwillison-net.json)"" \
'{ ""rows"": $rows, ""replace"": true }'
)
curl -X POST \
https://simon.datasette.cloud/data/hacker_news_posts/-/insert \
-H ""Content-Type: application/json"" \
-H ""Authorization: Bearer $DS_TOKEN"" \
-d $ROWS
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1473481262,Hacker News Datasette write demo,
https://github.com/simonw/datasette/issues/1928#issuecomment-1335870883,https://api.github.com/repos/simonw/datasette/issues/1928,1335870883,IC_kwDOBm6k_c5Pn8Wj,9599,simonw,2022-12-02T21:19:10Z,2022-12-02T21:19:10Z,OWNER,"I created the `hacker_news_posts` table like this:
```bash
export ROWS=$(
jq -n --argjson rows ""$(cat simonwillison-net.json)"" \
'{ ""table"": ""hacker_news_posts"", ""rows"": $rows, ""pk"": ""id"", ""replace"": true }'
)
# Use curl to POST some JSON to a URL
curl -X POST \
https://simon.datasette.cloud/data/-/create \
-H ""Content-Type: application/json"" \
-H ""Authorization: Bearer $DS_TOKEN"" \
-d $ROWS
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1473481262,Hacker News Datasette write demo,
https://github.com/simonw/datasette/issues/1928#issuecomment-1335870879,https://api.github.com/repos/simonw/datasette/issues/1928,1335870879,IC_kwDOBm6k_c5Pn8Wf,9599,simonw,2022-12-02T21:18:25Z,2022-12-02T21:18:25Z,OWNER,"This is the SQL view for the atom feed:
```sql
CREATE VIEW hacker_news_posts_atom as select
id as atom_id,
title as atom_title,
url,
commentsUrl as atom_link,
dt || 'Z' as atom_updated,
'Submitter: ' || submitter || ' - ' || points || ' points, ' || numComments || ' comments' as atom_content
from
hacker_news_posts
order by
dt desc
limit
100;
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1473481262,Hacker News Datasette write demo,
https://github.com/simonw/datasette/issues/1928#issuecomment-1335870877,https://api.github.com/repos/simonw/datasette/issues/1928,1335870877,IC_kwDOBm6k_c5Pn8Wd,9599,simonw,2022-12-02T21:17:50Z,2022-12-02T21:17:50Z,OWNER,https://simon.datasette.cloud/data/hacker_news_posts_atom.atom is working now.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1473481262,Hacker News Datasette write demo,
https://github.com/simonw/datasette/issues/1636#issuecomment-1334759315,https://api.github.com/repos/simonw/datasette/issues/1636,1334759315,IC_kwDOBm6k_c5Pjs-T,9599,simonw,2022-12-02T04:46:32Z,2022-12-02T04:46:32Z,OWNER,"Thankfully all of the logic for this already lives in just one place:
https://github.com/simonw/datasette/blob/d7e5e3c9f98d194fdfb12f1ecc60ed5b3afbc464/datasette/default_permissions.py#L23-L59","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1138008042,"""permissions"" propery in metadata for configuring arbitrary permissions",
https://github.com/simonw/datasette/issues/1636#issuecomment-1334758766,https://api.github.com/repos/simonw/datasette/issues/1636,1334758766,IC_kwDOBm6k_c5Pjs1u,9599,simonw,2022-12-02T04:45:16Z,2022-12-02T04:45:16Z,OWNER,"Also, this is another thing which should live in `config.yml` rather than being crammed into `metadata.yml` - but I can fix that when I address:
- #493","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1138008042,"""permissions"" propery in metadata for configuring arbitrary permissions",
https://github.com/simonw/datasette/issues/1636#issuecomment-1334757597,https://api.github.com/repos/simonw/datasette/issues/1636,1334757597,IC_kwDOBm6k_c5Pjsjd,9599,simonw,2022-12-02T04:42:35Z,2022-12-02T04:42:35Z,OWNER,"Should I call this key `permissions` or something else?
Some options:
- `permissions`
- `perms` - shorter to type
- `allow` - I like the word, but might be confusing to change its meaning since we use it already","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1138008042,"""permissions"" propery in metadata for configuring arbitrary permissions",
https://github.com/simonw/datasette/issues/1636#issuecomment-1334673179,https://api.github.com/repos/simonw/datasette/issues/1636,1334673179,IC_kwDOBm6k_c5PjX8b,9599,simonw,2022-12-02T02:07:20Z,2022-12-02T04:27:07Z,OWNER,"So the new mechanism needs to extend that to handle all of the other permissions as well.
The simplest design I can think of is this (here illustrated using YAML):
```yaml
# instance-level permissions - give every logged in user the debug menu:
permissions:
debug-menu:
id: *
databases:
content:
# Allow bob to create-table in the content database
permissions:
create-table:
id: bob
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1138008042,"""permissions"" propery in metadata for configuring arbitrary permissions",
https://github.com/simonw/datasette/issues/1636#issuecomment-1334666806,https://api.github.com/repos/simonw/datasette/issues/1636,1334666806,IC_kwDOBm6k_c5PjWY2,9599,simonw,2022-12-02T01:58:40Z,2022-12-02T02:00:53Z,OWNER,"Current design:
```json
{
""databases"": {
""private"": {
""allow"": {
""id"": ""*""
}
}
}
}
```
This can be applied at the instance, database, table or query level within the nested JSON.
https://docs.datasette.io/en/stable/authentication.html#controlling-access-to-specific-databases
It's actually controlling the following permissions:
- `view-instance`
- `view-database`
- `view-table`
- `view-query`
There's also a special case for allowing SQL queries,at the instance and database level:
```json
{
""databases"": {
""mydatabase"": {
""allow_sql"": {
""id"": ""root""
}
}
}
}
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1138008042,"""permissions"" propery in metadata for configuring arbitrary permissions",
https://github.com/simonw/datasette/issues/1926#issuecomment-1334508062,https://api.github.com/repos/simonw/datasette/issues/1926,1334508062,IC_kwDOBm6k_c5Pivoe,9599,simonw,2022-12-01T22:06:12Z,2022-12-01T22:06:12Z,OWNER,Released: https://pypi.org/project/datasette/1.0a1/,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1471969984,Release notes for 1.0a1 (and release it),
https://github.com/simonw/datasette/issues/1924#issuecomment-1334165225,https://api.github.com/repos/simonw/datasette/issues/1924,1334165225,IC_kwDOBm6k_c5Phb7p,9599,simonw,2022-12-01T18:15:15Z,2022-12-01T18:15:15Z,OWNER,Updated docs at the bottom of this section: https://docs.datasette.io/en/latest/json_api.html#inserting-rows,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1470509936,Docs for replace:true and ignore:true options for insert API,
https://github.com/simonw/datasette/issues/1924#issuecomment-1333042785,https://api.github.com/repos/simonw/datasette/issues/1924,1333042785,IC_kwDOBm6k_c5PdJ5h,9599,simonw,2022-12-01T02:00:06Z,2022-12-01T02:00:06Z,OWNER,"Looks like I did implement these already:
https://github.com/simonw/datasette/blob/9a1536b52a07e32da5900652da1bd7894c58fa9f/tests/test_api_write.py#L270-L273
But forgot to document them!","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1470509936,Docs for replace:true and ignore:true options for insert API,
https://github.com/simonw/datasette/issues/1924#issuecomment-1333025076,https://api.github.com/repos/simonw/datasette/issues/1924,1333025076,IC_kwDOBm6k_c5PdFk0,9599,simonw,2022-12-01T01:37:09Z,2022-12-01T01:37:09Z,OWNER,"Related question: what happens if you attempt to insert rows without either of these settings and 1:10 of them is an integrity error due to an existing primary key?
Does the entire batch fail to be inserted? Should it?
This may be the point that I need to think hard about how to use transactions here.
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1470509936,Docs for replace:true and ignore:true options for insert API,
https://github.com/simonw/datasette/issues/1924#issuecomment-1333023273,https://api.github.com/repos/simonw/datasette/issues/1924,1333023273,IC_kwDOBm6k_c5PdFIp,9599,simonw,2022-12-01T01:34:48Z,2022-12-01T01:34:48Z,OWNER,"This will enable some very cool Datasette write API demos - for example, Git scrapers that insert-replace the most recent copy of the scraped data to a table somewhere (which can then produce an atom feed with https://datasette.io/plugins/datasette-atom)","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1470509936,Docs for replace:true and ignore:true options for insert API,
https://github.com/simonw/datasette/issues/1922#issuecomment-1332903011,https://api.github.com/repos/simonw/datasette/issues/1922,1332903011,IC_kwDOBm6k_c5Pcnxj,9599,simonw,2022-11-30T23:45:29Z,2022-11-30T23:45:29Z,OWNER,"That worked for the preflight request - got this now:
So it looks like error responses (in this case for permission denied) are missing CORS headers.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1469973742,Make sure CORS works for write APIs,
https://github.com/simonw/datasette/issues/1922#issuecomment-1332855687,https://api.github.com/repos/simonw/datasette/issues/1922,1332855687,IC_kwDOBm6k_c5PccOH,9599,simonw,2022-11-30T23:09:31Z,2022-11-30T23:09:31Z,OWNER,"Still getting a CORS header.
I tried it in Chrome, which outputs helpful messages to the console:
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1469973742,Make sure CORS works for write APIs,
https://github.com/simonw/datasette/issues/1923#issuecomment-1332851215,https://api.github.com/repos/simonw/datasette/issues/1923,1332851215,IC_kwDOBm6k_c5PcbIP,9599,simonw,2022-11-30T23:04:56Z,2022-11-30T23:04:56Z,OWNER,That fixed it.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1470320227,latest.datasette.io Cloud Run deploys failing,
https://github.com/simonw/datasette/issues/1923#issuecomment-1332842435,https://api.github.com/repos/simonw/datasette/issues/1923,1332842435,IC_kwDOBm6k_c5PcY_D,9599,simonw,2022-11-30T22:58:33Z,2022-11-30T22:58:33Z,OWNER,https://stackoverflow.com/questions/74490465/github-actions-failing-for-setup-gcloud/74562740#74562740 suggests trying Python 3.9.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1470320227,latest.datasette.io Cloud Run deploys failing,
https://github.com/simonw/datasette/issues/1923#issuecomment-1332835664,https://api.github.com/repos/simonw/datasette/issues/1923,1332835664,IC_kwDOBm6k_c5PcXVQ,9599,simonw,2022-11-30T22:50:10Z,2022-11-30T22:51:25Z,OWNER,https://stackoverflow.com/questions/74490465/github-actions-failing-for-setup-gcloud/74562526#74562526 suggests setting `version: '318.0.0'` of `google-github-actions/setup-gcloud`.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1470320227,latest.datasette.io Cloud Run deploys failing,
https://github.com/simonw/datasette/issues/1922#issuecomment-1332698636,https://api.github.com/repos/simonw/datasette/issues/1922,1332698636,IC_kwDOBm6k_c5Pb14M,9599,simonw,2022-11-30T20:25:50Z,2022-11-30T20:25:50Z,OWNER,"I just shipped this:
Access-Control-Allow-Methods: GET, POST, HEAD, OPTIONS
I'll try this out on `latest.datasette.io` - but I need to research more to check if this is a safe thing to do or not.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1469973742,Make sure CORS works for write APIs,
https://github.com/simonw/datasette/issues/1922#issuecomment-1332689547,https://api.github.com/repos/simonw/datasette/issues/1922,1332689547,IC_kwDOBm6k_c5PbzqL,9599,simonw,2022-11-30T20:16:21Z,2022-11-30T20:16:46Z,OWNER,"That notebook:
```javascript
viewof token = Inputs.text({
label: ""Your API token""
})
```
```javascript
viewof createResponse = Inputs.button(""Create table"", {
value: null,
reduce: async () => {
const response = await fetch(
""https://latest.datasette.io/ephemeral/-/create"",
{
method: ""POST"",
mode: ""cors"",
headers: {
Authorization: `Bearer {$token}`,
""Content-Type"": ""application/json""
},
body: JSON.stringify({
table: ""my_new_table"",
row: {
task: ""Demonstrate a JSON creation from another domain""
}
})
}
);
return await response.json();
}
})
```
Based on this tip: https://talk.observablehq.com/t/best-pattern-for-click-here-to-submit-your-results-to-an-api-backend/7353/3","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1469973742,Make sure CORS works for write APIs,
https://github.com/simonw/datasette/issues/1922#issuecomment-1332688245,https://api.github.com/repos/simonw/datasette/issues/1922,1332688245,IC_kwDOBm6k_c5PbzV1,9599,simonw,2022-11-30T20:15:08Z,2022-11-30T20:15:08Z,OWNER,"Still getting a CORS error:
My hunch is this is because I'm not sending `Access-Control-Allow-Methods: GET,HEAD,POST`.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1469973742,Make sure CORS works for write APIs,
https://github.com/simonw/datasette/issues/1922#issuecomment-1332585861,https://api.github.com/repos/simonw/datasette/issues/1922,1332585861,IC_kwDOBm6k_c5PbaWF,9599,simonw,2022-11-30T18:43:46Z,2022-11-30T18:43:46Z,OWNER,"Here's what Django Rest Framework does: https://github.com/encode/django-rest-framework/blob/1ae812ea209392ad76cc5d2f35f9f7fb337f63e4/rest_framework/views.py#L514-L521
```python
def options(self, request, *args, **kwargs):
""""""
Handler method for HTTP 'OPTIONS' request.
""""""
if self.metadata_class is None:
return self.http_method_not_allowed(request, *args, **kwargs)
data = self.metadata_class().determine_metadata(request, self)
return Response(data, status=status.HTTP_200_OK)
```
That default `determine_metadata` method looks like this: https://github.com/encode/django-rest-framework/blob/1ae812ea209392ad76cc5d2f35f9f7fb337f63e4/rest_framework/metadata.py#L61-L71
```python
def determine_metadata(self, request, view):
metadata = OrderedDict()
metadata['name'] = view.get_view_name()
metadata['description'] = view.get_view_description()
metadata['renders'] = [renderer.media_type for renderer in view.renderer_classes]
metadata['parses'] = [parser.media_type for parser in view.parser_classes]
if hasattr(view, 'get_serializer'):
actions = self.determine_actions(request, view)
if actions:
metadata['actions'] = actions
return metadata
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1469973742,Make sure CORS works for write APIs,
https://github.com/simonw/datasette/issues/1922#issuecomment-1332580395,https://api.github.com/repos/simonw/datasette/issues/1922,1332580395,IC_kwDOBm6k_c5PbZAr,9599,simonw,2022-11-30T18:38:22Z,2022-11-30T18:38:22Z,OWNER,"> [@simon](https://fedi.simonwillison.net/@simon) IMO, it should always be a 2XX series response, typically with no content & an extra `Allow` header with a list of HTTP verbs it responds to.
https://mastodon.social/@daniellindsley/109434186252099323","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1469973742,Make sure CORS works for write APIs,
https://github.com/simonw/datasette/issues/1922#issuecomment-1332572453,https://api.github.com/repos/simonw/datasette/issues/1922,1332572453,IC_kwDOBm6k_c5PbXEl,9599,simonw,2022-11-30T18:30:38Z,2022-11-30T18:30:54Z,OWNER,Started a conversation about how OPTIONS should work on Mastodon: https://fedi.simonwillison.net/@simon/109434148676475291,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1469973742,Make sure CORS works for write APIs,
https://github.com/simonw/datasette/issues/1922#issuecomment-1332561813,https://api.github.com/repos/simonw/datasette/issues/1922,1332561813,IC_kwDOBm6k_c5PbUeV,9599,simonw,2022-11-30T18:20:05Z,2022-11-30T18:20:05Z,OWNER,"Weird, GitHub reply with a 404!
```
~ % curl -X OPTIONS https://github.com/ -i
HTTP/2 404
server: GitHub.com
date: Wed, 30 Nov 2022 18:19:39 GMT
content-type: text/html; charset=utf-8
content-length: 0
strict-transport-security: max-age=31536000; includeSubdomains; preload
x-frame-options: deny
x-content-type-options: nosniff
x-xss-protection: 0
referrer-policy: origin-when-cross-origin, strict-origin-when-cross-origin
content-security-policy: default-src 'none'; base-uri 'self'; block-all-mixed-content; child-src github.com/assets-cdn/worker/ gist.github.com/assets-cdn/worker/; connect-src 'self' uploads.github.com objects-origin.githubusercontent.com www.githubstatus.com collector.github.com raw.githubusercontent.com api.github.com github-cloud.s3.amazonaws.com github-production-repository-file-5c1aeb.s3.amazonaws.com github-production-upload-manifest-file-7fdce7.s3.amazonaws.com github-production-user-asset-6210df.s3.amazonaws.com cdn.optimizely.com logx.optimizely.com/v1/events; font-src github.githubassets.com; form-action 'self' github.com gist.github.com objects-origin.githubusercontent.com; frame-ancestors 'none'; frame-src viewscreen.githubusercontent.com notebooks.githubusercontent.com; img-src 'self' data: github.githubassets.com media.githubusercontent.com camo.githubusercontent.com identicons.github.com avatars.githubusercontent.com github-cloud.s3.amazonaws.com objects.githubusercontent.com objects-origin.githubusercontent.com secured-user-images.githubusercontent.com/ opengraph.githubassets.com github-production-user-asset-6210df.s3.amazonaws.com customer-stories-feed.github.com spotlights-feed.github.com; manifest-src 'self'; media-src github.com user-images.githubusercontent.com/ secured-user-images.githubusercontent.com/; script-src github.githubassets.com; style-src 'unsafe-inline' github.githubassets.com; worker-src github.com/assets-cdn/worker/ gist.github.com/assets-cdn/worker/
vary: Accept-Encoding, Accept, X-Requested-With
x-github-request-id: DD6B:5ACA:102E8A6:1164A99:63879EBB
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1469973742,Make sure CORS works for write APIs,
https://github.com/simonw/datasette/issues/1922#issuecomment-1332561059,https://api.github.com/repos/simonw/datasette/issues/1922,1332561059,IC_kwDOBm6k_c5PbUSj,9599,simonw,2022-11-30T18:19:20Z,2022-11-30T18:19:20Z,OWNER,"Two test failures:
```
____________________________ test_homepage_options _____________________________
[gw0] linux -- Python 3.11.0 /opt/hostedtoolcache/Python/3.11.0/x64/bin/python
app_client =
def test_homepage_options(app_client):
response = app_client.get(""/"", method=""OPTIONS"")
> assert response.status == 405
E assert 200 == 405
E + where 200 = .status
/home/runner/work/datasette/datasette/tests/test_html.py:58: AssertionError
______________________ test_client_methods[options-/-405] ______________________
[gw1] linux -- Python 3.11.0 /opt/hostedtoolcache/Python/3.11.0/x64/bin/python
datasette =
method = 'options', path = '/', expected_status = 405
@pytest.mark.asyncio
@pytest.mark.parametrize(
""method,path,expected_status"",
[
(""get"", ""/"", 200),
(""options"", ""/"", 405),
(""head"", ""/"", 200),
(""put"", ""/"", 405),
(""patch"", ""/"", 405),
(""delete"", ""/"", 405),
],
)
async def test_client_methods(datasette, method, path, expected_status):
client_method = getattr(datasette.client, method)
response = await client_method(path)
assert isinstance(response, httpx.Response)
> assert response.status_code == expected_status
E assert 200 == 405
E + where 200 = .status_code
/home/runner/work/datasette/datasette/tests/test_internals_datasette_client.py:29: AssertionError
=============================== warnings summary ===============================
tests/test_cli.py::test_inspect_cli_writes_to_file
tests/test_cli.py::test_inspect_cli
/home/runner/work/datasette/datasette/datasette/cli.py:163: DeprecationWarning: There is no current event loop
loop = asyncio.get_event_loop()
tests/test_cli_serve_get.py: 2 warnings
tests/test_cli.py: 12 warnings
tests/test_crossdb.py: 1 warning
/home/runner/work/datasette/datasette/datasette/cli.py:591: DeprecationWarning: There is no current event loop
asyncio.get_event_loop().run_until_complete(ds.invoke_startup())
tests/test_cli_serve_get.py: 2 warnings
tests/test_cli.py: 12 warnings
tests/test_crossdb.py: 1 warning
/home/runner/work/datasette/datasette/datasette/cli.py:594: DeprecationWarning: There is no current event loop
asyncio.get_event_loop().run_until_complete(check_databases(ds))
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
FAILED tests/test_html.py::test_homepage_options - assert 200 == 405
+ where 200 = .status
FAILED tests/test_internals_datasette_client.py::test_client_methods[options-/-405] - assert 200 == 405
+ where 200 = .status_code
====== 2 failed, 1195 passed, 1 skipped, 32 warnings in 191.06s (0:03:11) ======
Error: Process completed with exit code 1.
```
On reading https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/OPTIONS I feel like I should be a bit more thoughtful about how I treat OPTIONS - maybe it should work for every URL on the site, but return a `204 No Content` header?
Comparing a few different sites:
```
~ % curl -X OPTIONS https://www.google.com/ -i
HTTP/2 405
allow: GET, HEAD
date: Wed, 30 Nov 2022 18:18:15 GMT
content-type: text/html; charset=UTF-8
server: gws
content-length: 1592
x-xss-protection: 0
x-frame-options: SAMEORIGIN
alt-svc: h3="":443""; ma=2592000,h3-29="":443""; ma=2592000,h3-Q050="":443""; ma=2592000,h3-Q046="":443""; ma=2592000,h3-Q043="":443""; ma=2592000,quic="":443""; ma=2592000; v=""46,43""
Error 405 (Method Not Allowed)!!1
405. That’s an error.
The request method OPTIONS is inappropriate for the URL /. That’s all we know.
~ % curl -X OPTIONS https://www.mozilla.org/ -i
HTTP/2 405
content-type: text/html; charset=utf-8
content-length: 0
server: meinheld/1.0.2
date: Wed, 30 Nov 2022 18:18:38 GMT
allow: GET, HEAD
x-frame-options: DENY
content-security-policy: child-src 'self' *.mozilla.net *.mozilla.org *.mozilla.com www.googletagmanager.com www.google-analytics.com www.youtube-nocookie.com trackertest.org www.surveygizmo.com accounts.firefox.com accounts.firefox.com.cn www.youtube.com; connect-src 'self' *.mozilla.net *.mozilla.org *.mozilla.com www.googletagmanager.com www.google-analytics.com region1.google-analytics.com logs.convertexperiments.com 1003350.metrics.convertexperiments.com 1003343.metrics.convertexperiments.com sentry.prod.mozaws.net o1069899.sentry.io o1069899.ingest.sentry.io https://accounts.firefox.com/ stage.cjms.nonprod.cloudops.mozgcp.net cjms.services.mozilla.com; frame-src 'self' *.mozilla.net *.mozilla.org *.mozilla.com www.googletagmanager.com www.google-analytics.com www.youtube-nocookie.com trackertest.org www.surveygizmo.com accounts.firefox.com accounts.firefox.com.cn www.youtube.com; script-src 'self' *.mozilla.net *.mozilla.org *.mozilla.com 'unsafe-inline' 'unsafe-eval' www.googletagmanager.com www.google-analytics.com tagmanager.google.com www.youtube.com s.ytimg.com cdn-3.convertexperiments.com app.convert.com data.track.convertexperiments.com 1003350.track.convertexperiments.com 1003343.track.convertexperiments.com; img-src 'self' *.mozilla.net *.mozilla.org *.mozilla.com data: mozilla.org www.googletagmanager.com www.google-analytics.com adservice.google.com adservice.google.de adservice.google.dk creativecommons.org cdn-3.convertexperiments.com logs.convertexperiments.com images.ctfassets.net ad.doubleclick.net; style-src 'self' *.mozilla.net *.mozilla.org *.mozilla.com 'unsafe-inline' app.convert.com; default-src 'self' *.mozilla.net *.mozilla.org *.mozilla.com; font-src 'self'
cache-control: max-age=600
expires: Wed, 30 Nov 2022 18:28:38 GMT
x-backend-server: bedrock-prod-web-b95bc569d-grd25.iowa-a
strict-transport-security: max-age=31536000
x-content-type-options: nosniff
x-xss-protection: 1; mode=block
referrer-policy: strict-origin-when-cross-origin
via: 1.1 google, 1.1 6c90b631453c435bd0022caa657b67e8.cloudfront.net (CloudFront)
x-cache: Error from cloudfront
x-amz-cf-pop: SFO5-P2
x-amz-cf-id: A6-9mLztaE2tz840CbV9bXYiBMZRKEamDj6jGGEl1U7sg8egWfsDqg==
~ % curl -X OPTIONS https://example.com -i
HTTP/2 200
allow: OPTIONS, GET, HEAD, POST
cache-control: max-age=604800
content-type: text/html; charset=UTF-8
date: Wed, 30 Nov 2022 18:18:59 GMT
expires: Wed, 07 Dec 2022 18:18:59 GMT
server: EOS (vny/0451)
content-length: 0
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1469973742,Make sure CORS works for write APIs,
https://github.com/simonw/datasette/issues/1922#issuecomment-1332504654,https://api.github.com/repos/simonw/datasette/issues/1922,1332504654,IC_kwDOBm6k_c5PbGhO,9599,simonw,2022-11-30T17:27:39Z,2022-11-30T17:27:39Z,OWNER,I'll test this once it's deployed to https://latest.datasette.io/,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1469973742,Make sure CORS works for write APIs,
https://github.com/simonw/datasette/issues/1922#issuecomment-1332493004,https://api.github.com/repos/simonw/datasette/issues/1922,1332493004,IC_kwDOBm6k_c5PbDrM,9599,simonw,2022-11-30T17:18:10Z,2022-11-30T17:18:10Z,OWNER,"Here's why:
https://github.com/simonw/datasette/blob/4ddd77e51254bda3bac990ea662bac2e6b29c5e0/datasette/views/base.py#L71-L79
That's code in `BaseView` - but it turns out the code that adds CORS headers is in the `DataView` subclass of that (which the various write API endpoints do not use).
https://github.com/simonw/datasette/blob/4ddd77e51254bda3bac990ea662bac2e6b29c5e0/datasette/views/base.py#L158-L162","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1469973742,Make sure CORS works for write APIs,
https://github.com/simonw/datasette/issues/1922#issuecomment-1332492092,https://api.github.com/repos/simonw/datasette/issues/1922,1332492092,IC_kwDOBm6k_c5PbDc8,9599,simonw,2022-11-30T17:17:21Z,2022-11-30T17:17:21Z,OWNER,I tried running `fetch()` with a POST from a separate domain and got a browser error because it did a GET against the `/db/-/create` endpoint and the 405 method not supported response did not include the CORS headers.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1469973742,Make sure CORS works for write APIs,
https://github.com/simonw/datasette/issues/1605#issuecomment-1331694246,https://api.github.com/repos/simonw/datasette/issues/1605,1331694246,IC_kwDOBm6k_c5PYAqm,9599,simonw,2022-11-30T06:18:41Z,2022-11-30T06:18:41Z,OWNER,"Those sounds to me like they should be promoted to documented, supported internals.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1108671952,Scripted exports,
https://github.com/simonw/datasette/issues/1918#issuecomment-1331658629,https://api.github.com/repos/simonw/datasette/issues/1918,1331658629,IC_kwDOBm6k_c5PX3-F,9599,simonw,2022-11-30T05:21:51Z,2022-11-30T05:21:51Z,OWNER,"Much better:
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1469044738,API explorer should list mutable databases first,
https://github.com/simonw/datasette/issues/1919#issuecomment-1331657404,https://api.github.com/repos/simonw/datasette/issues/1919,1331657404,IC_kwDOBm6k_c5PX3q8,9599,simonw,2022-11-30T05:19:43Z,2022-11-30T05:19:43Z,OWNER,"This is the test: https://github.com/simonw/datasette/blob/8404b21556d133c89eda4bd1bf5335ed9a0785d6/tests/test_api_write.py#L342-L401
I'm suspicious that there's a timing error of some sort but I can't think what it might be.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1469062686,Intermittent `test_delete_row` test failure ,
https://github.com/simonw/datasette/issues/1916#issuecomment-1331651721,https://api.github.com/repos/simonw/datasette/issues/1916,1331651721,IC_kwDOBm6k_c5PX2SJ,9599,simonw,2022-11-30T05:10:27Z,2022-11-30T05:10:27Z,OWNER,"They should return 405 method not allowed with an `{""ok"":false, ""error"": ""Method not allowed""}` body.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1469015001,GET requests against POST endpoints should not 500 error,
https://github.com/simonw/datasette/issues/1917#issuecomment-1331644751,https://api.github.com/repos/simonw/datasette/issues/1917,1331644751,IC_kwDOBm6k_c5PX0lP,9599,simonw,2022-11-30T04:59:22Z,2022-11-30T04:59:22Z,OWNER,"Yeah it looks like I introduced this bug here:
https://github.com/simonw/datasette/commit/fb7e70d5e72a951efe4b29ad999d8915c032d021","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1469043836,Don't allow writable API to edit the `_memory` database,
https://github.com/simonw/datasette/issues/1917#issuecomment-1331644078,https://api.github.com/repos/simonw/datasette/issues/1917,1331644078,IC_kwDOBm6k_c5PX0au,9599,simonw,2022-11-30T04:58:06Z,2022-11-30T04:58:06Z,OWNER,"The problem might actually be here:
https://github.com/simonw/datasette/blob/9f5321ff1eca58c469a45cc406d7eb5ad05accbd/datasette/app.py#L280-L281
`is_mutable` defaults to `True`, so this line should probably be:
```python
self.add_database(Database(self, is_mutable=False, is_memory=True), name=""_memory"")
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1469043836,Don't allow writable API to edit the `_memory` database,
https://github.com/simonw/datasette/issues/1915#issuecomment-1331479606,https://api.github.com/repos/simonw/datasette/issues/1915,1331479606,IC_kwDOBm6k_c5PXMQ2,9599,simonw,2022-11-30T00:09:06Z,2022-11-30T00:09:06Z,OWNER,One last feature: I want to show an indication on the table page that the table has X seconds left to live.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1468709531,Interactive demo of Datasette 1.0 write APIs,
https://github.com/simonw/datasette/issues/1915#issuecomment-1331479328,https://api.github.com/repos/simonw/datasette/issues/1915,1331479328,IC_kwDOBm6k_c5PXMMg,9599,simonw,2022-11-30T00:08:41Z,2022-11-30T00:08:41Z,OWNER,Five minute has now passed and https://latest.datasette.io/ephemeral/new_table is gone.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1468709531,Interactive demo of Datasette 1.0 write APIs,
https://github.com/simonw/datasette/issues/1915#issuecomment-1331476246,https://api.github.com/repos/simonw/datasette/issues/1915,1331476246,IC_kwDOBm6k_c5PXLcW,9599,simonw,2022-11-30T00:04:35Z,2022-11-30T00:08:24Z,OWNER,"The new https://github.com/simonw/datasette-ephemeral-tables plugin is live now: https://latest.datasette.io/ephemeral - you have to navigate through https://latest.datasette.io/login-as-root first
It work! I created a table using https://latest.datasette.io/-/api#path=%2Fephemeral%2F-%2Fcreate&json=%7B%0A++%22table%22%3A+%22new_table%22%2C%0A++%22columns%22%3A+%5B%0A++++%7B%0A++++++%22name%22%3A+%22id%22%2C%0A++++++%22type%22%3A+%22integer%22%0A++++%7D%2C%0A++++%7B%0A++++++%22name%22%3A+%22name%22%2C%0A++++++%22type%22%3A+%22text%22%0A++++%7D%0A++%5D%2C%0A++%22pk%22%3A+%22id%22%0A%7D&method=POST
The table should vanish in a few minutes.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1468709531,Interactive demo of Datasette 1.0 write APIs,
https://github.com/simonw/datasette/issues/1915#issuecomment-1331478611,https://api.github.com/repos/simonw/datasette/issues/1915,1331478611,IC_kwDOBm6k_c5PXMBT,9599,simonw,2022-11-30T00:07:37Z,2022-11-30T00:07:37Z,OWNER,"Then I created an API token at https://latest.datasette.io/-/create-token and ran this:
```
curl -XPOST 'https://latest.datasette.io/ephemeral/new_table/-/insert' \
-H 'Authorization: Bearer xxx' \
-H 'Content-Type: application/json' \
-d '{""row"": {""name"": ""NAME""}}'
```
And it inserted a row into https://latest.datasette.io/ephemeral/new_table","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1468709531,Interactive demo of Datasette 1.0 write APIs,
https://github.com/simonw/datasette/issues/1915#issuecomment-1331432223,https://api.github.com/repos/simonw/datasette/issues/1915,1331432223,IC_kwDOBm6k_c5PXAsf,9599,simonw,2022-11-29T23:06:17Z,2022-11-29T23:06:17Z,OWNER,To (slightly) discourage abuse I'm going to make the demo database only visible to the root user - so people can't create tables with rude names and have them show to the public on https://latest.datasette.io/,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1468709531,Interactive demo of Datasette 1.0 write APIs,
https://github.com/simonw/datasette/issues/1915#issuecomment-1331331082,https://api.github.com/repos/simonw/datasette/issues/1915,1331331082,IC_kwDOBm6k_c5PWoAK,9599,simonw,2022-11-29T21:24:59Z,2022-11-29T21:34:53Z,OWNER,Maybe a plugin called `datasette-temporary-tables` or `datasette-demo-tables` or `datasette-demo-database`.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1468709531,Interactive demo of Datasette 1.0 write APIs,
https://github.com/simonw/datasette/issues/1850#issuecomment-1331238841,https://api.github.com/repos/simonw/datasette/issues/1850,1331238841,IC_kwDOBm6k_c5PWRe5,9599,simonw,2022-11-29T20:11:20Z,2022-11-29T20:11:20Z,OWNER,"Released this in Datasette 1.0a0:
- #1913","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1421529723,Write API in Datasette core,
https://github.com/simonw/datasette/issues/1913#issuecomment-1331238029,https://api.github.com/repos/simonw/datasette/issues/1913,1331238029,IC_kwDOBm6k_c5PWRSN,9599,simonw,2022-11-29T20:10:35Z,2022-11-29T20:10:35Z,OWNER,"Released:
- https://pypi.org/project/datasette/1.0a0/
- https://docs.datasette.io/en/latest/changelog.html#a0-2022-11-29","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1468603401,Release Datasette 1.0a0,
https://github.com/simonw/datasette/issues/1913#issuecomment-1331226346,https://api.github.com/repos/simonw/datasette/issues/1913,1331226346,IC_kwDOBm6k_c5PWObq,9599,simonw,2022-11-29T20:00:16Z,2022-11-29T20:00:36Z,OWNER,"Looks like a fix is coming: https://github.com/pypa/twine/issues/940#issuecomment-1331225509
> > Note that `must_decode` was defined in `pkg_info/_compat.py`, and was thus never an API: before 1.9.0, it was only imported and used in `pkginfo/distribution.py'.
>
> Nevertheless, I will push out a 1.9.1 release of `pkginfo` which restores a deprecated compatibility alias.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1468603401,Release Datasette 1.0a0,
https://github.com/simonw/datasette/issues/1913#issuecomment-1331225277,https://api.github.com/repos/simonw/datasette/issues/1913,1331225277,IC_kwDOBm6k_c5PWOK9,9599,simonw,2022-11-29T19:59:14Z,2022-11-29T19:59:34Z,OWNER,I deleted the tag and tried creating a new release. Now running here: https://github.com/simonw/datasette/actions/runs/3577554546,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1468603401,Release Datasette 1.0a0,
https://github.com/simonw/datasette/issues/1913#issuecomment-1331216652,https://api.github.com/repos/simonw/datasette/issues/1913,1331216652,IC_kwDOBm6k_c5PWMEM,9599,simonw,2022-11-29T19:54:22Z,2022-11-29T19:54:22Z,OWNER,Filed a bug report here: https://bugs.launchpad.net/pkginfo/+bug/1998249,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1468603401,Release Datasette 1.0a0,
https://github.com/simonw/datasette/issues/1913#issuecomment-1331208206,https://api.github.com/repos/simonw/datasette/issues/1913,1331208206,IC_kwDOBm6k_c5PWKAO,9599,simonw,2022-11-29T19:51:31Z,2022-11-29T19:51:31Z,OWNER,https://pypi.org/project/pkginfo/#history - 1.9.0 came out 39 minutes ago!,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1468603401,Release Datasette 1.0a0,
https://github.com/simonw/datasette/issues/1913#issuecomment-1331207334,https://api.github.com/repos/simonw/datasette/issues/1913,1331207334,IC_kwDOBm6k_c5PWJym,9599,simonw,2022-11-29T19:50:37Z,2022-11-29T19:50:37Z,OWNER,"https://pypi.org/project/setuptools/65.6.3/ came out most recently - 23rd November (wheel and twine are older).
No search results at all for that error message. This is very weird, I would have expected it to have been reported by now.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1468603401,Release Datasette 1.0a0,
https://github.com/simonw/datasette/issues/1913#issuecomment-1331205613,https://api.github.com/repos/simonw/datasette/issues/1913,1331205613,IC_kwDOBm6k_c5PWJXt,9599,simonw,2022-11-29T19:48:52Z,2022-11-29T19:48:52Z,OWNER,https://github.com/simonw/datasette/blob/07aad511769da9242260c850e8d975cbd8c29552/.github/workflows/publish.yml#L52-L61,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1468603401,Release Datasette 1.0a0,
https://github.com/simonw/datasette/issues/1913#issuecomment-1331204360,https://api.github.com/repos/simonw/datasette/issues/1913,1331204360,IC_kwDOBm6k_c5PWJEI,9599,simonw,2022-11-29T19:47:40Z,2022-11-29T19:47:40Z,OWNER,"... but the last step of the deploy failed, when it was meant to push to PyPI!
```
Uploading distributions to https://upload.pypi.org/legacy/
Traceback (most recent call last):
File ""/opt/hostedtoolcache/Python/3.11.0/x64/bin/twine"", line 8, in
sys.exit(main())
^^^^^^
File ""/opt/hostedtoolcache/Python/3.11.0/x64/lib/python3.11/site-packages/twine/__main__.py"", line 33, in main
error = cli.dispatch(sys.argv[1:])
^^^^^^^^^^^^^^^^^^^^^^^^^^
File ""/opt/hostedtoolcache/Python/3.11.0/x64/lib/python3.11/site-packages/twine/cli.py"", line 123, in dispatch
return main(args.args)
^^^^^^^^^^^^^^^
File ""/opt/hostedtoolcache/Python/3.11.0/x64/lib/python3.11/site-packages/twine/commands/upload.py"", line 198, in main
return upload(upload_settings, parsed_args.dists)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ""/opt/hostedtoolcache/Python/3.11.0/x64/lib/python3.11/site-packages/twine/commands/upload.py"", line 123, in upload
packages_to_upload = [
^
File ""/opt/hostedtoolcache/Python/3.11.0/x64/lib/python3.11/site-packages/twine/commands/upload.py"", line 124, in
_make_package(filename, signatures, upload_settings) for filename in uploads
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ""/opt/hostedtoolcache/Python/3.11.0/x64/lib/python3.11/site-packages/twine/commands/upload.py"", line 77, in _make_package
package = package_file.PackageFile.from_filename(filename, upload_settings.comment)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ""/opt/hostedtoolcache/Python/3.11.0/x64/lib/python3.11/site-packages/twine/package.py"", line 96, in from_filename
meta = DIST_TYPES[dtype](filename)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ""/opt/hostedtoolcache/Python/3.11.0/x64/lib/python3.11/site-packages/twine/wheel.py"", line 42, in __init__
self.extractMetadata()
File ""/opt/hostedtoolcache/Python/3.11.0/x64/lib/python3.11/site-packages/pkginfo/distribution.py"", line 121, in extractMetadata
self.parse(data)
File ""/opt/hostedtoolcache/Python/3.11.0/x64/lib/python3.11/site-packages/twine/wheel.py"", line 89, in parse
fp = io.StringIO(distribution.must_decode(data))
^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: module 'pkginfo.distribution' has no attribute 'must_decode'. Did you mean: '_must_decode'?
Error: Process completed with exit code 1.
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1468603401,Release Datasette 1.0a0,
https://github.com/simonw/datasette/issues/1913#issuecomment-1331203997,https://api.github.com/repos/simonw/datasette/issues/1913,1331203997,IC_kwDOBm6k_c5PWI-d,9599,simonw,2022-11-29T19:47:13Z,2022-11-29T19:47:13Z,OWNER,"Weird, retrying the tests DID get them to pass. https://github.com/simonw/datasette/actions/runs/3577355358/jobs/6016518244","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1468603401,Release Datasette 1.0a0,
https://github.com/simonw/datasette/issues/1877#issuecomment-1331201207,https://api.github.com/repos/simonw/datasette/issues/1877,1331201207,IC_kwDOBm6k_c5PWIS3,9599,simonw,2022-11-29T19:44:07Z,2022-11-29T19:44:07Z,OWNER,"I fixed the duplicate logic issue here: https://github.com/simonw/datasette/commit/ee64130fa8a5ff4a24791916c696e10cf2375102
- #1896
Decided not to address `views/table.py`.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1432012302,Refactor and tidy up final write API code,
https://github.com/simonw/datasette/pull/1912#issuecomment-1331196531,https://api.github.com/repos/simonw/datasette/issues/1912,1331196531,IC_kwDOBm6k_c5PWHJz,9599,simonw,2022-11-29T19:39:10Z,2022-11-29T19:39:10Z,OWNER,"Annoyingly it looks like I can't rebase this one, and I don't want to squash-merge and lose the commits, so I'm going to do a regular merge instead.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1468592292,Merge 1.0-dev (with initial write API) back into main,
https://github.com/simonw/datasette/issues/1891#issuecomment-1331181922,https://api.github.com/repos/simonw/datasette/issues/1891,1331181922,IC_kwDOBm6k_c5PWDli,9599,simonw,2022-11-29T19:23:41Z,2022-11-29T19:23:41Z,OWNER,https://github.com/simonw/datasette/blob/4d49a5a39739476e1ada43f70a0029abcef07977/docs/changelog.rst#10a0-2022-11-29,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1450303205,1.0a0 release notes,
https://github.com/simonw/datasette/issues/1891#issuecomment-1331143292,https://api.github.com/repos/simonw/datasette/issues/1891,1331143292,IC_kwDOBm6k_c5PV6J8,9599,simonw,2022-11-29T18:57:40Z,2022-11-29T18:57:40Z,OWNER,I'm going to keep these short - they'll mostly be links to the documentation for the new features.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1450303205,1.0a0 release notes,
https://github.com/simonw/datasette/issues/1891#issuecomment-1331140747,https://api.github.com/repos/simonw/datasette/issues/1891,1331140747,IC_kwDOBm6k_c5PV5iL,9599,simonw,2022-11-29T18:55:42Z,2022-11-29T18:55:42Z,OWNER,"All features for the alpha are complete now. Release notes should be based on these commits:
https://github.com/simonw/datasette/compare/0.63.2...6bda2257868a2cbd70b84b7a86a5bcb47dcc4874","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1450303205,1.0a0 release notes,
https://github.com/simonw/datasette/issues/1911#issuecomment-1331135709,https://api.github.com/repos/simonw/datasette/issues/1911,1331135709,IC_kwDOBm6k_c5PV4Td,9599,simonw,2022-11-29T18:50:58Z,2022-11-29T18:50:58Z,OWNER,Updated docs: https://docs.datasette.io/en/1.0-dev/json_api.html#creating-a-table,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1468519699,`/db/-/create` should support creating tables with compound primary keys,
https://github.com/simonw/datasette/issues/1911#issuecomment-1331120055,https://api.github.com/repos/simonw/datasette/issues/1911,1331120055,IC_kwDOBm6k_c5PV0e3,9599,simonw,2022-11-29T18:36:01Z,2022-11-29T18:36:01Z,OWNER,"Current API design:
```
POST //-/create
```
```json
{
""table"": ""name_of_new_table"",
""columns"": [
{
""name"": ""id"",
""type"": ""integer""
},
{
""name"": ""title"",
""type"": ""text""
}
],
""pk"": ""id""
}
```
I'm going to add a new `""pks""` key which is a list, and can be used in place of `""pk""`.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1468519699,`/db/-/create` should support creating tables with compound primary keys,
https://github.com/simonw/datasette/issues/1863#issuecomment-1331089156,https://api.github.com/repos/simonw/datasette/issues/1863,1331089156,IC_kwDOBm6k_c5PVs8E,9599,simonw,2022-11-29T18:08:53Z,2022-11-29T18:08:53Z,OWNER,"I do think this needs type checking - I just tried and you really can send a string to an integer column and have it work, which feels bad.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1425029242,Update a single record in an existing table,
https://github.com/simonw/datasette/issues/1863#issuecomment-1330974099,https://api.github.com/repos/simonw/datasette/issues/1863,1330974099,IC_kwDOBm6k_c5PVQ2T,9599,simonw,2022-11-29T17:03:00Z,2022-11-29T17:11:05Z,OWNER,I've decided that I won't do that validation for the first version of this - I'm going to teach `dclient` to send the correct types instead: https://github.com/simonw/dclient/issues/6#issuecomment-1330963953,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1425029242,Update a single record in an existing table,
https://github.com/simonw/datasette/issues/1909#issuecomment-1329624931,https://api.github.com/repos/simonw/datasette/issues/1909,1329624931,IC_kwDOBm6k_c5PQHdj,9599,simonw,2022-11-28T19:19:26Z,2022-11-28T19:19:26Z,OWNER,The list of states here is a good example of somewhere this might be useful: https://congress-legislators.datasettes.com/legislators/legislator_terms?_facet=state&_facet_size=max,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1466952626,Option to sort facets alphabetically,
https://github.com/simonw/datasette/issues/1605#issuecomment-1328169472,https://api.github.com/repos/simonw/datasette/issues/1605,1328169472,IC_kwDOBm6k_c5PKkIA,9599,simonw,2022-11-27T04:32:14Z,2022-11-27T04:32:14Z,OWNER,@eyeseast I started work on that plugin: https://github.com/simonw/datasette-export,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1108671952,Scripted exports,
https://github.com/simonw/datasette/issues/1863#issuecomment-1324539030,https://api.github.com/repos/simonw/datasette/issues/1863,1324539030,IC_kwDOBm6k_c5O8tyW,9599,simonw,2022-11-23T04:35:14Z,2022-11-23T04:35:14Z,OWNER,If I do that I should probably update `insert` to do those validation checks as well.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1425029242,Update a single record in an existing table,
https://github.com/simonw/datasette/issues/1863#issuecomment-1324531750,https://api.github.com/repos/simonw/datasette/issues/1863,1324531750,IC_kwDOBm6k_c5O8sAm,9599,simonw,2022-11-23T04:20:47Z,2022-11-23T04:20:47Z,OWNER,"... which does imply that I'm going to do an extra layer of validation over what SQLite provides. SQLite will happily allow a text string to be added to a supposedly integer column. I'm not going to allow that - I'll return a validation error instead, unless the string can be safely coerced to the correct type.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1425029242,Update a single record in an existing table,
https://github.com/simonw/datasette/issues/1863#issuecomment-1324531085,https://api.github.com/repos/simonw/datasette/issues/1863,1324531085,IC_kwDOBm6k_c5O8r2N,9599,simonw,2022-11-23T04:19:28Z,2022-11-23T04:19:28Z,OWNER,Had a design conversation with myself in https://github.com/simonw/dclient/issues/6 where I decided that the API should allow string values to be sent to integer columns which would be automatically converted *if possible to do so* - as an API usability feature.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1425029242,Update a single record in an existing table,
https://github.com/simonw/datasette/issues/1903#issuecomment-1321262142,https://api.github.com/repos/simonw/datasette/issues/1903,1321262142,IC_kwDOBm6k_c5OwNw-,9599,simonw,2022-11-20T22:35:01Z,2022-11-20T22:35:01Z,OWNER,A want to call this `datasette/exceptions.py` inspired by Takahē: https://github.com/andrewgodwin/takahe/blob/f491fdb56e2de9200e14b855b5576009ca99dfa5/core/exceptions.py,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1455928469,Refactor all error classes into a datasette.exceptions module,
https://github.com/simonw/datasette/issues/1905#issuecomment-1320721241,https://api.github.com/repos/simonw/datasette/issues/1905,1320721241,IC_kwDOBm6k_c5OuJtZ,9599,simonw,2022-11-19T01:12:05Z,2022-11-19T01:12:05Z,OWNER,Used it to deploy this: https://fivethirtyeight.datasettes.com/-/versions,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1456012874,`publish heroku` failing due to old Python version,
https://github.com/simonw/datasette/issues/1905#issuecomment-1320689643,https://api.github.com/repos/simonw/datasette/issues/1905,1320689643,IC_kwDOBm6k_c5OuB_r,9599,simonw,2022-11-19T00:17:19Z,2022-11-19T00:41:54Z,OWNER,"The tests don't cover this bit at the moment.
Would be easier to write tests if there was a `--generate-dir` option as seen in https://datasette.io/plugins/datasette-publish-vercel","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1456012874,`publish heroku` failing due to old Python version,
https://github.com/simonw/datasette/issues/1905#issuecomment-1320706596,https://api.github.com/repos/simonw/datasette/issues/1905,1320706596,IC_kwDOBm6k_c5OuGIk,9599,simonw,2022-11-19T00:41:44Z,2022-11-19T00:41:44Z,OWNER,"Tested a deploy:
```
% datasette publish heroku fixtures.db -n datasette-issue-1905
› Warning: heroku update available from 7.63.0 to 7.66.4.
› Warning: heroku update available from 7.63.0 to 7.66.4.
› Warning: heroku update available from 7.63.0 to 7.66.4.
Creating datasette-issue-1905... done
› Warning: heroku update available from 7.63.0 to 7.66.4.
▸ Couldn't detect GNU tar. Builds could fail due to decompression errors
▸ See https://devcenter.heroku.com/articles/platform-api-deploying-slugs#create-slug-archive
▸ Please install it, or specify the '--tar' option
▸ Falling back to node's built-in compressor
-----> Building on the Heroku-22 stack
-----> Determining which buildpack to use for this app
-----> Python app detected
-----> Using Python version specified in runtime.txt
-----> Installing python-3.11.0
-----> Installing pip 22.3.1, setuptools 63.4.3 and wheel 0.37.1
-----> Installing SQLite3
-----> Installing requirements with pip
Collecting datasette
Downloading datasette-0.63.1-py3-none-any.whl (231 kB)
...
-----> Running post-compile hook
-----> Discovering process types
Procfile declares types -> web
-----> Compressing...
Done: 28M
-----> Launching...
Released v3
https://datasette-issue-1905.herokuapp.com/ deployed to Heroku
Starting November 28th, 2022, free Heroku Dynos, free Heroku Postgres, and free Heroku Data for Redis® will no longer be available.
If you have apps using any of these resources, you must upgrade to paid plans by this date to ensure your apps continue to run and to retain your data. For students, we will announce a new program by the end of September. Learn more at https://blog.heroku.com/next-chapter
```
I had to then pay for the dino because I'd run out of free hours.
https://datasette-issue-1905.herokuapp.com/-/versions shows:
```json
{
""python"": {
""version"": ""3.11.0"",
""full"": ""3.11.0 (main, Oct 24 2022, 21:34:02) [GCC 11.2.0]""
},
""datasette"": {
""version"": ""0.63.1""
}
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1456012874,`publish heroku` failing due to old Python version,
https://github.com/simonw/datasette/issues/1905#issuecomment-1320678715,https://api.github.com/repos/simonw/datasette/issues/1905,1320678715,IC_kwDOBm6k_c5Ot_U7,9599,simonw,2022-11-19T00:02:28Z,2022-11-19T00:02:28Z,OWNER,This is a strong argument for extracting the Heroku support out to a plugin - it would allow this to be fixed with a plugin release without needing to push a full release of Datasette itself.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1456012874,`publish heroku` failing due to old Python version,
https://github.com/simonw/datasette/issues/1891#issuecomment-1320625260,https://api.github.com/repos/simonw/datasette/issues/1891,1320625260,IC_kwDOBm6k_c5OtyRs,9599,simonw,2022-11-18T23:01:03Z,2022-11-18T23:01:48Z,OWNER,I think this actually needs to include a whole section of the documentation about the road to 1.0 - what to expect (planned breaking changes) etc. I can add that to the https://docs.datasette.io/en/stable/contributing.html page perhaps - or even create a Roadmap page.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1450303205,1.0a0 release notes,
https://github.com/simonw/datasette/issues/1896#issuecomment-1320616559,https://api.github.com/repos/simonw/datasette/issues/1896,1320616559,IC_kwDOBm6k_c5OtwJv,9599,simonw,2022-11-18T22:51:14Z,2022-11-18T22:51:14Z,OWNER,New methods are documented here: https://docs.datasette.io/en/1.0-dev/internals.html#resolve-database-request,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1452364777,Extract logic for resolving a URL to a database / table / row,
https://github.com/simonw/datasette/issues/1903#issuecomment-1320614541,https://api.github.com/repos/simonw/datasette/issues/1903,1320614541,IC_kwDOBm6k_c5OtvqN,9599,simonw,2022-11-18T22:47:41Z,2022-11-18T22:47:41Z,OWNER,"When I do this it's important to update the documentation for `resolve_database()` and the like:
https://github.com/simonw/datasette/blob/ee64130fa8a5ff4a24791916c696e10cf2375102/docs/internals.rst#L594","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1455928469,Refactor all error classes into a datasette.exceptions module,
https://github.com/simonw/datasette/issues/1896#issuecomment-1320588299,https://api.github.com/repos/simonw/datasette/issues/1896,1320588299,IC_kwDOBm6k_c5OtpQL,9599,simonw,2022-11-18T22:16:59Z,2022-11-18T22:17:06Z,OWNER,"Found myself needing an `await db.view_exists()` method for this, similar to the existing `await db.table_exists()` one.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1452364777,Extract logic for resolving a URL to a database / table / row,
https://github.com/simonw/datasette/issues/1896#issuecomment-1317757949,https://api.github.com/repos/simonw/datasette/issues/1896,1317757949,IC_kwDOBm6k_c5Oi2P9,9599,simonw,2022-11-16T22:27:47Z,2022-11-18T21:48:29Z,OWNER,"Open question: should `resolve_table()` know how to identify named canned queries too?
I think not, at least for the moment. Feels a bit too specialist to expose in a documented API.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1452364777,Extract logic for resolving a URL to a database / table / row,
https://github.com/simonw/datasette/issues/1863#issuecomment-1320563197,https://api.github.com/repos/simonw/datasette/issues/1863,1320563197,IC_kwDOBm6k_c5OtjH9,9599,simonw,2022-11-18T21:47:35Z,2022-11-18T21:48:07Z,OWNER,"Incomplete implementation of this view:
```python
class RowUpdateView(BaseView):
name = ""row-update""
def __init__(self, datasette):
self.ds = datasette
async def post(self, request):
database_route = tilde_decode(request.url_vars[""database""])
table = tilde_decode(request.url_vars[""table""])
try:
db = self.ds.get_database(route=database_route)
except KeyError:
return _error([""Database not found: {}"".format(database_route)], 404)
database_name = db.name
if not await db.table_exists(table):
return _error([""Table not found: {}"".format(table)], 404)
pk_values = urlsafe_components(request.url_vars[""pks""])
sql, params, pks = await row_sql_params_pks(db, table, pk_values)
results = await db.execute(sql, params, truncate=True)
rows = list(results.rows)
if not rows:
return _error([f""Record not found: {pk_values}""], 404)
# Ensure user has permission to update this row
if not await self.ds.permission_allowed(
request.actor, ""update-row"", resource=(database_name, table)
):
return _error([""Permission denied""], 403)
body = await request.post_body()
try:
data = json.loads(body)
except json.JSONDecodeError as e:
return _error([""Invalid JSON: {}"".format(e)])
if not isinstance(data, dict):
return _error([""JSON must be a dictionary""])
def update_row(conn):
sqlite_utils.Database(conn)[table].update(pk_values, updates)
await db.execute_write_fn(update_row)
result = {""ok"": True}
if data.get(""return""):
result[""row""] = {""row-here"": ""TODO""}
return Response.json(result, status=200)
```
This is before the refactor in:
- #1896","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1425029242,Update a single record in an existing table,
https://github.com/simonw/datasette/pull/1898#issuecomment-1319642535,https://api.github.com/repos/simonw/datasette/issues/1898,1319642535,IC_kwDOBm6k_c5OqCWn,9599,simonw,2022-11-18T07:28:45Z,2022-11-18T07:28:45Z,OWNER,Thanks!,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1452485922,Use DOMContentLoaded instead of load event for CodeMirror initialization,
https://github.com/simonw/datasette/issues/1899#issuecomment-1319642338,https://api.github.com/repos/simonw/datasette/issues/1899,1319642338,IC_kwDOBm6k_c5OqCTi,9599,simonw,2022-11-18T07:28:28Z,2022-11-18T07:28:28Z,OWNER,Demo: https://latest.datasette.io/fixtures,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1452495049,Clicking within the CodeMirror area below the SQL (i.e. when there's only a single line) doesn't cause the editor to get focused ,
https://github.com/simonw/datasette/issues/1901#issuecomment-1319525520,https://api.github.com/repos/simonw/datasette/issues/1901,1319525520,IC_kwDOBm6k_c5OplyQ,9599,simonw,2022-11-18T04:21:14Z,2022-11-18T07:22:37Z,OWNER,This search helps too: [https://ripgrep.datasette.io/-/ripgrep?pattern=%7B%25+block+nav&literal=on&ignore=on&glob=%21datasette%2F**](https://ripgrep.datasette.io/-/ripgrep?pattern=%7B%25+block+nav&literal=on&ignore=on&glob=%21datasette%2F**),"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1453813400,"Some plugins show ""home"" breadcrumbs twice in the top left",
https://github.com/simonw/datasette/issues/1900#issuecomment-1319631421,https://api.github.com/repos/simonw/datasette/issues/1900,1319631421,IC_kwDOBm6k_c5Op_o9,9599,simonw,2022-11-18T07:13:00Z,2022-11-18T07:13:00Z,OWNER,"You get:
```
=> [internal] load metadata for docker.io/library/python:3.11.0-slim-bullseye 0.9s
=> [internal] load build context 2.3s
=> => transferring context: 72.38MB 2.3s
=> CACHED [1/6] FROM docker.io/library/python:3.11.0-slim-bullseye@sha256:1cd45c5dad845af18d71745c017325725dc979571c1bbe625b67e6051533716c 0.0s
```
I get:
```
=> [internal] load metadata for docker.io/library/python:3.11.0-slim-bullseye 1.0s
=> [internal] load build context 0.0s
=> => transferring context: 705B 0.0s
=> CACHED [1/6] FROM docker.io/library/python:3.11.0-slim-bullseye@sha256:1cd45c5dad845af18d71745c017325725dc979571c1bbe625b67e6051533716c 0.0s
```
Both the image name and the hash are _exactly_ the same. So why are you getting an error while mine works OK?
For my machine:
```
~ % docker --version
Docker version 20.10.12, build e91ed57
~ % uname -a
Darwin Simons-MacBook-Pro-2.local 22.1.0 Darwin Kernel Version 22.1.0: Sun Oct 9 20:14:54 PDT 2022; root:xnu-8792.41.9~2/RELEASE_X86_64 x86_64
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1452572348,datasette package --spatialite throws error during build,
https://github.com/simonw/datasette/issues/1900#issuecomment-1319629469,https://api.github.com/repos/simonw/datasette/issues/1900,1319629469,IC_kwDOBm6k_c5Op_Kd,9599,simonw,2022-11-18T07:10:17Z,2022-11-18T07:10:17Z,OWNER,This is so weird! What version of Datasette do you get from `datasette --version` there - and what's your Docker version / operating system version?,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1452572348,datasette package --spatialite throws error during build,
https://github.com/simonw/datasette/issues/1901#issuecomment-1319627012,https://api.github.com/repos/simonw/datasette/issues/1901,1319627012,IC_kwDOBm6k_c5Op-kE,9599,simonw,2022-11-18T07:07:03Z,2022-11-18T07:07:03Z,OWNER,"Here's the full list of 10 plugin releases for this issue:
* [datasette-search-all 1.1.1](https://github.com/simonw/datasette-search-all/releases/tag/1.1.1)
* [datasette-ripgrep 0.7.1](https://github.com/simonw/datasette-ripgrep/releases/tag/0.7.1)
* [datasette-socrata 0.3.1](https://github.com/simonw/datasette-socrata/releases/tag/0.3.1)
* [datasette-configure-fts 1.1.1](https://github.com/simonw/datasette-configure-fts/releases/tag/1.1.1)
* [datasette-edit-templates 0.2](https://github.com/simonw/datasette-edit-templates/releases/tag/0.2)
* [datasette-copyable 0.3.2](https://github.com/simonw/datasette-copyable/releases/tag/0.3.2)
* [datasette-public 0.2.1](https://github.com/simonw/datasette-public/releases/tag/0.2.1)
* [datasette-import-table 0.3.1](https://github.com/simonw/datasette-import-table/releases/tag/0.3.1)
* [datasette-indieauth 1.2.2](https://github.com/simonw/datasette-indieauth/releases/tag/1.2.2)
* [datasette-edit-schema 0.5.2](https://github.com/simonw/datasette-edit-schema/releases/tag/0.5.2)","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1453813400,"Some plugins show ""home"" breadcrumbs twice in the top left",
https://github.com/simonw/datasette/issues/1901#issuecomment-1319493475,https://api.github.com/repos/simonw/datasette/issues/1901,1319493475,IC_kwDOBm6k_c5Opd9j,9599,simonw,2022-11-18T03:19:25Z,2022-11-18T07:03:03Z,OWNER,"Other plugins this looks like it will affect:
- [x] `datasette-ripgrep` https://github.com/simonw/datasette-ripgrep/blob/03446464420130368582022eeb5944993f64ec8f/datasette_ripgrep/templates/ripgrep.html#L37-L42
- [x] `datasette-socrata` https://github.com/simonw/datasette-socrata/blob/32fb256a461bf0e790eca10bdc7dd9d96c20f7c4/datasette_socrata/templates/datasette_socrata_error.html#L5-L10
- [x] `datasette-configure-fts` https://github.com/simonw/datasette-configure-fts/blob/eca742e5d4b9190fc22d68bc0a406c575e6d09a0/datasette_configure_fts/templates/configure_fts_database.html#L9-L14
- [x] `datasette-edit-templates` https://github.com/simonw/datasette-edit-templates/blob/f772aff4a2a4080c949746668a8ec6302dbeb0d9/datasette_edit_templates/templates/edit_template.html#L17-L23
- [x] `datasette-copyable` https://github.com/simonw/datasette-copyable/blob/204d5c912a8d48c49155c67fba7339d4bb26ab9a/datasette_copyable/templates/copyable.html#L36-L43
- [x] `datasette-public` https://github.com/simonw/datasette-public/blob/32b6a0ba53bd5714b6b41eddd8705b213c105efc/datasette_public/templates/public_table_change_privacy.html#L5-L11
- [x] `datasette-import-table` https://github.com/simonw/datasettecloud-datasette/blob/37d0fe525c6649c1aec3d1ee8bc35a684570e87f/templates/import_data.html#L5-L10
- [x] `datasette-edit-schema` (three places)
- [x] `datasette-indieauth` https://github.com/simonw/datasette-indieauth/blob/a08ce67ddad6098b1240adbeff37d040e4df53b1/datasette_indieauth/templates/indieauth.html#L5-L10","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1453813400,"Some plugins show ""home"" breadcrumbs twice in the top left",
https://github.com/simonw/datasette/issues/1901#issuecomment-1319623911,https://api.github.com/repos/simonw/datasette/issues/1901,1319623911,IC_kwDOBm6k_c5Op9zn,9599,simonw,2022-11-18T07:02:56Z,2022-11-18T07:02:56Z,OWNER,That's all of them!,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1453813400,"Some plugins show ""home"" breadcrumbs twice in the top left",
https://github.com/simonw/datasette/issues/1901#issuecomment-1319588163,https://api.github.com/repos/simonw/datasette/issues/1901,1319588163,IC_kwDOBm6k_c5Op1FD,9599,simonw,2022-11-18T06:05:11Z,2022-11-18T06:05:11Z,OWNER,"For `datasette-copyable` I want to show breadcrumbs that take database/instance permissions into account, so I'm removing `{% block nav %}` entirely and replacing it with this:
```html+jinja
{% block crumbs %}
{{ crumbs.nav(request=request, database=database, table=table) }}
{% endblock %}
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1453813400,"Some plugins show ""home"" breadcrumbs twice in the top left",
https://github.com/simonw/datasette/issues/1899#issuecomment-1319584553,https://api.github.com/repos/simonw/datasette/issues/1899,1319584553,IC_kwDOBm6k_c5Op0Mp,9599,simonw,2022-11-18T06:00:10Z,2022-11-18T06:01:50Z,OWNER,"I can't actually remember where that `min-height: 70px` came from. I just tried without it and it seems fine - especially since any time you add a newline in the editor it increases its height to fit.
I ran this in the DevTools console:
```javascript
document.querySelector('.cm-editor').style.minHeight = 'none';
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1452495049,Clicking within the CodeMirror area below the SQL (i.e. when there's only a single line) doesn't cause the editor to get focused ,
https://github.com/simonw/datasette/issues/1900#issuecomment-1319583703,https://api.github.com/repos/simonw/datasette/issues/1900,1319583703,IC_kwDOBm6k_c5Opz_X,9599,simonw,2022-11-18T05:58:31Z,2022-11-18T05:58:31Z,OWNER,Could you provide full steps to reproduce plus a SpatiaLite database file that triggered this for you? I'm not able to recreate the problem.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1452572348,datasette package --spatialite throws error during build,
https://github.com/simonw/datasette/issues/1900#issuecomment-1319583281,https://api.github.com/repos/simonw/datasette/issues/1900,1319583281,IC_kwDOBm6k_c5Opz4x,9599,simonw,2022-11-18T05:57:44Z,2022-11-18T05:57:44Z,OWNER,"Did you use the `--spatialite` option?
I just tried this:
datasette package nps-spatialite.db
It built the image OK (I didn't see the error you reported), but running the container failed with an error:
```
/tmp % docker run -p 8001:8001 7298e8e6bbfb
Usage: datasette serve [OPTIONS] [FILES]...
Try 'datasette serve --help' for help.
Error: It looks like you're trying to load a SpatiaLite database without first loading the SpatiaLite module.
Read more: https://docs.datasette.io/en/stable/spatialite.html
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1452572348,datasette package --spatialite throws error during build,
https://github.com/simonw/datasette/issues/1900#issuecomment-1319582239,https://api.github.com/repos/simonw/datasette/issues/1900,1319582239,IC_kwDOBm6k_c5Opzof,9599,simonw,2022-11-18T05:55:38Z,2022-11-18T05:55:38Z,OWNER,"Trying this out locally with this 69MB SpatiaLite file I happened to have lying around (from testing `shapefile-to-sqlite` a while ago): https://static.simonwillison.net/static/2022/nps-spatialite.db
```
% datasette package nps-spatialite.db --spatialite
...
=> [2/6] COPY . /app 0.4s
=> [3/6] WORKDIR /app 0.0s
=> [4/6] RUN apt-get update && apt-get install -y python3-dev gcc libsqlite3-mod-spatialite && rm -rf /var/lib/apt/lists/* 29.6s
=> [5/6] RUN pip install -U datasette 12.0s
=> [6/6] RUN datasette inspect nps-spatialite.db --inspect-file inspect-data.json 2.6s
=> exporting to image 3.0s
=> => exporting layers 3.0s
=> => writing image sha256:4dfef1c373c5c057ef7ac22344f834d522acef24313a1b25d2eba9e500066b8f 0.0s
```
And then:
docker run -p 8001:8001 4dfef1c373c5
This worked fine for me. I ran `datasette package` using Datasette 0.63.1.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1452572348,datasette package --spatialite throws error during build,
https://github.com/simonw/datasette/issues/1900#issuecomment-1319574972,https://api.github.com/repos/simonw/datasette/issues/1900,1319574972,IC_kwDOBm6k_c5Opx28,9599,simonw,2022-11-18T05:41:28Z,2022-11-18T05:41:28Z,OWNER,Oh this is with `datasette package`? That should work. Will investigate.,"{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1452572348,datasette package --spatialite throws error during build,
https://github.com/simonw/datasette/issues/1900#issuecomment-1319571220,https://api.github.com/repos/simonw/datasette/issues/1900,1319571220,IC_kwDOBm6k_c5Opw8U,9599,simonw,2022-11-18T05:34:35Z,2022-11-18T05:34:35Z,OWNER,Which Docker image are you using here? It looks like it's missing SpatiaLite from the image.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1452572348,datasette package --spatialite throws error during build,
https://github.com/simonw/datasette/issues/1897#issuecomment-1319570586,https://api.github.com/repos/simonw/datasette/issues/1897,1319570586,IC_kwDOBm6k_c5Opwya,9599,simonw,2022-11-18T05:33:20Z,2022-11-18T05:33:20Z,OWNER,"One of the big changes still left to do for Datasette 1.0 is to unify the JSON representation with the context psssed to the templates (via an `?_extra=` mechanism to add extra context needed by the HTML templates), because a goal for 1.0 is for the template context to be a documented API contract such that custom templates won't break with future releases.
As such I expect to do quite a bit of refactoring and cleanup on how the template context works later on.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1452457263,Serve schema JSON to the SQL editor to enable autocomplete,
https://github.com/simonw/datasette/issues/1901#issuecomment-1319528359,https://api.github.com/repos/simonw/datasette/issues/1901,1319528359,IC_kwDOBm6k_c5Opmen,9599,simonw,2022-11-18T04:27:00Z,2022-11-18T04:27:00Z,OWNER,Also `datasette-indieauth` https://github.com/simonw/datasette-indieauth/blob/a08ce67ddad6098b1240adbeff37d040e4df53b1/datasette_indieauth/templates/indieauth.html#L5-L10,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1453813400,"Some plugins show ""home"" breadcrumbs twice in the top left",
https://github.com/simonw/datasette/issues/1901#issuecomment-1319483555,https://api.github.com/repos/simonw/datasette/issues/1901,1319483555,IC_kwDOBm6k_c5Opbij,9599,simonw,2022-11-18T03:02:35Z,2022-11-18T03:02:35Z,OWNER,Looks like this issue could affect a bunch of other plugins too: https://cs.github.com/?scopeName=All+repos&scope=&q=%3Cp+class%3D%22crumbs%22%3E+user%3Asimonw,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1453813400,"Some plugins show ""home"" breadcrumbs twice in the top left",
https://github.com/simonw/datasette/issues/1901#issuecomment-1319482791,https://api.github.com/repos/simonw/datasette/issues/1901,1319482791,IC_kwDOBm6k_c5OpbWn,9599,simonw,2022-11-18T03:01:36Z,2022-11-18T03:01:36Z,OWNER,"Good catch. Looks like that bug was introduced by this change: https://github.com/simonw/datasette/commit/1a5e5f2aa951e5bd731067a49819efba68fbe8ef
From:
- https://github.com/simonw/datasette/issues/1831
The search all plugin includes this code which interacts poorly with that refactor:
https://github.com/simonw/datasette-search-all/blob/847b55c368a285e4567627029624d7872ee75cac/datasette_search_all/templates/search_all.html#L31-L36
```html+jinja
{% block nav %}
{{ super() }}
{% endblock %}
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1453813400,"Some plugins show ""home"" breadcrumbs twice in the top left",
https://github.com/simonw/datasette/issues/1897#issuecomment-1319478811,https://api.github.com/repos/simonw/datasette/issues/1897,1319478811,IC_kwDOBm6k_c5OpaYb,9599,simonw,2022-11-18T02:53:57Z,2022-11-18T02:53:57Z,OWNER,"I decided to just go for the view names, not their columns.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1452457263,Serve schema JSON to the SQL editor to enable autocomplete,
https://github.com/simonw/datasette/issues/1897#issuecomment-1319477721,https://api.github.com/repos/simonw/datasette/issues/1897,1319477721,IC_kwDOBm6k_c5OpaHZ,9599,simonw,2022-11-18T02:51:40Z,2022-11-18T02:51:40Z,OWNER,Views aren't currently available in the `_internal` schema.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1452457263,Serve schema JSON to the SQL editor to enable autocomplete,
https://github.com/simonw/datasette/issues/1897#issuecomment-1319435374,https://api.github.com/repos/simonw/datasette/issues/1897,1319435374,IC_kwDOBm6k_c5OpPxu,9599,simonw,2022-11-18T01:33:30Z,2022-11-18T01:33:30Z,OWNER,"Just noticed that this isn't including views, which it should.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1452457263,Serve schema JSON to the SQL editor to enable autocomplete,
https://github.com/simonw/datasette/issues/1897#issuecomment-1319401843,https://api.github.com/repos/simonw/datasette/issues/1897,1319401843,IC_kwDOBm6k_c5OpHlz,9599,simonw,2022-11-18T00:42:03Z,2022-11-18T00:42:23Z,OWNER,"This function works even if the SQLite JSON functions are not available:
```python
async def _table_columns(datasette, database_name):
internal = datasette.get_database(""_internal"")
result = await internal.execute(
""select table_name, name from columns where database_name = ?"",
[database_name],
)
table_columns = {}
for row in result.rows:
table_columns.setdefault(row[""table_name""], []).append(row[""name""])
return table_columns
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1452457263,Serve schema JSON to the SQL editor to enable autocomplete,
https://github.com/simonw/datasette/issues/1897#issuecomment-1317840727,https://api.github.com/repos/simonw/datasette/issues/1897,1317840727,IC_kwDOBm6k_c5OjKdX,9599,simonw,2022-11-16T23:57:52Z,2022-11-16T23:57:52Z,OWNER,In terms of permissions: if you have `execute-sql` permission for a database then it's OK for you to see the table columns for that database.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1452457263,Serve schema JSON to the SQL editor to enable autocomplete,
https://github.com/simonw/datasette/issues/1897#issuecomment-1317839781,https://api.github.com/repos/simonw/datasette/issues/1897,1317839781,IC_kwDOBm6k_c5OjKOl,9599,simonw,2022-11-16T23:56:47Z,2022-11-16T23:56:47Z,OWNER,I'm going to call this `table_columns` in the template context (because `schema` might mean `CREATE TABLE ...`.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1452457263,Serve schema JSON to the SQL editor to enable autocomplete,
https://github.com/simonw/datasette/issues/1897#issuecomment-1317838892,https://api.github.com/repos/simonw/datasette/issues/1897,1317838892,IC_kwDOBm6k_c5OjKAs,9599,simonw,2022-11-16T23:55:42Z,2022-11-16T23:55:42Z,OWNER,"Here's where the schema is hard-coded at the moment: https://github.com/simonw/datasette/blob/00e233d7a7f6443cb95fb5227c23580c48551cad/datasette/templates/_codemirror_foot.html#L2-L7
I figured out how to extract that data from the `_internal` table in this comment: https://github.com/simonw/datasette/pull/1893#issuecomment-1317475720
Although that used JSON functions which may (in a real edge-case) not be available in the version of SQLite that Datasette is running on, so probably going to use a regular SQL query and then assemble the JSON separately.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1452457263,Serve schema JSON to the SQL editor to enable autocomplete,
https://github.com/simonw/datasette/pull/1893#issuecomment-1317837416,https://api.github.com/repos/simonw/datasette/issues/1893,1317837416,IC_kwDOBm6k_c5OjJpo,9599,simonw,2022-11-16T23:54:02Z,2022-11-16T23:54:02Z,OWNER,"I'm going to tackle #1897 in the next few minutes.
Tests failed due to Prettier check, just pushed a fix so it would ignore `.bundle.js` too.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1450363982,"Upgrade to CodeMirror 6, add SQL autocomplete",
https://github.com/simonw/datasette/pull/1893#issuecomment-1317831555,https://api.github.com/repos/simonw/datasette/issues/1893,1317831555,IC_kwDOBm6k_c5OjIOD,9599,simonw,2022-11-16T23:47:13Z,2022-11-16T23:47:13Z,OWNER,I'll open a follow-up issue to fix the schema.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1450363982,"Upgrade to CodeMirror 6, add SQL autocomplete",
https://github.com/simonw/datasette/pull/1893#issuecomment-1317831425,https://api.github.com/repos/simonw/datasette/issues/1893,1317831425,IC_kwDOBm6k_c5OjIMB,9599,simonw,2022-11-16T23:47:05Z,2022-11-16T23:47:05Z,OWNER,"OK, let's do it! Thanks so much for this.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1450363982,"Upgrade to CodeMirror 6, add SQL autocomplete",
https://github.com/simonw/datasette/pull/1893#issuecomment-1317829214,https://api.github.com/repos/simonw/datasette/issues/1893,1317829214,IC_kwDOBm6k_c5OjHpe,9599,simonw,2022-11-16T23:44:36Z,2022-11-16T23:44:36Z,OWNER,Deployed that to https://datasette-pr-1893.vercel.app/fixtures - looks good to me!,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1450363982,"Upgrade to CodeMirror 6, add SQL autocomplete",
https://github.com/simonw/datasette/pull/1893#issuecomment-1317797044,https://api.github.com/repos/simonw/datasette/issues/1893,1317797044,IC_kwDOBm6k_c5Oi_y0,9599,simonw,2022-11-16T23:08:34Z,2022-11-16T23:08:34Z,OWNER,"> I can push up a commit that uses the static fixtures schema for testing, but given that the query used to generate it is authed we would still need some work to make that work on live data, right?
Yeah, push that up. I'm happy to wire in the query right after we land this.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1450363982,"Upgrade to CodeMirror 6, add SQL autocomplete",
https://github.com/simonw/datasette/issues/1896#issuecomment-1317757112,https://api.github.com/repos/simonw/datasette/issues/1896,1317757112,IC_kwDOBm6k_c5Oi2C4,9599,simonw,2022-11-16T22:26:52Z,2022-11-16T22:33:13Z,OWNER,"Some ideas from walking the dog:
Challenge: standard URL routing of request to database/table/row
Standardize on the named components of the URL patterns - `database`, `table`, `pks`
Async function that takes the request and the Datasette instance and returns a Resolved instance with:
```
.level - database or table or row (better name?)
.database - the name of the database
.db - the database object
.table - the name of the table (or view)
.is_view perhaps?
.pk_values if it's a row
```
Should this attempt to resolve names queries too?
```
.where_sql - the where fragment you use
.where_params - accompanying dictionary
await datasette.resolve_request(request)
```
Or even better three methods:
```python
datasette.resolve_database(request)
datasette.resolve_table(request)
datasette.resolve_row(request)
```
These can be typed correctly
Methods raise `NotFound` if not found","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1452364777,Extract logic for resolving a URL to a database / table / row,
https://github.com/simonw/datasette/issues/1863#issuecomment-1317755263,https://api.github.com/repos/simonw/datasette/issues/1863,1317755263,IC_kwDOBm6k_c5Oi1l_,9599,simonw,2022-11-16T22:24:59Z,2022-11-16T22:24:59Z,OWNER,"In trying to write this I realize that there's a lot of duplicated code with delete row, specifically around resolving the incoming URL into a row (or a database or a table).
Since this is so common, I think it's worth extracting the logic out first.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1425029242,Update a single record in an existing table,
https://github.com/simonw/datasette/pull/1893#issuecomment-1317746206,https://api.github.com/repos/simonw/datasette/issues/1893,1317746206,IC_kwDOBm6k_c5OizYe,9599,simonw,2022-11-16T22:17:24Z,2022-11-16T22:17:24Z,OWNER,Deployed 0a649e8f78c23e8db6869442eeb0dfe36a5443da: https://datasette-pr-1893.vercel.app/fixtures,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1450363982,"Upgrade to CodeMirror 6, add SQL autocomplete",
https://github.com/simonw/datasette/pull/1893#issuecomment-1317744563,https://api.github.com/repos/simonw/datasette/issues/1893,1317744563,IC_kwDOBm6k_c5Oiy-z,9599,simonw,2022-11-16T22:16:03Z,2022-11-16T22:16:03Z,OWNER,"Honestly I'm not too bothered if table names with weird characters don't work correctly here - I care about those in the Datasette `fixtures.db` database because Datasette aims to support ANY valid SQLite database, so I need stuff in the test suite that includes weird edge cases like this. But I would hope very few people actually create tables with spaces in their names, so it's not a huge concern to me if autocompletion doesn't work properly for those.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1450363982,"Upgrade to CodeMirror 6, add SQL autocomplete",
https://github.com/simonw/datasette/pull/1893#issuecomment-1317475720,https://api.github.com/repos/simonw/datasette/issues/1893,1317475720,IC_kwDOBm6k_c5OhxWI,9599,simonw,2022-11-16T18:25:16Z,2022-11-16T18:25:16Z,OWNER,"Here's a query that returns the exact JSON we need to pass to the schema:
https://latest.datasette.io/_internal?sql=with+inner+as+%28%0D%0A++select%0D%0A++++table_name%2C%0D%0A++++json_group_array%28name%29+as+table_columns%0D%0A++from%0D%0A++++columns%0D%0A++where%0D%0A++++database_name+%3D+%3Adatabase%0D%0A++group+by%0D%0A++++table_name%0D%0A%29%0D%0Aselect%0D%0A++json_group_object%28table_name%2C+table_columns%29%0D%0Afrom%0D%0A++inner&database=fixtures
```sql
with inner as (
select
table_name,
json_group_array(name) as table_columns
from
columns
where
database_name = :database
group by
table_name
)
select
json_group_object(table_name, table_columns)
from
inner
```
Returns (after pretty-printing):
```json
{
""123_starts_with_digits"": [
""content""
],
""Table With Space In Name"": [
""content"",
""pk""
],
""attraction_characteristic"": [
""name"",
""pk""
],
""binary_data"": [
""data""
],
""complex_foreign_keys"": [
""f1"",
""f2"",
""f3"",
""pk""
],
""compound_primary_key"": [
""content"",
""pk1"",
""pk2""
],
""compound_three_primary_keys"": [
""content"",
""pk1"",
""pk2"",
""pk3""
],
""custom_foreign_key_label"": [
""foreign_key_with_custom_label"",
""pk""
],
""facet_cities"": [
""id"",
""name""
],
""facetable"": [
""_city_id"",
""_neighborhood"",
""complex_array"",
""created"",
""distinct_some_null"",
""n"",
""on_earth"",
""pk"",
""planet_int"",
""state"",
""tags""
],
""foreign_key_references"": [
""foreign_key_compound_pk1"",
""foreign_key_compound_pk2"",
""foreign_key_with_blank_label"",
""foreign_key_with_label"",
""foreign_key_with_no_label"",
""pk""
],
""infinity"": [
""value""
],
""no_primary_key"": [
""a"",
""b"",
""c"",
""content""
],
""primary_key_multiple_columns"": [
""content"",
""content2"",
""id""
],
""primary_key_multiple_columns_explicit_label"": [
""content"",
""content2"",
""id""
],
""roadside_attraction_characteristics"": [
""attraction_id"",
""characteristic_id""
],
""roadside_attractions"": [
""address"",
""latitude"",
""longitude"",
""name"",
""pk"",
""url""
],
""searchable"": [
""name with . and spaces"",
""pk"",
""text1"",
""text2""
],
""searchable_fts"": [
""__langid"",
""docid"",
""name with . and spaces"",
""searchable_fts"",
""text1"",
""text2""
],
""searchable_fts_docsize"": [
""docid"",
""size""
],
""searchable_fts_segdir"": [
""end_block"",
""idx"",
""leaves_end_block"",
""level"",
""root"",
""start_block""
],
""searchable_fts_segments"": [
""block"",
""blockid""
],
""searchable_fts_stat"": [
""id"",
""value""
],
""searchable_tags"": [
""searchable_id"",
""tag""
],
""select"": [
""and"",
""group"",
""having"",
""json""
],
""simple_primary_key"": [
""content"",
""id""
],
""sortable"": [
""content"",
""pk1"",
""pk2"",
""sortable"",
""sortable_with_nulls"",
""sortable_with_nulls_2"",
""text""
],
""table/with/slashes.csv"": [
""content"",
""pk""
],
""tags"": [
""tag""
],
""units"": [
""distance"",
""frequency"",
""pk""
]
}
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1450363982,"Upgrade to CodeMirror 6, add SQL autocomplete",
https://github.com/simonw/datasette/pull/1893#issuecomment-1317465874,https://api.github.com/repos/simonw/datasette/issues/1893,1317465874,IC_kwDOBm6k_c5Ohu8S,9599,simonw,2022-11-16T18:21:17Z,2022-11-16T18:21:54Z,OWNER,"I was worrying about the server-side overhead of collecting together all of the tables and column names for databases that might have hundreds of tables... but then I remember that I built the `_internal` table precisely for this kind of thing - so gathering all of that data should still only be a single SQL query against an in-memory database.
https://latest.datasette.io/login-as-root and then visit this page for an example query: https://latest.datasette.io/_internal?sql=select%0D%0A++database_name%2C%0D%0A++table_name%2C%0D%0A++json_group_array%28name%29%0D%0Afrom%0D%0A++columns%0D%0Awhere%0D%0A++database_name+%21%3D+%27_internal%27%0D%0Agroup+by%0D%0A++database_name%2C%0D%0A++table_name
```sql
select
database_name,
table_name,
json_group_array(name)
from
columns
where
database_name != '_internal'
group by
database_name,
table_name
```
database_name | table_name | json_group_array(name)
-- | -- | --
extra_database | searchable | [""pk"",""text1"",""text2""]
extra_database | searchable_fts | [""__langid"",""content"",""docid"",""searchable_fts"",""text1"",""text2""]
extra_database | searchable_fts_content | [""c0text1"",""c1text2"",""c2content"",""docid""]
extra_database | searchable_fts_segdir | [""end_block"",""idx"",""leaves_end_block"",""level"",""root"",""start_block""]
extra_database | searchable_fts_segments | [""block"",""blockid""]
fixtures | 123_starts_with_digits | [""content""]
fixtures | Table With Space In Name | [""content"",""pk""]
fixtures | attraction_characteristic | [""name"",""pk""]
fixtures | binary_data | [""data""]
fixtures | complex_foreign_keys | [""f1"",""f2"",""f3"",""pk""]
fixtures | compound_primary_key | [""content"",""pk1"",""pk2""]
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1450363982,"Upgrade to CodeMirror 6, add SQL autocomplete",
https://github.com/simonw/datasette/pull/1893#issuecomment-1317456909,https://api.github.com/repos/simonw/datasette/issues/1893,1317456909,IC_kwDOBm6k_c5OhswN,9599,simonw,2022-11-16T18:17:39Z,2022-11-16T18:17:39Z,OWNER,"Tiny feature request (since you're in this code already) - I keep hitting Command+Enter on my macOS keyboard to submit the query, but the correct shortcut is Shift+Enter. Would be great if both worked!","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1450363982,"Upgrade to CodeMirror 6, add SQL autocomplete",
https://github.com/simonw/datasette/pull/1893#issuecomment-1317452541,https://api.github.com/repos/simonw/datasette/issues/1893,1317452541,IC_kwDOBm6k_c5Ohrr9,9599,simonw,2022-11-16T18:15:52Z,2022-11-16T18:15:52Z,OWNER,"Deployed latest copy with:
```
datasette publish vercel fixtures.db \
--project datasette-pr-1893 \
--about 'PR 1893' \
--about_url https://github.com/simonw/datasette/pull/1893 \
--scope datasette \
--branch eccb1c6c781d69d8ec3c542ef65c78a4a0927a7c
```
https://datasette-pr-1893.vercel.app/fixtures
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1450363982,"Upgrade to CodeMirror 6, add SQL autocomplete",
https://github.com/simonw/datasette/pull/1893#issuecomment-1317449610,https://api.github.com/repos/simonw/datasette/issues/1893,1317449610,IC_kwDOBm6k_c5Ohq-K,9599,simonw,2022-11-16T18:14:28Z,2022-11-16T18:14:28Z,OWNER,"> I'm thinking of also adding `count` to the list since that's a common thing people would want to autocomplete. I notice BQ console highlights `count` in the same manner as other keywords like `select` as well.
Huh, yeah we should definitely have `count` - surprised it's not on the list on https://www.sqlite.org/lang_keywords.html which is why we didn't get it from the GPT-3 generated schema.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1450363982,"Upgrade to CodeMirror 6, add SQL autocomplete",
https://github.com/simonw/sqlite-utils/issues/512#issuecomment-1316530539,https://api.github.com/repos/simonw/sqlite-utils/issues/512,1316530539,IC_kwDOCGYnMM5OeKlr,9599,simonw,2022-11-16T07:49:50Z,2022-11-16T07:49:50Z,OWNER,Tests passed.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1450952393,mypy failures in CI,
https://github.com/simonw/sqlite-utils/issues/512#issuecomment-1316447182,https://api.github.com/repos/simonw/sqlite-utils/issues/512,1316447182,IC_kwDOCGYnMM5Od2PO,9599,simonw,2022-11-16T06:32:31Z,2022-11-16T06:32:31Z,OWNER,"Test failed again: https://github.com/simonw/sqlite-utils/actions/runs/3476950474/jobs/5812663096
`E: Failed to fetch http://azure.archive.ubuntu.com/ubuntu/pool/universe/s/spatialite/libsqlite3-mod-spatialite_4.3.0a-6build1_amd64.deb Unable to connect to azure.archive.ubuntu.com:http:`
That looks like an intermittent error. I'll try running it again in the morning.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1450952393,mypy failures in CI,
https://github.com/simonw/sqlite-utils/issues/512#issuecomment-1316437748,https://api.github.com/repos/simonw/sqlite-utils/issues/512,1316437748,IC_kwDOCGYnMM5Odz70,9599,simonw,2022-11-16T06:24:31Z,2022-11-16T06:24:31Z,OWNER,"```
sqlite-utils % pipx run no_implicit_optional .
Calculating full-repo metadata...
Executing codemod...
11.43s 98% complete, 0.24s estimated for 5 files to go...
```
Then:
```
Finished codemodding 239 files!
- Transformed 239 files successfully.
- Skipped 0 files.
- Failed to codemod 0 files.
- 0 warnings were generated.
```
Here's the diff:
```diff
diff --git a/sqlite_utils/db.py b/sqlite_utils/db.py
index a06f4b7..e819d17 100644
--- a/sqlite_utils/db.py
+++ b/sqlite_utils/db.py
@@ -297,12 +297,12 @@ class Database:
def __init__(
self,
- filename_or_conn: Union[str, pathlib.Path, sqlite3.Connection] = None,
+ filename_or_conn: Optional[Union[str, pathlib.Path, sqlite3.Connection]] = None,
memory: bool = False,
- memory_name: str = None,
+ memory_name: Optional[str] = None,
recreate: bool = False,
recursive_triggers: bool = True,
- tracer: Callable = None,
+ tracer: Optional[Callable] = None,
use_counts_table: bool = False,
):
assert (filename_or_conn is not None and (not memory and not memory_name)) or (
@@ -341,7 +341,7 @@ class Database:
self.conn.close()
@contextlib.contextmanager
- def tracer(self, tracer: Callable = None):
+ def tracer(self, tracer: Optional[Callable] = None):
""""""
Context manager to temporarily set a tracer function - all executed SQL queries will
be passed to this.
@@ -378,7 +378,7 @@ class Database:
def register_function(
self,
- fn: Callable = None,
+ fn: Optional[Callable] = None,
deterministic: bool = False,
replace: bool = False,
name: Optional[str] = None,
@@ -879,7 +879,7 @@ class Database:
pk: Optional[Any] = None,
foreign_keys: Optional[ForeignKeysType] = None,
column_order: Optional[List[str]] = None,
- not_null: Iterable[str] = None,
+ not_null: Optional[Iterable[str]] = None,
defaults: Optional[Dict[str, Any]] = None,
hash_id: Optional[str] = None,
hash_id_columns: Optional[Iterable[str]] = None,
@@ -1129,7 +1129,7 @@ class Database:
sql += "" [{}]"".format(name)
self.execute(sql)
- def init_spatialite(self, path: str = None) -> bool:
+ def init_spatialite(self, path: Optional[str] = None) -> bool:
""""""
The ``init_spatialite`` method will load and initialize the SpatiaLite extension.
The ``path`` argument should be an absolute path to the compiled extension, which
@@ -1182,7 +1182,7 @@ class Queryable:
def count_where(
self,
- where: str = None,
+ where: Optional[str] = None,
where_args: Optional[Union[Iterable, dict]] = None,
) -> int:
""""""
@@ -1213,12 +1213,12 @@ class Queryable:
def rows_where(
self,
- where: str = None,
+ where: Optional[str] = None,
where_args: Optional[Union[Iterable, dict]] = None,
- order_by: str = None,
+ order_by: Optional[str] = None,
select: str = ""*"",
- limit: int = None,
- offset: int = None,
+ limit: Optional[int] = None,
+ offset: Optional[int] = None,
) -> Generator[dict, None, None]:
""""""
Iterate over every row in this table or view that matches the specified where clause.
@@ -1251,11 +1251,11 @@ class Queryable:
def pks_and_rows_where(
self,
- where: str = None,
+ where: Optional[str] = None,
where_args: Optional[Union[Iterable, dict]] = None,
- order_by: str = None,
- limit: int = None,
- offset: int = None,
+ order_by: Optional[str] = None,
+ limit: Optional[int] = None,
+ offset: Optional[int] = None,
) -> Generator[Tuple[Any, Dict], None, None]:
""""""
Like ``.rows_where()`` but returns ``(pk, row)`` pairs - ``pk`` can be a single value or tuple.
@@ -1345,7 +1345,7 @@ class Table(Queryable):
pk: Optional[Any] = None,
foreign_keys: Optional[ForeignKeysType] = None,
column_order: Optional[List[str]] = None,
- not_null: Iterable[str] = None,
+ not_null: Optional[Iterable[str]] = None,
defaults: Optional[Dict[str, Any]] = None,
batch_size: int = 100,
hash_id: Optional[str] = None,
@@ -1545,7 +1545,7 @@ class Table(Queryable):
pk: Optional[Any] = None,
foreign_keys: Optional[ForeignKeysType] = None,
column_order: Optional[List[str]] = None,
- not_null: Iterable[str] = None,
+ not_null: Optional[Iterable[str]] = None,
defaults: Optional[Dict[str, Any]] = None,
hash_id: Optional[str] = None,
hash_id_columns: Optional[Iterable[str]] = None,
@@ -2464,7 +2464,7 @@ class Table(Queryable):
columns: Optional[Iterable[str]] = None,
limit: Optional[int] = None,
offset: Optional[int] = None,
- where: str = None,
+ where: Optional[str] = None,
where_args: Optional[Union[Iterable, dict]] = None,
quote: bool = False,
) -> Generator[dict, None, None]:
@@ -2527,7 +2527,7 @@ class Table(Queryable):
def delete_where(
self,
- where: str = None,
+ where: Optional[str] = None,
where_args: Optional[Union[Iterable, dict]] = None,
analyze: bool = False,
) -> ""Table"":
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1450952393,mypy failures in CI,
https://github.com/simonw/datasette/pull/1893#issuecomment-1316412234,https://api.github.com/repos/simonw/datasette/issues/1893,1316412234,IC_kwDOBm6k_c5OdttK,9599,simonw,2022-11-16T06:00:39Z,2022-11-16T06:01:36Z,OWNER,"Should note though that this is a classic example of GPT-3 making stuff up in places.
> current: Returns the current date, time, or timestamp
`select current` throws an error for me: https://latest.datasette.io/_memory?sql=select+current
`select current_date, current_time, current_timestamp` works though: https://latest.datasette.io/_memory?sql=select+current_date%2C+current_time%2C+current_timestamp
So let's drop `current` from the list. I'm OK with it though, I think it's likely good enough for the first attempt at this.
We should drop `temp` and `temporary` too.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1450363982,"Upgrade to CodeMirror 6, add SQL autocomplete",
https://github.com/simonw/datasette/pull/1893#issuecomment-1316401895,https://api.github.com/repos/simonw/datasette/issues/1893,1316401895,IC_kwDOBm6k_c5OdrLn,9599,simonw,2022-11-16T05:50:40Z,2022-11-16T05:50:40Z,OWNER,"So I think our dialect (at least to start with) should be:
```
keywords:
""and as asc between by case cast current current_date current_time current_timestamp desc distinct each else escape except exists explain filter first for from full generated group having if in index inner intersect into isnull join last left like limit not null or order outer over pragma primary query raise range regexp right rollback row select set table temp temporary then to union unique using values view virtual when where"",
// https://www.sqlite.org/datatype3.html
types: ""null integer real text blob"",
builtin:
""""
```
I left `builtin` blank here because I don't think we need any of those things at all.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1450363982,"Upgrade to CodeMirror 6, add SQL autocomplete",
https://github.com/simonw/datasette/pull/1893#issuecomment-1316400688,https://api.github.com/repos/simonw/datasette/issues/1893,1316400688,IC_kwDOBm6k_c5Odq4w,9599,simonw,2022-11-16T05:49:31Z,2022-11-16T05:49:31Z,OWNER,"OK, I decided to play around with GPT-3 for this. https://beta.openai.com/playground
I prompted:
```
For each keyword say it it can be used in a SQLite select statement: and as asc between by case cast
cross current current_date current_time current_timestamp desc detach distinct each else end escape
except exclude exclusive exists explain fail filter first following for foreign from full generated glob
group groups having if ignore immediate in index indexed initially inner insert instead intersect into
is isnull join key last left like limit match materialized natural no not nothing notnull null nulls of
offset on or order others outer over partition plan pragma preceding primary query raise range recursive
references regexp reindex release rename replace restrict returning right rollback row rows savepoint
select set table temp temporary then ties to transaction trigger unbounded union unique update using
vacuum values view virtual when where window with without
- and: yes
- as: yes
- asc: yes
- between: yes
- by: yes
- case: yes
- cast: yes
- current: yes
- current_date: yes
- current_time: yes
- current_timestamp: yes
- desc: yes
- detach: no
- distinct: yes
- each: yes
- else: yes
- end: yes
- escape: yes
- except: no
- exclude: yes
- exclusive: no
```
Having given it these examples I ran it a few times:
```
- exists: yes
- explain: no
- fail: no
- filter: yes
- first: yes
- following: no
- for: yes
- foreign: no
- from: yes
- full: no
- generated: no
- glob: yes
- group: yes
- groups: no
- having: yes
- if: yes
- ignore: no
- immediate: no
- in: yes
- index: yes
- indexed: no
- initially: no
- inner: no
- insert: no
- instead: no
- intersect: no
- into: yes
- is: yes
- isnull: yes
- join: yes
- key: no
- last: yes
- left: yes
- like: yes
- limit: yes
- match: no
- materialized: no
-natural: yes
- no: no
- not: no
- nothing: no
- notnull: yes
- null: yes
- nulls: yes
- of: no
- offset: yes
- on: yes
- or: yes
- order: yes
- others: yes
- outer: no
- over: no
- partition: no
- pragma: no
- preceding: no
- primary: no
- query: no
- raise: no
- range: no
- recursive: no
- references: no
- regexp: yes
- reindex: no
- release: no
- rename: no
- replace: no
- restrict: no
- returning: yes
- right: yes
- rollback: no
- row: yes
- rows: no
- savepoint: no
- select: yes
- set: no
- table: yes
- temp: yes
- temporary: yes
- then: yes
- ties: no
- to: yes
- transaction: no
- trigger: no
- unbounded: no
- union: yes
- unique: yes
- update: no
- using: yes
- vacuum: no
- values: yes
- view: yes
- virtual: no
- when: yes
- where: yes
- window: no
- with: yes
- without: no
```
I added this:
```
A space separated list of all of the keywords for which the answer was yes:
```
It replied:
```
and as asc between by case cast current current_date current_time current_timestamp desc distinct each else escape except exists explain filter first for from full generated group having if in index inner intersect into isnull join last left like limit not null or order outer over pragma primary query raise range regexp right rollback row select set table temp temporary then to union unique using values view virtual when where
```
Then I tried doing this:
```
Produce a short summary of how each of those are used:
- and:
```
This kind of worked too! It ended up defining every keyword, not just the ones for which the ""is it used in select"" was ""yes"", but it gave me this list:
> - and: Used to combine multiple conditions in a WHERE clause
> - as: Rename a column or table in the output of a query
> - asc: Sort results in ascending order
> - between: Used in a WHERE clause to specify a range
> - by: Specifies how to sort the results of a query
> - case: Used to create different outputs based on conditions
> - cast: Converts a value from one data type to another
> - current: Returns the current date, time, or timestamp
> - current_date: Returns the current date
> - current_time: Returns the current time
> - current_timestamp: Returns the current timestamp
> - desc: Sort results in descending order
> - distinct: Selects only unique values
> - each: Used with aggregate functions
> - else: Used with the CASE statement
> - escape: Used to escape special characters in a string
> - except: Returns all rows from the first query that are not in the second query
> - exists: Used with subqueries
> - explain: Analyzes a query and returns information about it
> - fail: Used with the ON CONFLICT clause
> - filter: Used to specify conditions for aggregate functions
> - first: Used to specify conditions for aggregate functions
> - following: Used with window functions
> - for: Used to specify conditions for aggregate functions
> - foreign: Creates a foreign key
> - from: Specifies the table(s) to select data from
> - full: Used with outer joins
> - generated: Used with generated columns
> - glob: Used to match a string with a pattern
> - group: Groups together rows that have the same values
> - groups: Used with aggregate functions
> - having: Specifies conditions for groups
> - if: Used with the ON CONFLICT clause
> - ignore: Used with the ON CONFLICT clause
> - immediate: Used with the ON CONFLICT clause
> - in: Used to check if a value is in a list
> - index: Creates an index on a column or columns
> - indexed: Used with the ON CONFLICT clause
> - initially: Used with the ON CONFLICT clause
> - inner: Creates an inner join
> - insert: Inserts new data into a table
> - instead: Used with triggers
> - intersect: Returns all rows that are in both queries
> - into: Specifies the table to insert data into
> - is: Used to compare values
> - isnull: Checks if a value is null
> - join: Joins two or more tables together
> - key: Creates a primary key or unique key
> - last: Used to specify conditions for aggregate functions
> - left: Creates a left join
> - like: Checks if a value matches a pattern
> - limit: Specifies the maximum number of rows to return
> - not: Negates a condition
> - null: Specifies a null value
> - nulls: Used with the ORDER BY clause
> - of: Used with window functions
> - offset: Specifies the number of rows to skip
> - on: Specifies conditions for a join
> - or: Checks if either condition is true
> - order: Specifies how to sort the results of a query
> - others: Used with the CASE statement
> - outer: Creates an outer join
> - over: Used with window functions
> - partition: Used with window functions
> - pragma: Used to query or change various internal settings
> - preceding: Used with window functions
> - primary: Creates a primary key
> - query: Used with the ON CONFLICT clause
> - raise: Used with the ON CONFLICT clause
> - range: Used with window functions
> - recursive: Used with common table expressions
> - references: Creates a foreign key
> - regexp: Checks if a value matches a regular expression
> - reindex: Recreates an index
> - release: Used with savepoints
> - rename: Changes the name of a table or column
> - replace: Inserts new data into a table or updates existing data
> - restrict: Used with savepoints
> - returning: Returns data from inserted or updated rows
> - right: Creates a right join
> - rollback: Rolls back a transaction
> - row: Used with aggregate functions
> - select: Selects data from a table
> - set: Changes the value of a column
> - table: Specifies the table(s) to select data from
> - temp: Creates a temporary table
> - temporary: Creates a temporary table
> - then: Used with the CASE statement
> - ties: Used with the ORDER BY clause
> - to: Used with common table expressions
> - transaction: Starts a transaction
> - trigger: Creates a trigger
> - unbounded: Used with window functions
> - union: Returns all rows from both queries
> - unique: Creates a unique constraint
> - update: Updates existing data in a table
> - using: Specifies the table(s) to join
> - vacuum: Optimizes a database
> - values: Specifies the values to insert into a table
> - view: Creates a view
> - virtual: Creates a virtual table
> - when: Used with the CASE statement
> - where: Specifies conditions for a query
> - window: Used with window functions
> - with: Specifies common table expressions used in a query
> - without: Used with indexes","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1450363982,"Upgrade to CodeMirror 6, add SQL autocomplete",
https://github.com/simonw/datasette/pull/1893#issuecomment-1316340865,https://api.github.com/repos/simonw/datasette/issues/1893,1316340865,IC_kwDOBm6k_c5OdcSB,9599,simonw,2022-11-16T04:49:30Z,2022-11-16T04:49:43Z,OWNER,"> The main issue is that we don't pass the relevant table data down to QueryView.
If you can come up with a static example JSON data structure example that does the right thing, I'm happy to refactor QueryView to make that available to the template - or even have a separate `fetch()` that grabs just the data needed for the autocomplete as a separate hit when the page loads (whichever has better performance implications). I'm working a fair amount in the view classes at the moment so adding this to that work would make sense.
","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1450363982,"Upgrade to CodeMirror 6, add SQL autocomplete",
https://github.com/simonw/datasette/pull/1893#issuecomment-1316294156,https://api.github.com/repos/simonw/datasette/issues/1893,1316294156,IC_kwDOBm6k_c5OdQ4M,9599,simonw,2022-11-16T04:00:12Z,2022-11-16T04:00:12Z,OWNER,Have you ever seen CodeMirror correctly auto-completing columns? I'm not entirely sure I believe that the feature works anywhere else.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1450363982,"Upgrade to CodeMirror 6, add SQL autocomplete",
https://github.com/simonw/datasette/pull/1893#issuecomment-1316293353,https://api.github.com/repos/simonw/datasette/issues/1893,1316293353,IC_kwDOBm6k_c5OdQrp,9599,simonw,2022-11-16T03:59:03Z,2022-11-16T03:59:03Z,OWNER,"Deployed a fresh copy:
```
datasette publish vercel fixtures.db \
--branch b7b2942b13f9ea09cfa9f8c73e2869b9bd2349ae \
--project datasette-pr-1893 \
--about 'PR 1893' \
--about_url https://github.com/simonw/datasette/pull/1893 \
--scope datasette
```
https://datasette-pr-1893.vercel.app/fixtures","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1450363982,"Upgrade to CodeMirror 6, add SQL autocomplete",
https://github.com/simonw/datasette/issues/1890#issuecomment-1316262169,https://api.github.com/repos/simonw/datasette/issues/1890,1316262169,IC_kwDOBm6k_c5OdJEZ,9599,simonw,2022-11-16T03:22:40Z,2022-11-16T03:22:40Z,OWNER,"Actually this works as it should in desktop Safari:
![autocomplete-safari](https://user-images.githubusercontent.com/9599/202075764-fbc4b4c8-c92f-4f69-81fd-84002de5aea7.gif)
I'm going to just put up with the weird behaviour in Mobile Safari.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1448143294,Autocomplete text entry for filter values that correspond to facets,
https://github.com/simonw/datasette/pull/1893#issuecomment-1316253186,https://api.github.com/repos/simonw/datasette/issues/1893,1316253186,IC_kwDOBm6k_c5OdG4C,9599,simonw,2022-11-16T03:16:36Z,2022-11-16T03:16:36Z,OWNER,Yeah I haven't written this down anywhere but Datasette definitely has an undocumented preference for lower-case SQL.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1450363982,"Upgrade to CodeMirror 6, add SQL autocomplete",
https://github.com/simonw/datasette/issues/1890#issuecomment-1316242752,https://api.github.com/repos/simonw/datasette/issues/1890,1316242752,IC_kwDOBm6k_c5OdEVA,9599,simonw,2022-11-16T03:10:52Z,2022-11-16T03:12:47Z,OWNER,"https://bugs.webkit.org/show_bug.cgi?id=201768 - "" Datalist option's label not used"" - marked as RESOLVED FIXED on March 31st 2020.
The commit: https://trac.webkit.org/changeset/259330/webkit
And here's the test mirrored on GitHub: https://cs.github.com/qtwebkit/webkit-mirror/blob/cc3fcd0b4bad1f7cf77c26e34aa01d16618d6d5e/LayoutTests/fast/forms/datalist/datalist-option-labels.html?q=datalist-option-labels.html","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1448143294,Autocomplete text entry for filter values that correspond to facets,
https://github.com/simonw/datasette/issues/1890#issuecomment-1316240839,https://api.github.com/repos/simonw/datasette/issues/1890,1316240839,IC_kwDOBm6k_c5OdD3H,9599,simonw,2022-11-16T03:09:11Z,2022-11-16T03:09:11Z,OWNER,"Here's a polyfill for `