html_url,issue_url,id,node_id,user,user_label,created_at,updated_at,author_association,body,reactions,issue,issue_label,performed_via_github_app
https://github.com/simonw/datasette/pull/1870#issuecomment-1294238862,https://api.github.com/repos/simonw/datasette/issues/1870,1294238862,IC_kwDOBm6k_c5NJISO,22429695,codecov[bot],2022-10-27T23:44:25Z,2022-10-27T23:44:25Z,NONE,"# [Codecov](https://codecov.io/gh/simonw/datasette/pull/1870?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report
Base: **92.55**% // Head: **92.55**% // No change to project coverage :thumbsup:
> Coverage data is based on head [(`4faa4fd`)](https://codecov.io/gh/simonw/datasette/pull/1870?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) compared to base [(`bf00b0b`)](https://codecov.io/gh/simonw/datasette/commit/bf00b0b59b6692bdec597ac9db4e0b497c5a47b4?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).
> Patch has no changes to coverable lines.
Additional details and impacted files
```diff
@@ Coverage Diff @@
## main #1870 +/- ##
=======================================
Coverage 92.55% 92.55%
=======================================
Files 35 35
Lines 4432 4432
=======================================
Hits 4102 4102
Misses 330 330
```
| [Impacted Files](https://codecov.io/gh/simonw/datasette/pull/1870?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage Δ | |
|---|---|---|
| [datasette/app.py](https://codecov.io/gh/simonw/datasette/pull/1870/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL2FwcC5weQ==) | `94.30% <ø> (ø)` | |
Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)
![]()
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1421544654,API to insert a single record into an existing table,
https://github.com/simonw/datasette/issues/1851#issuecomment-1292997608,https://api.github.com/repos/simonw/datasette/issues/1851,1292997608,IC_kwDOBm6k_c5NEZPo,9599,simonw,2022-10-27T04:54:53Z,2022-10-27T19:05:50Z,OWNER,"I'm going to change the design of this to:
```
{
""insert"": {
""title"" :""...""
}
}
```
Renaming `""row""` to `""insert""`.
This will be consistent with adding `""drop"": true` for dropping a table, and maybe other verbs like for modifying the schema.
The API response will look like this:
```json
{
""inserted_row"": {
""id"": 1,
""title"": ""...""
}
}
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1421544654,API to insert a single record into an existing table,
https://github.com/simonw/datasette/issues/1860#issuecomment-1293939737,https://api.github.com/repos/simonw/datasette/issues/1860,1293939737,IC_kwDOBm6k_c5NH_QZ,9599,simonw,2022-10-27T18:57:37Z,2022-10-27T18:57:37Z,OWNER,The new code is now live at https://latest.datasette.io/fixtures,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1424378012,SQL query field can't begin by a comment,
https://github.com/simonw/datasette/issues/1860#issuecomment-1293928738,https://api.github.com/repos/simonw/datasette/issues/1860,1293928738,IC_kwDOBm6k_c5NH8ki,9599,simonw,2022-10-27T18:46:31Z,2022-10-27T18:46:31Z,OWNER,I think mine has a better pattern for handling `/* ... anything in here that isn't */ ... */`,"{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1424378012,SQL query field can't begin by a comment,
https://github.com/simonw/datasette/issues/1860#issuecomment-1293928230,https://api.github.com/repos/simonw/datasette/issues/1860,1293928230,IC_kwDOBm6k_c5NH8cm,9599,simonw,2022-10-27T18:46:03Z,2022-10-27T18:46:03Z,OWNER,"Here's yours on Debuggex: https://www.debuggex.com/r/HjdJryTy9ezGsuWK
![]()
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1424378012,SQL query field can't begin by a comment,
https://github.com/simonw/datasette/issues/1860#issuecomment-1293926417,https://api.github.com/repos/simonw/datasette/issues/1860,1293926417,IC_kwDOBm6k_c5NH8AR,9599,simonw,2022-10-27T18:44:20Z,2022-10-27T18:45:21Z,OWNER,"Hah, I just came up with this one - we were clearly working on this at the same time!
`^\s*((?:\-\-.*?\n\s*)|(?:\/\*((?!\*\/)[\s\S])*\*\/)\s*)*\s*select\b`
![]()
https://www.debuggex.com/r/Rbw-UWD9PdOU2GyO
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1424378012,SQL query field can't begin by a comment,
https://github.com/simonw/datasette/issues/1860#issuecomment-1293912781,https://api.github.com/repos/simonw/datasette/issues/1860,1293912781,IC_kwDOBm6k_c5NH4rN,562352,CharlesNepote,2022-10-27T18:31:15Z,2022-10-27T18:31:15Z,NONE,"Here is my suggestion:
`^\s*((?:\-\-.*?\n\s*)|(?:/\*.*?(?=\*/)\*/\s*))*select\b`
See the following test: https://regex101.com/r/Doeqqa/1
And here I played all your tests: https://regexr.com/713ir
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1424378012,SQL query field can't begin by a comment,
https://github.com/simonw/datasette/issues/1866#issuecomment-1293893789,https://api.github.com/repos/simonw/datasette/issues/1866,1293893789,IC_kwDOBm6k_c5NH0Cd,9599,simonw,2022-10-27T18:13:00Z,2022-10-27T18:13:00Z,OWNER,If people care about that kind of thing they could always push all of their inserts to a table called `_tablename` and then atomically rename that once they've uploaded all of the data (assuming I provide an atomic-rename-this-table mechanism).,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1426001541,API for bulk inserting records into a table,
https://github.com/simonw/datasette/issues/1866#issuecomment-1293892818,https://api.github.com/repos/simonw/datasette/issues/1866,1293892818,IC_kwDOBm6k_c5NHzzS,9599,simonw,2022-10-27T18:12:02Z,2022-10-27T18:12:02Z,OWNER,"There's one catch with batched inserts: if your CLI tool fails half way through you could end up with a partially populated table - since a bunch of batches will have succeeded first.
I think that's OK. In the future I may want to come up with a way to run multiple batches of inserts inside a single transaction, but I can ignore that for the first release of this feature.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1426001541,API for bulk inserting records into a table,
https://github.com/simonw/datasette/issues/1866#issuecomment-1293891876,https://api.github.com/repos/simonw/datasette/issues/1866,1293891876,IC_kwDOBm6k_c5NHzkk,9599,simonw,2022-10-27T18:11:05Z,2022-10-27T18:11:05Z,OWNER,Likewise for newline-delimited JSON. While it's tempting to want to accept that as an ingest format (because it's nice to generate and stream) I think it's better to have a client application that can turn a stream of newline-delimited JSON into batched JSON inserts.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1426001541,API for bulk inserting records into a table,
https://github.com/simonw/datasette/issues/1866#issuecomment-1293891191,https://api.github.com/repos/simonw/datasette/issues/1866,1293891191,IC_kwDOBm6k_c5NHzZ3,9599,simonw,2022-10-27T18:10:22Z,2022-10-27T18:10:22Z,OWNER,"So for the moment I'm just going to concentrate on the JSON API. I can consider CSV variants later on, or as plugins, or both.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1426001541,API for bulk inserting records into a table,
https://github.com/simonw/datasette/issues/1866#issuecomment-1293890684,https://api.github.com/repos/simonw/datasette/issues/1866,1293890684,IC_kwDOBm6k_c5NHzR8,9599,simonw,2022-10-27T18:09:52Z,2022-10-27T18:09:52Z,OWNER,"Should this API accept CSV/TSV etc in addition to JSON?
I'm torn on this one. My initial instinct is that it should not - and there should instead be a Datasette client library / CLI tool you can use that knows how to turn CSV into batches of JSON calls for when you want to upload a CSV file.
I don't think the usability of `curl https://datasette/db/table -F 'data=@path/to/file.csv' -H 'Authentication: Bearer xxx'` is particularly great compared to something like`datasette client insert https://datasette/ db table file.csv --csv` (where the command version could store API tokens for you too).","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1426001541,API for bulk inserting records into a table,
https://github.com/simonw/datasette/issues/1866#issuecomment-1293887808,https://api.github.com/repos/simonw/datasette/issues/1866,1293887808,IC_kwDOBm6k_c5NHylA,9599,simonw,2022-10-27T18:07:02Z,2022-10-27T18:07:02Z,OWNER,"Error handling is really important here.
What should happen if you submit 100 records and one of them has some kind of validation error? How should that error be reported back to you?
I'm inclined to say that it defaults to all-or-nothing in a transaction - but there should be a `""continue_on_error"": true` option (or similar) which causes it to insert the ones that are valid while reporting back the ones that are invalid.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1426001541,API for bulk inserting records into a table,
https://github.com/simonw/datasette/issues/1860#issuecomment-1293863145,https://api.github.com/repos/simonw/datasette/issues/1860,1293863145,IC_kwDOBm6k_c5NHsjp,562352,CharlesNepote,2022-10-27T17:43:37Z,2022-10-27T17:43:37Z,NONE,"Sorry I forgot the `-- comments like that`.
I'm afraid there is an issue in your regexp, see: https://regex101.com/r/pyubJf/1
I guess I can fix it.
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1424378012,SQL query field can't begin by a comment,
https://github.com/simonw/datasette/issues/1862#issuecomment-1293857306,https://api.github.com/repos/simonw/datasette/issues/1862,1293857306,IC_kwDOBm6k_c5NHrIa,9599,simonw,2022-10-27T17:38:17Z,2022-10-27T17:38:17Z,OWNER,"Strongly related to:
- #1866","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1425011030,"Create a new table from one or more records, `sqlite-utils` style",
https://github.com/simonw/datasette/issues/1865#issuecomment-1293568194,https://api.github.com/repos/simonw/datasette/issues/1865,1293568194,IC_kwDOBm6k_c5NGkjC,9599,simonw,2022-10-27T13:58:26Z,2022-10-27T13:58:26Z,OWNER,"Here's the issue where I started doing this:
- #849","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1425682079,Stop syncing main to master,
https://github.com/simonw/datasette/issues/849#issuecomment-649908756,https://api.github.com/repos/simonw/datasette/issues/849,649908756,MDEyOklzc3VlQ29tbWVudDY0OTkwODc1Ng==,9599,simonw,2020-06-26T02:09:09Z,2022-10-27T13:57:08Z,OWNER,"I mentioned this issue here: https://simonwillison.net/2020/Jun/26/weeknotes-plugins-sqlite-generate/
Repositories created by following the README in https://github.com/simonw/datasette-template and https://github.com/simonw/click-app have a `main` branch instead of `master` so I have a few examples live now. https://github.com/simonw/datasette-saved-queries is one example.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",639072811,Rename master branch to main,
https://github.com/simonw/datasette/issues/1851#issuecomment-1292999579,https://api.github.com/repos/simonw/datasette/issues/1851,1292999579,IC_kwDOBm6k_c5NEZub,9599,simonw,2022-10-27T04:59:06Z,2022-10-27T04:59:12Z,OWNER,"I should probably refactor this to use `sqlite-utils`, since I'm going to want to use that later for the feature that automatically creates tables.
Might make it easier to solve the rowid issues too.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1421544654,API to insert a single record into an existing table,
https://github.com/simonw/datasette/issues/1851#issuecomment-1292996181,https://api.github.com/repos/simonw/datasette/issues/1851,1292996181,IC_kwDOBm6k_c5NEY5V,9599,simonw,2022-10-27T04:51:47Z,2022-10-27T04:51:47Z,OWNER,Also need a test for invalid JSON (currently triggers a 500 HTML error).,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1421544654,API to insert a single record into an existing table,
https://github.com/simonw/datasette/issues/1855#issuecomment-1292962813,https://api.github.com/repos/simonw/datasette/issues/1855,1292962813,IC_kwDOBm6k_c5NEQv9,9599,simonw,2022-10-27T04:31:40Z,2022-10-27T04:31:40Z,OWNER,"My hunch on this is that anyone with that level of complex permissions requirements needs to be using a custom authentication plugin which includes much more concrete token rules, rather than the default signed stateless token implementation that ships with Datasette core.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1423336089,`datasette create-token` ability to create tokens with a reduced set of permissions,
https://github.com/simonw/datasette/issues/1855#issuecomment-1292959886,https://api.github.com/repos/simonw/datasette/issues/1855,1292959886,IC_kwDOBm6k_c5NEQCO,9599,simonw,2022-10-27T04:30:07Z,2022-10-27T04:30:07Z,OWNER,"Here's an interesting edge-case to consider: what if a user creates themselves a token for a specific table, then deletes that table, and waits for another user to create a table of the same name... and then uses their previously created token to write to the table that someone else created?
Not sure if this is a threat I need to actively consider, but it's worth thinking a little bit about the implications of such a thing - since there will be APIs that allow users to create tables, and there may be cases where people want to have a concept of users ""owning"" specific tables.
This is probably something that could be left for plugins to solve, but it still needs to be understood and potentially documented.
There may even be a world in which tracking the timestamp at which a table was created becomes useful - because that could then be baked into API tokens, such that a token created BEFORE the table was created does not grant access to that table.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1423336089,`datasette create-token` ability to create tokens with a reduced set of permissions,
https://github.com/simonw/datasette/issues/1851#issuecomment-1292952121,https://api.github.com/repos/simonw/datasette/issues/1851,1292952121,IC_kwDOBm6k_c5NEOI5,9599,simonw,2022-10-27T04:24:09Z,2022-10-27T04:24:20Z,OWNER,"And come up with a whole bunch of tests for weird table shapes, surprising column names, different types etc.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1421544654,API to insert a single record into an existing table,
https://github.com/simonw/datasette/issues/1851#issuecomment-1292951833,https://api.github.com/repos/simonw/datasette/issues/1851,1292951833,IC_kwDOBm6k_c5NEOEZ,9599,simonw,2022-10-27T04:23:40Z,2022-10-27T04:23:40Z,OWNER,Also need to think about transactions - it should use them!,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1421544654,API to insert a single record into an existing table,
https://github.com/simonw/datasette/issues/1851#issuecomment-1292939146,https://api.github.com/repos/simonw/datasette/issues/1851,1292939146,IC_kwDOBm6k_c5NEK-K,9599,simonw,2022-10-27T04:00:17Z,2022-10-27T04:23:15Z,OWNER,"Documentation for this first draft of the API: https://docs.datasette.io/en/1.0-dev/json_api.html#inserting-a-single-row
It currently returns errors as HTML - it needs to return errors as JSON. Also the errors need comprehensive test coverage.
I'm also worried about what happens if you use it on a table that doesn't use an integer primary key - need to check that. I think this code may break:
https://github.com/simonw/datasette/blob/51c436fed29205721dcf17fa31d7e7090d34ebb8/datasette/views/table.py#L155-L171
Plus will `rowid` tables without an explicit primary key return the `rowid` column? They should.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1421544654,API to insert a single record into an existing table,
https://github.com/simonw/datasette/issues/1850#issuecomment-1292940011,https://api.github.com/repos/simonw/datasette/issues/1850,1292940011,IC_kwDOBm6k_c5NELLr,9599,simonw,2022-10-27T04:01:59Z,2022-10-27T04:01:59Z,OWNER,"Working on that first ""insert row"" implementation:
- https://github.com/simonw/datasette/issues/1851
Has made it very clear to me that I should go the whole hog and build the basic form-based interface for this as well.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1421529723,Write API in Datasette core,