{"html_url": "https://github.com/simonw/datasette/issues/1851#issuecomment-1294224185", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1851", "id": 1294224185, "node_id": "IC_kwDOBm6k_c5NJEs5", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-10-27T23:18:24Z", "updated_at": "2022-11-03T23:26:05Z", "author_association": "OWNER", "body": "So new API design is:\r\n\r\n```\r\nPOST /db/table/-/insert\r\nAuthorization: Bearer xxx\r\nContent-Type: application/json\r\n{\r\n \"row\": {\r\n \"id\": 1,\r\n \"name\": \"New record\"\r\n }\r\n}\r\n```\r\nReturns:\r\n```\r\n201 Created\r\n{\r\n \"row\": [{\r\n \"id\": 1,\r\n \"name\": \"New record\"\r\n }]\r\n}\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1421544654, "label": "API to insert a single record into an existing table"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1851#issuecomment-1294281451", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1851", "id": 1294281451, "node_id": "IC_kwDOBm6k_c5NJSrr", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-10-28T00:59:25Z", "updated_at": "2022-10-28T00:59:25Z", "author_association": "OWNER", "body": "I'm going to use this endpoint for bulk inserts too, so I'm closing this issue and continuing the work here:\r\n- #1866", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1421544654, "label": "API to insert a single record into an existing table"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1851#issuecomment-1289712350", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1851", "id": 1289712350, "node_id": "IC_kwDOBm6k_c5M33Le", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-10-24T22:28:39Z", "updated_at": "2022-10-27T23:18:48Z", "author_association": "OWNER", "body": "API design: (**UPDATE: this was [later changed to POST /db/table/-/insert](https://github.com/simonw/datasette/issues/1851#issuecomment-1294224185))\r\n\r\n```\r\nPOST /db/table\r\nAuthorization: Bearer xxx\r\nContent-Type: application/json\r\n{\r\n \"row\": {\r\n \"id\": 1,\r\n \"name\": \"New record\"\r\n }\r\n}\r\n```\r\nReturns:\r\n```\r\n201 Created\r\n{\r\n \"row\": {\r\n \"id\": 1,\r\n \"name\": \"New record\"\r\n }\r\n}\r\n```\r\nYou can omit optional fields in the input, including the ID field. The returned object will always include all fields - and will even include `rowid` if your object doesn't have a primary key of its own.\r\n\r\nI decided to use `\"row\"` as the key in both request and response, to preserve space for other future keys - one that tells you that the table has been created, for example.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1421544654, "label": "API to insert a single record into an existing table"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1851#issuecomment-1294012583", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1851", "id": 1294012583, "node_id": "IC_kwDOBm6k_c5NIRCn", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-10-27T20:11:22Z", "updated_at": "2022-10-27T20:11:22Z", "author_association": "OWNER", "body": "And the response to `\"inserted\": [{...}]` - it will be the same for bulk inserts.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1421544654, "label": "API to insert a single record into an existing table"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1851#issuecomment-1294012084", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1851", "id": 1294012084, "node_id": "IC_kwDOBm6k_c5NIQ60", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-10-27T20:10:47Z", "updated_at": "2022-10-27T20:10:47Z", "author_association": "OWNER", "body": "I'm going to change the incoming JSON back to `{\"row\": {...}}` - no need to POST `{\"insert\": ...}` to something with `/-/insert` in the URL already.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1421544654, "label": "API to insert a single record into an existing table"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1851#issuecomment-1294009354", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1851", "id": 1294009354, "node_id": "IC_kwDOBm6k_c5NIQQK", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-10-27T20:07:42Z", "updated_at": "2022-10-27T20:07:42Z", "author_association": "OWNER", "body": "Need to implement the new URL design from:\r\n- #1868\r\n\r\nThis is now going to be `/db/table/-/insert` - and it will eventually handle bulk inserts as well as single inserts.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1421544654, "label": "API to insert a single record into an existing table"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1851#issuecomment-1293996735", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1851", "id": 1293996735, "node_id": "IC_kwDOBm6k_c5NINK_", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-10-27T19:54:53Z", "updated_at": "2022-10-27T19:54:53Z", "author_association": "OWNER", "body": "Updated docs: https://docs.datasette.io/en/1.0-dev/json_api.html#inserting-a-single-row\r\n\r\n\"image\"\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1421544654, "label": "API to insert a single record into an existing table"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1851#issuecomment-1292997608", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1851", "id": 1292997608, "node_id": "IC_kwDOBm6k_c5NEZPo", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-10-27T04:54:53Z", "updated_at": "2022-10-27T19:05:50Z", "author_association": "OWNER", "body": "I'm going to change the design of this to:\r\n```\r\n{\r\n \"insert\": {\r\n \"title\" :\"...\"\r\n }\r\n}\r\n```\r\nRenaming `\"row\"` to `\"insert\"`.\r\n\r\nThis will be consistent with adding `\"drop\": true` for dropping a table, and maybe other verbs like for modifying the schema.\r\n\r\nThe API response will look like this:\r\n\r\n```json\r\n{\r\n \"inserted_row\": {\r\n \"id\": 1,\r\n \"title\": \"...\"\r\n }\r\n}\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1421544654, "label": "API to insert a single record into an existing table"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1851#issuecomment-1292999579", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1851", "id": 1292999579, "node_id": "IC_kwDOBm6k_c5NEZub", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-10-27T04:59:06Z", "updated_at": "2022-10-27T04:59:12Z", "author_association": "OWNER", "body": "I should probably refactor this to use `sqlite-utils`, since I'm going to want to use that later for the feature that automatically creates tables.\r\n\r\nMight make it easier to solve the rowid issues too.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1421544654, "label": "API to insert a single record into an existing table"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1851#issuecomment-1292996181", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1851", "id": 1292996181, "node_id": "IC_kwDOBm6k_c5NEY5V", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-10-27T04:51:47Z", "updated_at": "2022-10-27T04:51:47Z", "author_association": "OWNER", "body": "Also need a test for invalid JSON (currently triggers a 500 HTML error).", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1421544654, "label": "API to insert a single record into an existing table"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1851#issuecomment-1292952121", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1851", "id": 1292952121, "node_id": "IC_kwDOBm6k_c5NEOI5", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-10-27T04:24:09Z", "updated_at": "2022-10-27T04:24:20Z", "author_association": "OWNER", "body": "And come up with a whole bunch of tests for weird table shapes, surprising column names, different types etc.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1421544654, "label": "API to insert a single record into an existing table"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1851#issuecomment-1292951833", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1851", "id": 1292951833, "node_id": "IC_kwDOBm6k_c5NEOEZ", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-10-27T04:23:40Z", "updated_at": "2022-10-27T04:23:40Z", "author_association": "OWNER", "body": "Also need to think about transactions - it should use them!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1421544654, "label": "API to insert a single record into an existing table"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1851#issuecomment-1292939146", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1851", "id": 1292939146, "node_id": "IC_kwDOBm6k_c5NEK-K", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-10-27T04:00:17Z", "updated_at": "2022-10-27T04:23:15Z", "author_association": "OWNER", "body": "Documentation for this first draft of the API: https://docs.datasette.io/en/1.0-dev/json_api.html#inserting-a-single-row\r\n\r\nIt currently returns errors as HTML - it needs to return errors as JSON. Also the errors need comprehensive test coverage.\r\n\r\nI'm also worried about what happens if you use it on a table that doesn't use an integer primary key - need to check that. I think this code may break:\r\n\r\nhttps://github.com/simonw/datasette/blob/51c436fed29205721dcf17fa31d7e7090d34ebb8/datasette/views/table.py#L155-L171\r\n\r\nPlus will `rowid` tables without an explicit primary key return the `rowid` column? They should.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1421544654, "label": "API to insert a single record into an existing table"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1851#issuecomment-1292592210", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1851", "id": 1292592210, "node_id": "IC_kwDOBm6k_c5NC2RS", "user": {"value": 25778, "label": "eyeseast"}, "created_at": "2022-10-26T20:03:46Z", "updated_at": "2022-10-26T20:03:46Z", "author_association": "CONTRIBUTOR", "body": "Yeah, every time I see something cool done with triggers, I remember that I need to start using triggers.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1421544654, "label": "API to insert a single record into an existing table"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1851#issuecomment-1292544296", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1851", "id": 1292544296, "node_id": "IC_kwDOBm6k_c5NCqko", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-10-26T19:33:34Z", "updated_at": "2022-10-26T19:33:34Z", "author_association": "OWNER", "body": "That trigger solution is pretty neat!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1421544654, "label": "API to insert a single record into an existing table"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1851#issuecomment-1292519956", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1851", "id": 1292519956, "node_id": "IC_kwDOBm6k_c5NCkoU", "user": {"value": 15178711, "label": "asg017"}, "created_at": "2022-10-26T19:20:33Z", "updated_at": "2022-10-26T19:20:33Z", "author_association": "CONTRIBUTOR", "body": "> This could use a new plugin hook, too. I don't want to complicate your life too much, but for things like GIS, I'd want a way to turn regular JSON into SpatiaLite geometries or combine X/Y coordinates into point geometries and such. Happy to help however I can.\r\n\r\n @eyeseast Maybe you could do this with triggers? Like you can insert JSON-friendly data into a \"raw\" table, and create a trigger that transforms that inserted data into the proper table\r\n\r\nHere's an example:\r\n\r\n```sql\r\n-- meant to be updated from a Datasette insert\r\ncreate table points_raw(longitude int, latitude int);\r\n\r\n-- the target table with proper spatliate geometries\r\ncreate table points(point geometry);\r\n\r\nCREATE TRIGGER insert_points_raw INSERT ON points_raw \r\n BEGIN\r\n insert into points(point) values (makepoint(new.longitude, new.latitude))\r\n END;\r\n```\r\n\r\nYou could then POST a new row to `points_raw` like this:\r\n```\r\nPOST /db/points_raw\r\nAuthorization: Bearer xxx\r\nContent-Type: application/json\r\n{\r\n \"row\": {\r\n \"longitude\": 27.64356,\r\n \"latitude\": -47.29384\r\n }\r\n}\r\n```\r\n\r\nThen SQLite with run the trigger and insert a new row in `points` with the correct geometry point. Downside is you'd have duplicated data with `points_raw`, but maybe it could be a `TEMP` table (or have a cron that deletes all rows from that table every so often?)", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1421544654, "label": "API to insert a single record into an existing table"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1851#issuecomment-1291228502", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1851", "id": 1291228502, "node_id": "IC_kwDOBm6k_c5M9pVW", "user": {"value": 25778, "label": "eyeseast"}, "created_at": "2022-10-25T23:02:10Z", "updated_at": "2022-10-25T23:02:10Z", "author_association": "CONTRIBUTOR", "body": "That's reasonable. Canned queries and custom endpoints are certainly going to give more room for specific needs. ", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1421544654, "label": "API to insert a single record into an existing table"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1851#issuecomment-1291226367", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1851", "id": 1291226367, "node_id": "IC_kwDOBm6k_c5M9oz_", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-10-25T22:58:30Z", "updated_at": "2022-10-25T22:58:30Z", "author_association": "OWNER", "body": "The `datasette insert` concept included plugin support, with the idea of being able to support things like SpatiaLite files:\r\n- #1160\r\n\r\nI think this API mechanism is going to be a bit less exciting than that - it will be low-level for inserting rows, and if you want to do something fancier you can use a canned query that feeds incoming GeoJSON to a SpatiaLite function instead.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1421544654, "label": "API to insert a single record into an existing table"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1851#issuecomment-1290615599", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1851", "id": 1290615599, "node_id": "IC_kwDOBm6k_c5M7Tsv", "user": {"value": 25778, "label": "eyeseast"}, "created_at": "2022-10-25T14:05:12Z", "updated_at": "2022-10-25T14:05:12Z", "author_association": "CONTRIBUTOR", "body": "This could use a new plugin hook, too. I don't want to complicate your life too much, but for things like GIS, I'd want a way to turn regular JSON into SpatiaLite geometries or combine X/Y coordinates into point geometries and such. Happy to help however I can.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1421544654, "label": "API to insert a single record into an existing table"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1851#issuecomment-1289865317", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1851", "id": 1289865317, "node_id": "IC_kwDOBm6k_c5M4chl", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-10-25T01:42:47Z", "updated_at": "2022-10-25T01:42:47Z", "author_association": "OWNER", "body": "This is going to tie into Datasette's existing permissions mechanism, so plugins will be able to define their own custom mechanisms for tokens to be attached to a specific identity: https://docs.datasette.io/en/stable/authentication.html\r\n\r\nThere's only one plugin for API tokens at the moment, which is this one: https://datasette.io/plugins/datasette-auth-tokens\r\n\r\nI'm actually planning on adding another, default token mechanism to Datasette itself as part of this work:\r\n\r\n- #1852\r\n\r\nIt may well be that `datasette-sandstorm-support` needs to add something custom here too.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1421544654, "label": "API to insert a single record into an existing table"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1851#issuecomment-1289752130", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1851", "id": 1289752130, "node_id": "IC_kwDOBm6k_c5M4A5C", "user": {"value": 4399499, "label": "ocdtrekkie"}, "created_at": "2022-10-24T23:07:30Z", "updated_at": "2022-10-24T23:07:30Z", "author_association": "NONE", "body": "How are you tying the bearer token to identity? I'm excited to see this feature, and since Sandstorm controls API access using the same header, it also will transparently support the API documentation here, but we strip the bearer before the request reaches the app (replacing it with our existing auth headers, of course).", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1421544654, "label": "API to insert a single record into an existing table"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1851#issuecomment-1289713513", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1851", "id": 1289713513, "node_id": "IC_kwDOBm6k_c5M33dp", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-10-24T22:29:58Z", "updated_at": "2022-10-24T22:30:15Z", "author_association": "OWNER", "body": "Interesting open question: how should validation errors (if any) be returned?\r\n\r\nThe two forms of validation I can think of at first are:\r\n\r\n- Missing keys which are marked as `not null` in the schema\r\n- Keys that do not match to existing columns (if you didn't pass `\"alter\": true`, an option I am considering)", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1421544654, "label": "API to insert a single record into an existing table"}, "performed_via_github_app": null}