{"html_url": "https://github.com/simonw/datasette/issues/1882#issuecomment-1314813205", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1882", "id": 1314813205, "node_id": "IC_kwDOBm6k_c5OXnUV", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-15T06:00:41Z", "updated_at": "2022-11-15T06:00:41Z", "author_association": "OWNER", "body": "Documentation:\r\n\r\n- https://docs.datasette.io/en/1.0-dev/json_api.html#creating-a-table\r\n- https://docs.datasette.io/en/1.0-dev/json_api.html#creating-a-table-from-example-data\r\n\r\nWrote a TIL about how I wrote some of those tests with Copilot: https://til.simonwillison.net/gpt3/writing-test-with-copilot", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1435294468, "label": "`/db/-/create` API for creating tables"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1882#issuecomment-1312582512", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1882", "id": 1312582512, "node_id": "IC_kwDOBm6k_c5OPGtw", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-12T22:11:18Z", "updated_at": "2022-11-12T22:11:18Z", "author_association": "OWNER", "body": "I like this:\r\n\r\n<img width=\"1080\" alt=\"image\" src=\"https://user-images.githubusercontent.com/9599/201496508-2fe102e8-f0de-4dd8-87c5-40b6aae97413.png\">\r\n\r\n```json\r\n{\r\n  \"ok\": true,\r\n  \"database\": \"data\",\r\n  \"table\": \"agai2n\",\r\n  \"table_url\": \"http://127.0.0.1:8001/data/agai2n\",\r\n  \"schema\": \"CREATE TABLE [agai2n] (\\n   [hello] INTEGER\\n)\"\r\n}\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1435294468, "label": "`/db/-/create` API for creating tables"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1882#issuecomment-1312581121", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1882", "id": 1312581121, "node_id": "IC_kwDOBm6k_c5OPGYB", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-12T22:01:32Z", "updated_at": "2022-11-12T22:01:32Z", "author_association": "OWNER", "body": "I'm going to change it to `table` in the output AND the input.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1435294468, "label": "`/db/-/create` API for creating tables"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1882#issuecomment-1312581008", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1882", "id": 1312581008, "node_id": "IC_kwDOBm6k_c5OPGWQ", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-12T22:00:52Z", "updated_at": "2022-11-12T22:00:52Z", "author_association": "OWNER", "body": "Tried out my prototype in the API explorer:\r\n\r\n<img width=\"1229\" alt=\"image\" src=\"https://user-images.githubusercontent.com/9599/201496194-a6ebcefe-2be8-47d7-8f15-8c6345cba7cf.png\">\r\n\r\nThe `\"name\"` on the output is bothering me a bit - should it be `table_name` or `table` instead?\r\n\r\nProblem is I really like `name` for the input, so should it be consistent to have the same name on the output here, or should I aim for consistency with other endpoints that might return `table` rather than the ambiguous `name` elsewhere?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1435294468, "label": "`/db/-/create` API for creating tables"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1882#issuecomment-1312580348", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1882", "id": 1312580348, "node_id": "IC_kwDOBm6k_c5OPGL8", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-12T21:55:54Z", "updated_at": "2022-11-12T21:56:45Z", "author_association": "OWNER", "body": "What should this API return?\r\n\r\nI think the name of the table (`name`), the URL to that table (`table_url` - for consistency with how faceting API works already) and the schema of the table (`schema`).", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1435294468, "label": "`/db/-/create` API for creating tables"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1882#issuecomment-1312575048", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1882", "id": 1312575048, "node_id": "IC_kwDOBm6k_c5OPE5I", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-12T21:22:58Z", "updated_at": "2022-11-12T21:22:58Z", "author_association": "OWNER", "body": "Need to validate the table name. SQLite supports almost any table name - but they can't contain a newline character and cannot start with `sqlite_` - according to https://stackoverflow.com/questions/3694276/what-are-valid-table-names-in-sqlite/43049720#43049720", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1435294468, "label": "`/db/-/create` API for creating tables"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1882#issuecomment-1312556044", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1882", "id": 1312556044, "node_id": "IC_kwDOBm6k_c5OPAQM", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-12T19:29:11Z", "updated_at": "2022-11-12T19:29:11Z", "author_association": "OWNER", "body": "Thought of an edge-case: with `sqlite-utils` one feature I really like is that I can pipe data into it without caring if the table already exists or not:\r\n\r\n    cat data.json | sqlite-utils insert my.db mytable -\r\n\r\nHow could this new API support that?\r\n\r\nI thought about adding a `\"create\": true` option to `/db/table/-/insert` which creates the table if it doesn't already exist, but if I do that I'll need to start adding other options to that endpoint - to set the primary key, add foreign keys and suchlike - which would be ignored except for the cases where the table was being created from scratch.\r\n\r\nThis doesn't feel right to me - I want to keep those options here, on `/db/-/create`.\r\n\r\nOne idea I had was to implement it such that you can call `/db/-/create` multiple times for the same table, but only if you are using the `\"rows\"` option. If so, and if the rows can be safely inserted, it would let you do that.\r\n\r\nBut instead, I'm going to outsource this to the CLI tool I plan to write that feeds data into this API. I'm already planning to use that tool for CSV inserts (so the API doesn't need to accept CSV directly). I think it's a good place for other usability enhancements like \"insert this, creating the table if it does not exist\" as well.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1435294468, "label": "`/db/-/create` API for creating tables"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1882#issuecomment-1302818784", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1882", "id": 1302818784, "node_id": "IC_kwDOBm6k_c5Np2_g", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-04T00:25:18Z", "updated_at": "2022-11-04T16:12:39Z", "author_association": "OWNER", "body": "On that basis I think the core API design should change to this:\r\n```\r\nPOST /db/-/create\r\nAuthorization: Bearer xxx\r\nContent-Type: application/json\r\n{\r\n    \"name\": \"my new table\",\r\n    \"columns\": [\r\n        {\r\n            \"name\": \"id\",\r\n            \"type\": \"integer\"\r\n        },\r\n        {\r\n            \"name\": \"title\",\r\n            \"type\": \"text\"\r\n        }\r\n    ]\r\n   \"pk\": \"id\"\r\n}\r\n```\r\nThis leaves room for a `\"rows\": []` key at the root too. Having that as a child of `\"table\"` felt unintuitive to me, and I didn't like the way this looked either:\r\n\r\n```json\r\n{\r\n  \"table\": {\r\n    \"name\": \"my_new_table\"\r\n  },\r\n  \"rows\": [\r\n    {\"id\": 1, \"title\": \"Title\"}\r\n  ]\r\n}\r\n```\r\nWeird to have the table `name` nested inside `table` when `rows` wasn't.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1435294468, "label": "`/db/-/create` API for creating tables"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1882#issuecomment-1302818153", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1882", "id": 1302818153, "node_id": "IC_kwDOBm6k_c5Np21p", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-04T00:23:58Z", "updated_at": "2022-11-04T00:23:58Z", "author_association": "OWNER", "body": "I made a decision here that this endpoint should also accept an optional `\"rows\": [...]` list which is used to automatically create the table using a schema derived from those example rows (which then get inserted):\r\n\r\n- https://github.com/simonw/datasette/issues/1862#issuecomment-1302817807", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1435294468, "label": "`/db/-/create` API for creating tables"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1882#issuecomment-1302716350", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1882", "id": 1302716350, "node_id": "IC_kwDOBm6k_c5Npd--", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-03T21:51:14Z", "updated_at": "2022-11-03T22:35:54Z", "author_association": "OWNER", "body": "Validating this JSON object is getting a tiny bit complex. I'm tempted to adopt https://pydantic-docs.helpmanual.io/ at this point.\r\n\r\nThe `create_model` example on https://stackoverflow.com/questions/66168517/generate-dynamic-model-using-pydantic/66168682#66168682 is particularly relevant, especially when I work on this issue:\r\n\r\n- #1863\r\n\r\n```python\r\nfrom pydantic import create_model\r\n\r\nd = {\"strategy\": {\"name\": \"test_strat2\", \"periods\": 10}}\r\n\r\nStrategy = create_model(\"Strategy\", **d[\"strategy\"])\r\n\r\nprint(Strategy.schema_json(indent=2))\r\n```\r\n`create_model()`: https://pydantic-docs.helpmanual.io/usage/models/#dynamic-model-creation", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1435294468, "label": "`/db/-/create` API for creating tables"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1882#issuecomment-1302721916", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1882", "id": 1302721916, "node_id": "IC_kwDOBm6k_c5NpfV8", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-03T21:58:50Z", "updated_at": "2022-11-03T21:59:17Z", "author_association": "OWNER", "body": "Mocked up a quick HTML+JavaScript form for creating that JSON structure using some iteration against Copilot prompts:\r\n```html\r\n<pre>\r\n/* JSON format:\r\n{\r\n  \"table\": {\r\n      \"name\": \"my new table\",\r\n      \"columns\": [\r\n          {\r\n              \"name\": \"id\",\r\n              \"type\": \"integer\"\r\n          },\r\n          {\r\n              \"name\": \"title\",\r\n              \"type\": \"text\"\r\n          }\r\n      ]\r\n     \"pk\": \"id\"\r\n  }\r\n}\r\n\r\nHTML form with Javascript for creating this JSON:\r\n*/</pre>\r\n<form id=\"create-table-form\">\r\n  <label for=\"table-name\">Table name</label>\r\n  <input type=\"text\" id=\"table-name\" name=\"table-name\" required><br>\r\n  <label for=\"table-pk\">Primary key</label>\r\n  <input type=\"text\" id=\"table-pk\" name=\"table-pk\" required><br>\r\n  <label for=\"column-name\">Column name</label>\r\n  <input type=\"text\" id=\"column-name\" name=\"column-name\" required>\r\n  <label for=\"column-type\">Column type</label>\r\n  <input type=\"text\" id=\"column-type\" name=\"column-type\" required>\r\n  <button type=\"button\" id=\"add-column\">Add column</button>\r\n  <p>Current columns:</p>\r\n  <ul id=\"columns\"></ul>\r\n  <button type=\"button\" id=\"create-table\">Create table</button>\r\n</form>\r\n<script>\r\nvar form = document.getElementById('create-table-form');\r\nvar tableName = document.getElementById('table-name');\r\nvar tablePk = document.getElementById('table-pk');\r\nvar columnName = document.getElementById('column-name');\r\nvar columnType = document.getElementById('column-type');\r\nvar addColumn = document.getElementById('add-column');\r\nvar createTable = document.getElementById('create-table');\r\nvar columns = [];\r\naddColumn.addEventListener('click', () => {\r\n  columns.push({\r\n    name: columnName.value,\r\n    type: columnType.value\r\n  });\r\n  var li = document.createElement('li');\r\n  li.textContent = columnName.value + ' (' + columnType.value + ')';\r\n  // Add a delete button to each column\r\n  var deleteButton = document.createElement('button');\r\n  deleteButton.textContent = 'Delete';\r\n  deleteButton.addEventListener('click', () => {\r\n    columns.splice(columns.indexOf(li), 1);\r\n    li.remove();\r\n  });\r\n  li.appendChild(deleteButton);\r\n  document.getElementById('columns').appendChild(li);\r\n});\r\ncreateTable.addEventListener('click', () => {\r\n  var table = {\r\n    name: tableName.value,\r\n    pk: tablePk.value,\r\n    columns: columns\r\n  };\r\n  console.log(JSON.stringify(table, null, 2));\r\n});\r\n</script>\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1435294468, "label": "`/db/-/create` API for creating tables"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1882#issuecomment-1302715662", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1882", "id": 1302715662, "node_id": "IC_kwDOBm6k_c5Npd0O", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-11-03T21:50:27Z", "updated_at": "2022-11-03T21:50:27Z", "author_association": "OWNER", "body": "API design for this:\r\n```\r\nPOST /db/-/create\r\nAuthorization: Bearer xxx\r\nContent-Type: application/json\r\n{\r\n    \"table\": {\r\n        \"name\": \"my new table\",\r\n        \"columns\": [\r\n            {\r\n                \"name\": \"id\",\r\n                \"type\": \"integer\"\r\n            },\r\n            {\r\n                \"name\": \"title\",\r\n                \"type\": \"text\"\r\n            }\r\n        ]\r\n       \"pk\": \"id\"\r\n    }\r\n}\r\n```\r\nSupported column types are:\r\n\r\n- `integer`\r\n- `text`\r\n- `float` (even though SQLite calls it a \"real\")\r\n- `blob`\r\n\r\nThis matches my design for `sqlite-utils`: https://sqlite-utils.datasette.io/en/stable/cli.html#cli-create-table", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1435294468, "label": "`/db/-/create` API for creating tables"}, "performed_via_github_app": null}