{"id": 775666296, "node_id": "MDU6SXNzdWU3NzU2NjYyOTY=", "number": 1160, "title": "\"datasette insert\" command and plugin hook", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 23, "created_at": "2020-12-29T02:37:03Z", "updated_at": "2021-06-17T18:12:32Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "Tools for loading data into Datasette currently mostly exist as separate utilities - `yaml-to-sqlite` and `csvs-to-sqlite` and suchlike.\r\n\r\nBringing these into Datasette could have some interesting properties:\r\n\r\n- A `datasette insert` command could be extended with plugins to handle more formats\r\n- Any format that can be inserted on the command-line could also be inserted using a web UI or web API - which would benefit from new format plugin hooks\r\n- If Datasette ever grows beyond SQLite (see #670) a built-in import mechanism could work for those other databases as well - without me needing to write `yaml-to-postgresql` and suchlike", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1160/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 1857234285, "node_id": "I_kwDOBm6k_c5usyVt", "number": 2145, "title": "If a row has a primary key of `null` various things break", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 23, "created_at": "2023-08-18T20:06:28Z", "updated_at": "2023-08-21T17:30:01Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "Stumbled across this while experimenting with `datasette-write-ui`. The error I got was a 500 on the `/db` page:\r\n\r\n> `'NoneType' object has no attribute 'encode'`\r\n\r\nTracked it down to this code, which assembles the URL for a row page:\r\n\r\nhttps://github.com/simonw/datasette/blob/943df09dcca93c3b9861b8c96277a01320db8662/datasette/utils/__init__.py#L120-L134\r\n\r\nThat's because `tilde_encode` can't handle `None`: https://github.com/simonw/datasette/blob/943df09dcca93c3b9861b8c96277a01320db8662/datasette/utils/__init__.py#L1175-L1178\r\n\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/2145/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 944846776, "node_id": "MDU6SXNzdWU5NDQ4NDY3NzY=", "number": 297, "title": "Option for importing CSV data using the SQLite .import mechanism", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 23, "created_at": "2021-07-14T22:36:41Z", "updated_at": "2023-09-22T20:49:52Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "As seen in https://til.simonwillison.net/sqlite/import-csv - `.mode csv` and then `.import school.csv schools` is hugely faster than importing via `sqlite-utils insert` and doing the work in Python - but it can only be implemented by shelling out to the `sqlite3` CLI tool, it's not functionality that is exposed to the Python `sqlite3` module.\r\n\r\nAn option to use this would be useful - maybe something like this:\r\n\r\n    sqlite-utils insert blah.db blah blah.csv --fast", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/297/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 459882902, "node_id": "MDU6SXNzdWU0NTk4ODI5MDI=", "number": 526, "title": "Stream all results for arbitrary SQL and canned queries", "user": {"value": 50578294, "label": "matej-fr"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 23, "created_at": "2019-06-24T13:09:45Z", "updated_at": "2022-09-28T04:01:25Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "I think that there is a difficulty with canned queries.\r\n\r\nWhen I want to stream all results of a canned query TwoDays I get only first 1.000 records.\r\n\r\nExample:\r\n`http://myserver/history_sample/two_days.csv?_stream=on`\r\n\r\nreturns only first 1.000 records.\r\n\r\nIf I do the same with the whole database i.e.\r\n`http://myserver/history_sample/database.csv?_stream=on`\r\n\r\nI get correctly all records.\r\n\r\nAny ideas?", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/526/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}