{"html_url": "https://github.com/simonw/datasette/issues/1605#issuecomment-1332310772", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1605", "id": 1332310772, "node_id": "IC_kwDOBm6k_c5PaXL0", "user": {"value": 25778, "label": "eyeseast"}, "created_at": "2022-11-30T15:06:37Z", "updated_at": "2022-11-30T15:06:37Z", "author_association": "CONTRIBUTOR", "body": "I'll add issues for both and do a documentation PR.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1108671952, "label": "Scripted exports"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1605#issuecomment-1331187551", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1605", "id": 1331187551, "node_id": "IC_kwDOBm6k_c5PWE9f", "user": {"value": 25778, "label": "eyeseast"}, "created_at": "2022-11-29T19:29:42Z", "updated_at": "2022-11-29T19:29:42Z", "author_association": "CONTRIBUTOR", "body": "Interesting. I started a version using metadata like I outlined up top, but I realized that there's no documented way for a plugin to access either metadata or canned queries. Or at least, I couldn't find a way.\r\n\r\nThere is this method: https://github.com/simonw/datasette/blob/main/datasette/app.py#L472 but I don't want to rely on it if it's not documented. Same with this: https://github.com/simonw/datasette/blob/main/datasette/app.py#L544\r\n\r\nIf those are safe, I'll build on them. I'm also happy to document them, if that greases the wheels.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1108671952, "label": "Scripted exports"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1605#issuecomment-1018778667", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1605", "id": 1018778667, "node_id": "IC_kwDOBm6k_c48uVQr", "user": {"value": 25778, "label": "eyeseast"}, "created_at": "2022-01-21T19:00:01Z", "updated_at": "2022-01-21T19:00:01Z", "author_association": "CONTRIBUTOR", "body": "Let me know if you want help prototyping any of this, because I'm thinking about it and trying stuff out. Happy to be a sounding board, if it helps.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1108671952, "label": "Scripted exports"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1605#issuecomment-1018741262", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1605", "id": 1018741262, "node_id": "IC_kwDOBm6k_c48uMIO", "user": {"value": 25778, "label": "eyeseast"}, "created_at": "2022-01-21T18:05:09Z", "updated_at": "2022-01-21T18:05:09Z", "author_association": "CONTRIBUTOR", "body": "Thinking about this more, as well as #1356 and various other tickets related to output formats, I think there's a missing plugin hook for formatting results, separate from `register_output_renderer` (or maybe part of it, depending on #1101). \r\n\r\nRight now, as I understand it, getting output in any format goes through the normal view stack -- a table, a row or a query -- and so by the time `register_output_renderer` gets it, the results have already been truncated or paginated. What I'd want, I think, is to be able to register ways to format results independent of where those results are sent.\r\n\r\nIt's possible this could be done using [`conn.row_factory`](https://docs.python.org/3/library/sqlite3.html#sqlite3.Connection.row_factory) (maybe in the `prepare_connection` hook), but I'm not sure that's where it belongs.\r\n\r\nAnother option is some kind of registry of serializers, which `register_output_renderer` and other plugin hooks could use. What I'm trying to avoid here is writing a plugin that also needs plugins for formats I haven't thought of yet.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1108671952, "label": "Scripted exports"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1605#issuecomment-1016994329", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1605", "id": 1016994329, "node_id": "IC_kwDOBm6k_c48nhoZ", "user": {"value": 25778, "label": "eyeseast"}, "created_at": "2022-01-20T00:27:17Z", "updated_at": "2022-01-20T00:27:17Z", "author_association": "CONTRIBUTOR", "body": "Right now, I usually have a line in a Makefile like this:\r\n\r\n```make\r\ncombined.geojson: project.db\r\n    pipenv run datasette project.db --get /project/combined.geojson \\\r\n     --load-extension spatialite \\\r\n     --setting sql_time_limit_ms 5000 \\\r\n     --setting max_returned_rows 20000 \\\r\n     -m metadata.yml > $@\r\n```\r\n\r\nThat all assumes I've loaded whatever I need into `project.db` and created a canned query called `combined` (and then uses `datasette-geojson` for geojson output). \r\n\r\nIt works, but as you can see, it's a lot to manage, a lot of boilerplate, and it wasn't obvious how to get there. If there's an error in the canned query, I get an HTML error page, so that's hard to debug. And it's only one query, so each output needs a line like this. Make isn't ideal, either, for that reason.\r\n\r\nThe thing I really liked with `datafreeze` was doing templated filenames. I have a project now where I need to export a bunch of litttle geojson files, based on queries, and it would be awesome to be able to do something like this:\r\n\r\n```yml\r\ndatabases:\r\n  project:\r\n    queries:\r\n      boundaries:\r\n        sql: \"SELECT * FROM boundaries\"\r\n        filename: \"boundaries/{id}.geojson\"\r\n        mode: \"item\"\r\n        format: geojson\r\n```\r\n\r\nAnd then do:\r\n\r\n```sh\r\ndatasette freeze -m metadata.yml project.db\r\n```\r\n\r\nFor HTML export, maybe there's a `template` argument, or `format: template` or something. And that gets you a static site generator, kinda for free.\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1108671952, "label": "Scripted exports"}, "performed_via_github_app": null}