github
html_url | issue_url | id | node_id | user | created_at | updated_at | author_association | body | reactions | issue | performed_via_github_app |
---|---|---|---|---|---|---|---|---|---|---|---|
https://github.com/simonw/datasette/issues/1699#issuecomment-1094453751 | https://api.github.com/repos/simonw/datasette/issues/1699 | 1094453751 | IC_kwDOBm6k_c5BPAn3 | 25778 | 2022-04-11T01:32:12Z | 2022-04-11T01:32:12Z | CONTRIBUTOR | Was looking through old issues and realized a bunch of this got discussed in #1101 (including by me!), so sorry to rehash all this. Happy to help with whatever piece of it I can. Would be very excited to be able to use format plugins with exports. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1193090967 | |
https://github.com/simonw/datasette/issues/1699#issuecomment-1092386254 | https://api.github.com/repos/simonw/datasette/issues/1699 | 1092386254 | IC_kwDOBm6k_c5BHH3O | 25778 | 2022-04-08T02:39:25Z | 2022-04-08T02:39:25Z | CONTRIBUTOR | And just to think this through a little more, here's what `stream_geojson` might look like: ```python async def stream_geojson(datasette, columns, rows, database, stream): db = datasette.get_database(database) for row in rows: feature = await row_to_geojson(row, db) stream.write(feature + "\n") # just assuming newline mode for now ``` Alternately, that could be an async generator, like this: ```python async def stream_geojson(datasette, columns, rows, database): db = datasette.get_database(database) for row in rows: feature = await row_to_geojson(row, db) yield feature ``` Not sure which makes more sense, but I think this pattern would open up a lot of possibility. If you had your [stream_indented_json](https://til.simonwillison.net/python/output-json-array-streaming) function, you could do `yield from stream_indented_json(rows, 2)` and be one your way. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1193090967 | |
https://github.com/simonw/datasette/issues/1699#issuecomment-1092370880 | https://api.github.com/repos/simonw/datasette/issues/1699 | 1092370880 | IC_kwDOBm6k_c5BHEHA | 25778 | 2022-04-08T02:07:40Z | 2022-04-08T02:07:40Z | CONTRIBUTOR | So maybe `render_output_render` returns something like this: ```python @hookimpl def register_output_renderer(datasette): return { "extension": "geojson", "render": render_geojson, "stream": stream_geojson, "can_render": can_render_geojson, } ``` And stream gets an iterator, instead of a list of rows, so it can efficiently handle large queries. Maybe it also gets passed a destination stream, or it returns an iterator. I'm not sure what makes more sense. Either way, that might cover both CLI exports and streaming responses. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1193090967 | |
https://github.com/simonw/datasette/issues/1699#issuecomment-1092357672 | https://api.github.com/repos/simonw/datasette/issues/1699 | 1092357672 | IC_kwDOBm6k_c5BHA4o | 25778 | 2022-04-08T01:39:40Z | 2022-04-08T01:39:40Z | CONTRIBUTOR | > My best thought on how to differentiate them so far is plugins: if Datasette plugins that provide alternative outputs - like .geojson and .yml and suchlike - also work for the datasette query command that would make a lot of sense to me. That's my thinking, too. It's really the thing I've been wanting since writing `datasette-geojson`, since I'm always exporting with `datasette --get`. The workflow I'm always looking for is something like this: ```sh cd alltheplaces-datasette datasette query dunkin_in_suffolk -f geojson -o dunkin_in_suffolk.geojson ``` I think this probably needs either a new plugin hook separate from `register_output_renderer` or a way to use that without going through the HTTP stack. Or maybe a render mode that writes to a stream instead of a response. Maybe there's a new key in the dictionary that `register_output_renderer` returns that handles CLI exports. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1193090967 |