issue_comments
74 rows where user = 25778 sorted by updated_at descending
This data as json, CSV (advanced)
user 1
- eyeseast · 49 ✖
id | html_url | issue_url | node_id | user | created_at | updated_at ▲ | author_association | body | reactions | issue | performed_via_github_app |
---|---|---|---|---|---|---|---|---|---|---|---|
1105642187 | https://github.com/simonw/datasette/issues/1101#issuecomment-1105642187 | https://api.github.com/repos/simonw/datasette/issues/1101 | IC_kwDOBm6k_c5B5sLL | eyeseast 25778 | 2022-04-21T18:59:08Z | 2022-04-21T18:59:08Z | CONTRIBUTOR | Ha! That was your idea (and a good one). But it's probably worth measuring to see what overhead it adds. It did require both passing in the database and making the whole thing Just timing the queries themselves:
Looking at the network panel:
I'm not sure how best to time the GeoJSON generation, but it would be interesting to check. Maybe I'll write a plugin to add query times to response headers. The other thing to consider with async streaming is that it might be well-suited for a slower response. When I have to get the whole result and send a response in a fixed amount of time, I need the most efficient query possible. If I can hang onto a connection and get things one chunk at a time, maybe it's ok if there's some overhead. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
register_output_renderer() should support streaming data 749283032 | |
1105588651 | https://github.com/simonw/datasette/issues/1101#issuecomment-1105588651 | https://api.github.com/repos/simonw/datasette/issues/1101 | IC_kwDOBm6k_c5B5fGr | eyeseast 25778 | 2022-04-21T18:15:39Z | 2022-04-21T18:15:39Z | CONTRIBUTOR | What if you split rendering and streaming into two things:
That way current plugins still work, and streaming is purely additive. A |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
register_output_renderer() should support streaming data 749283032 | |
1099540225 | https://github.com/simonw/datasette/issues/1713#issuecomment-1099540225 | https://api.github.com/repos/simonw/datasette/issues/1713 | IC_kwDOBm6k_c5BiacB | eyeseast 25778 | 2022-04-14T19:09:57Z | 2022-04-14T19:09:57Z | CONTRIBUTOR | I wonder if this overlaps with what I outlined in #1605. You could run something like this:
And maybe that does what you need. Of course, that plugin isn't built yet. But that's the idea. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Datasette feature for publishing snapshots of query results 1203943272 | |
1094453751 | https://github.com/simonw/datasette/issues/1699#issuecomment-1094453751 | https://api.github.com/repos/simonw/datasette/issues/1699 | IC_kwDOBm6k_c5BPAn3 | eyeseast 25778 | 2022-04-11T01:32:12Z | 2022-04-11T01:32:12Z | CONTRIBUTOR | Was looking through old issues and realized a bunch of this got discussed in #1101 (including by me!), so sorry to rehash all this. Happy to help with whatever piece of it I can. Would be very excited to be able to use format plugins with exports. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Proposal: datasette query 1193090967 | |
1092386254 | https://github.com/simonw/datasette/issues/1699#issuecomment-1092386254 | https://api.github.com/repos/simonw/datasette/issues/1699 | IC_kwDOBm6k_c5BHH3O | eyeseast 25778 | 2022-04-08T02:39:25Z | 2022-04-08T02:39:25Z | CONTRIBUTOR | And just to think this through a little more, here's what
Alternately, that could be an async generator, like this:
Not sure which makes more sense, but I think this pattern would open up a lot of possibility. If you had your stream_indented_json function, you could do |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Proposal: datasette query 1193090967 | |
1092370880 | https://github.com/simonw/datasette/issues/1699#issuecomment-1092370880 | https://api.github.com/repos/simonw/datasette/issues/1699 | IC_kwDOBm6k_c5BHEHA | eyeseast 25778 | 2022-04-08T02:07:40Z | 2022-04-08T02:07:40Z | CONTRIBUTOR | So maybe
And stream gets an iterator, instead of a list of rows, so it can efficiently handle large queries. Maybe it also gets passed a destination stream, or it returns an iterator. I'm not sure what makes more sense. Either way, that might cover both CLI exports and streaming responses. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Proposal: datasette query 1193090967 | |
1092357672 | https://github.com/simonw/datasette/issues/1699#issuecomment-1092357672 | https://api.github.com/repos/simonw/datasette/issues/1699 | IC_kwDOBm6k_c5BHA4o | eyeseast 25778 | 2022-04-08T01:39:40Z | 2022-04-08T01:39:40Z | CONTRIBUTOR |
That's my thinking, too. It's really the thing I've been wanting since writing
I think this probably needs either a new plugin hook separate from |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Proposal: datasette query 1193090967 | |
1077671779 | https://github.com/simonw/sqlite-utils/issues/399#issuecomment-1077671779 | https://api.github.com/repos/simonw/sqlite-utils/issues/399 | IC_kwDOCGYnMM5AO_dj | eyeseast 25778 | 2022-03-24T14:11:33Z | 2022-03-24T14:11:43Z | CONTRIBUTOR | Coming back to this. I was about to add a utility function to datasette-geojson to convert lat/lng columns to geometries. Thankfully I googled first. There's a SpatiaLite function for this: MakePoint.
I'm not sure if that would work with |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Make it easier to insert geometries, with documentation and maybe code 1124731464 | |
1067981656 | https://github.com/simonw/sqlite-utils/issues/131#issuecomment-1067981656 | https://api.github.com/repos/simonw/sqlite-utils/issues/131 | IC_kwDOCGYnMM4_qBtY | eyeseast 25778 | 2022-03-15T13:21:42Z | 2022-03-15T13:21:42Z | CONTRIBUTOR | Just ran into this issue last night. I have a big table that's mostly numbers, but also a zip code column in a state where ZIP codes start with 0. Would be great to run something like this:
Maybe I'll take a crack at this one. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
sqlite-utils insert: options for column types 675753042 | |
1065477258 | https://github.com/simonw/sqlite-utils/issues/411#issuecomment-1065477258 | https://api.github.com/repos/simonw/sqlite-utils/issues/411 | IC_kwDOCGYnMM4_geSK | eyeseast 25778 | 2022-03-11T20:14:59Z | 2022-03-11T20:14:59Z | CONTRIBUTOR | Good call on adding this to |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Support for generated columns 1160034488 | |
1059647114 | https://github.com/simonw/sqlite-utils/issues/412#issuecomment-1059647114 | https://api.github.com/repos/simonw/sqlite-utils/issues/412 | IC_kwDOCGYnMM4_KO6K | eyeseast 25778 | 2022-03-05T01:54:24Z | 2022-03-05T01:54:24Z | CONTRIBUTOR | I haven't tried this, but it looks like Pandas has a method for this: https://pandas.pydata.org/docs/reference/api/pandas.read_sql_query.html |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Optional Pandas integration 1160182768 | |
1040998433 | https://github.com/simonw/sqlite-utils/pull/407#issuecomment-1040998433 | https://api.github.com/repos/simonw/sqlite-utils/issues/407 | IC_kwDOCGYnMM4-DGAh | eyeseast 25778 | 2022-02-16T01:29:39Z | 2022-02-16T01:29:39Z | CONTRIBUTOR | Happy to do it and have it in the library. Going to use it a bunch. This whole SpatiaLite toolchain become a huge part of my work in the past year. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Add SpatiaLite helpers to CLI 1138948786 | |
1040580250 | https://github.com/simonw/sqlite-utils/pull/407#issuecomment-1040580250 | https://api.github.com/repos/simonw/sqlite-utils/issues/407 | IC_kwDOCGYnMM4-Bf6a | eyeseast 25778 | 2022-02-15T17:40:00Z | 2022-02-15T17:40:00Z | CONTRIBUTOR | @simonw I think this is ready for a look. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Add SpatiaLite helpers to CLI 1138948786 | |
1038336591 | https://github.com/simonw/sqlite-utils/issues/398#issuecomment-1038336591 | https://api.github.com/repos/simonw/sqlite-utils/issues/398 | IC_kwDOCGYnMM4948JP | eyeseast 25778 | 2022-02-13T18:48:21Z | 2022-02-13T18:49:49Z | CONTRIBUTOR | Been chipping away at this between other things and realized I think it's probably worth keeping a SpatiaLite flag on |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Add SpatiaLite helpers to CLI 1124237013 | |
1035057014 | https://github.com/simonw/sqlite-utils/issues/402#issuecomment-1035057014 | https://api.github.com/repos/simonw/sqlite-utils/issues/402 | IC_kwDOCGYnMM49sbd2 | eyeseast 25778 | 2022-02-10T15:30:28Z | 2022-02-10T15:30:40Z | CONTRIBUTOR | Yeah, the CLI experience is probably where any kind of multi-column, configured setup is going to fall apart. Sticking with GIS examples, one way I might think about this is using the fiona CLI: ```sh assuming a database is already created and has SpatiaLitefio cat boundary.shp | sqlite-utils insert boundaries --conversion geometry GeometryGeoJSON - ``` Anyway, very interested to see where you land here. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Advanced class-based `conversions=` mechanism 1125297737 | |
1032732242 | https://github.com/simonw/sqlite-utils/issues/402#issuecomment-1032732242 | https://api.github.com/repos/simonw/sqlite-utils/issues/402 | IC_kwDOCGYnMM49jj5S | eyeseast 25778 | 2022-02-08T15:26:59Z | 2022-02-08T15:26:59Z | CONTRIBUTOR | What if you did something like this: ```python class Conversion: def init(self, args, *kwargs): "Put whatever settings you need here"
``` This way, you're always passing an instance, which has methods that do the conversion. (Or you're passing a SQL string, as you would now.) The You'd then use it like this: ```python subclass might be unneeded here, if methods are presentclass LngLatConversion(Conversion): def init(self, x="longitude", y="latitude"): self.x = x self.y = y
table.insert_all(rows, conversions={"point": LngLatConversion("lng", "lat"))} ``` I haven't thought through all the implementation details here, and it'll probably break in ways I haven't foreseen, but wanted to get this idea out of my head. Hope it helps. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Advanced class-based `conversions=` mechanism 1125297737 | |
1031791783 | https://github.com/simonw/sqlite-utils/issues/402#issuecomment-1031791783 | https://api.github.com/repos/simonw/sqlite-utils/issues/402 | IC_kwDOCGYnMM49f-Sn | eyeseast 25778 | 2022-02-07T18:37:40Z | 2022-02-07T18:37:40Z | CONTRIBUTOR | I've never used it either, but it's interesting, right? Feel like I should try it for something. I'm trying to get my head around how this conversions feature might work, because I really like the idea of it. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Advanced class-based `conversions=` mechanism 1125297737 | |
1031779460 | https://github.com/simonw/sqlite-utils/issues/402#issuecomment-1031779460 | https://api.github.com/repos/simonw/sqlite-utils/issues/402 | IC_kwDOCGYnMM49f7SE | eyeseast 25778 | 2022-02-07T18:24:56Z | 2022-02-07T18:24:56Z | CONTRIBUTOR | I wonder if there's any overlap with the goals here and the I'm not sure that's exactly what we're talking about here, but it might be a parallel with some useful ideas to borrow. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Advanced class-based `conversions=` mechanism 1125297737 | |
1030741289 | https://github.com/simonw/sqlite-utils/issues/399#issuecomment-1030741289 | https://api.github.com/repos/simonw/sqlite-utils/issues/399 | IC_kwDOCGYnMM49b90p | eyeseast 25778 | 2022-02-06T03:03:43Z | 2022-02-06T03:03:43Z | CONTRIBUTOR |
Off the top of my head:
Some of this is easy enough with SQL functions, some is easier in Python. Maybe that's where having pre-built classes gets really handy, because it saves you from thinking about which way it's implemented. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Make it easier to insert geometries, with documentation and maybe code 1124731464 | |
1030740826 | https://github.com/simonw/sqlite-utils/issues/399#issuecomment-1030740826 | https://api.github.com/repos/simonw/sqlite-utils/issues/399 | IC_kwDOCGYnMM49b9ta | eyeseast 25778 | 2022-02-06T02:59:10Z | 2022-02-06T02:59:10Z | CONTRIBUTOR | All this said, I don't think it's unreasonable to point people to dedicated tools like |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Make it easier to insert geometries, with documentation and maybe code 1124731464 | |
1030740653 | https://github.com/simonw/sqlite-utils/issues/399#issuecomment-1030740653 | https://api.github.com/repos/simonw/sqlite-utils/issues/399 | IC_kwDOCGYnMM49b9qt | eyeseast 25778 | 2022-02-06T02:57:17Z | 2022-02-06T02:57:17Z | CONTRIBUTOR | I like the idea of having stock conversions you could import. I'd actually move them to a dedicated module (call it ```python from sqlite_utils.conversions import LongitudeLatitude db["places"].insert( { "name": "London", "lng": -0.118092, "lat": 51.509865, }, conversions={"point": LongitudeLatitude("lng", "lat")}, ) ``` I would definitely use that for every CSV I get with lat/lng columns where I actually need GeoJSON. |
{ "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Make it easier to insert geometries, with documentation and maybe code 1124731464 | |
1030629879 | https://github.com/simonw/sqlite-utils/issues/398#issuecomment-1030629879 | https://api.github.com/repos/simonw/sqlite-utils/issues/398 | IC_kwDOCGYnMM49bin3 | eyeseast 25778 | 2022-02-05T13:57:33Z | 2022-02-05T19:49:38Z | CONTRIBUTOR | I'm mostly using geojson-to-sqlite at the moment. Even with shapefiles, I'm usually converting to GeoJSON and projecting to EPSG:4326 (with ogr2ogr) first. I think an open question here is how much you want to leave to external libraries and how much you want here. My thinking has been that adding Spatialite helpers here would make external stuff easier, but it would be nice to have some standard way to insert geometries. I'm in the middle of adding GeoJSON and Spatialite support to geocode-sqlite, and that will probably use WKT. Since that's all points, I think I can just make the string inline. But for polygons, I'd generally use Shapely, which probably isn't a dependency you want to add to sqlite-utils. I've also been trying to get some of the approaches here to work, but haven't had any success so far. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Add SpatiaLite helpers to CLI 1124237013 | |
1030002502 | https://github.com/simonw/sqlite-utils/pull/385#issuecomment-1030002502 | https://api.github.com/repos/simonw/sqlite-utils/issues/385 | IC_kwDOCGYnMM49ZJdG | eyeseast 25778 | 2022-02-04T13:50:19Z | 2022-02-04T13:50:19Z | CONTRIBUTOR | Awesome. Thanks for your help getting it in. Will now look at adding CLI versions of this. It's going to be super helpful on a bunch of my projects. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Add new spatialite helper methods 1102899312 | |
1029370537 | https://github.com/simonw/sqlite-utils/pull/385#issuecomment-1029370537 | https://api.github.com/repos/simonw/sqlite-utils/issues/385 | IC_kwDOCGYnMM49WvKp | eyeseast 25778 | 2022-02-03T20:25:58Z | 2022-02-03T20:25:58Z | CONTRIBUTOR | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Add new spatialite helper methods 1102899312 | ||
1029338360 | https://github.com/simonw/sqlite-utils/pull/385#issuecomment-1029338360 | https://api.github.com/repos/simonw/sqlite-utils/issues/385 | IC_kwDOCGYnMM49WnT4 | eyeseast 25778 | 2022-02-03T19:43:56Z | 2022-02-03T19:43:56Z | CONTRIBUTOR | Works for me. I was just looking at how the FTS extensions work and they're just methods, too. So this can be consistent with that. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Add new spatialite helper methods 1102899312 | |
1029326568 | https://github.com/simonw/sqlite-utils/pull/385#issuecomment-1029326568 | https://api.github.com/repos/simonw/sqlite-utils/issues/385 | IC_kwDOCGYnMM49Wkbo | eyeseast 25778 | 2022-02-03T19:28:26Z | 2022-02-03T19:28:26Z | CONTRIBUTOR |
This is fixed now. I had to take out the type annotations for |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Add new spatialite helper methods 1102899312 | |
1029317527 | https://github.com/simonw/sqlite-utils/issues/79#issuecomment-1029317527 | https://api.github.com/repos/simonw/sqlite-utils/issues/79 | IC_kwDOCGYnMM49WiOX | eyeseast 25778 | 2022-02-03T19:18:02Z | 2022-02-03T19:18:02Z | CONTRIBUTOR | Taking part of the conversation from #385 here.
Yes. And also |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Helper methods for working with SpatiaLite 557842245 | |
1029306428 | https://github.com/simonw/sqlite-utils/pull/385#issuecomment-1029306428 | https://api.github.com/repos/simonw/sqlite-utils/issues/385 | IC_kwDOCGYnMM49Wfg8 | eyeseast 25778 | 2022-02-03T19:03:43Z | 2022-02-03T19:03:43Z | CONTRIBUTOR | I thought about adding these as methods on I do sort of like having all the Spatialite stuff in its own module, just because it's built around an extension you might not have or want, but I don't know if that's a good reason to have a different API. You could have |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Add new spatialite helper methods 1102899312 | |
1029180984 | https://github.com/simonw/sqlite-utils/pull/385#issuecomment-1029180984 | https://api.github.com/repos/simonw/sqlite-utils/issues/385 | IC_kwDOCGYnMM49WA44 | eyeseast 25778 | 2022-02-03T16:42:04Z | 2022-02-03T16:42:04Z | CONTRIBUTOR | Fixed my spelling. That's a useful thing. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Add new spatialite helper methods 1102899312 | |
1029175907 | https://github.com/simonw/sqlite-utils/pull/385#issuecomment-1029175907 | https://api.github.com/repos/simonw/sqlite-utils/issues/385 | IC_kwDOCGYnMM49V_pj | eyeseast 25778 | 2022-02-03T16:36:54Z | 2022-02-03T16:36:54Z | CONTRIBUTOR | @simonw Not sure if you've seen this, but any chance you can run the tests? |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Add new spatialite helper methods 1102899312 | |
1018778667 | https://github.com/simonw/datasette/issues/1605#issuecomment-1018778667 | https://api.github.com/repos/simonw/datasette/issues/1605 | IC_kwDOBm6k_c48uVQr | eyeseast 25778 | 2022-01-21T19:00:01Z | 2022-01-21T19:00:01Z | CONTRIBUTOR | Let me know if you want help prototyping any of this, because I'm thinking about it and trying stuff out. Happy to be a sounding board, if it helps. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Scripted exports 1108671952 | |
1018741262 | https://github.com/simonw/datasette/issues/1605#issuecomment-1018741262 | https://api.github.com/repos/simonw/datasette/issues/1605 | IC_kwDOBm6k_c48uMIO | eyeseast 25778 | 2022-01-21T18:05:09Z | 2022-01-21T18:05:09Z | CONTRIBUTOR | Thinking about this more, as well as #1356 and various other tickets related to output formats, I think there's a missing plugin hook for formatting results, separate from Right now, as I understand it, getting output in any format goes through the normal view stack -- a table, a row or a query -- and so by the time It's possible this could be done using Another option is some kind of registry of serializers, which |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Scripted exports 1108671952 | |
1016994329 | https://github.com/simonw/datasette/issues/1605#issuecomment-1016994329 | https://api.github.com/repos/simonw/datasette/issues/1605 | IC_kwDOBm6k_c48nhoZ | eyeseast 25778 | 2022-01-20T00:27:17Z | 2022-01-20T00:27:17Z | CONTRIBUTOR | Right now, I usually have a line in a Makefile like this:
That all assumes I've loaded whatever I need into It works, but as you can see, it's a lot to manage, a lot of boilerplate, and it wasn't obvious how to get there. If there's an error in the canned query, I get an HTML error page, so that's hard to debug. And it's only one query, so each output needs a line like this. Make isn't ideal, either, for that reason. The thing I really liked with
And then do:
For HTML export, maybe there's a |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Scripted exports 1108671952 | |
1016651485 | https://github.com/simonw/datasette/issues/1601#issuecomment-1016651485 | https://api.github.com/repos/simonw/datasette/issues/1601 | IC_kwDOBm6k_c48mN7d | eyeseast 25778 | 2022-01-19T16:39:03Z | 2022-01-19T16:39:03Z | CONTRIBUTOR | I think both of these are Spatialite specific. They get generated when you first initialize the extension. KNN is actually deprecated in favor of KNN2, as I understand it. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Add KNN and data_licenses to hidden tables list 1105916061 | |
1013698557 | https://github.com/simonw/sqlite-utils/issues/79#issuecomment-1013698557 | https://api.github.com/repos/simonw/sqlite-utils/issues/79 | IC_kwDOCGYnMM48a8_9 | eyeseast 25778 | 2022-01-15T15:15:22Z | 2022-01-15T15:15:22Z | CONTRIBUTOR | @simonw I have a PR here https://github.com/simonw/sqlite-utils/pull/385 that adds Spatialite helpers on the Python side. Please let me know how it looks. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Helper methods for working with SpatiaLite 557842245 | |
1012413729 | https://github.com/simonw/sqlite-utils/issues/79#issuecomment-1012413729 | https://api.github.com/repos/simonw/sqlite-utils/issues/79 | IC_kwDOCGYnMM48WDUh | eyeseast 25778 | 2022-01-13T18:50:00Z | 2022-01-13T18:50:00Z | CONTRIBUTOR | One more thing I'm going to add: A method to add a geometry column, which I'll need to do to create a spatial index on a table. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Helper methods for working with SpatiaLite 557842245 | |
1012253198 | https://github.com/simonw/sqlite-utils/issues/79#issuecomment-1012253198 | https://api.github.com/repos/simonw/sqlite-utils/issues/79 | IC_kwDOCGYnMM48VcIO | eyeseast 25778 | 2022-01-13T15:39:14Z | 2022-01-13T15:39:14Z | CONTRIBUTOR | Other thing: If there get to be enough utils, I think it's worth moving all the spatialite stuff into its own file ( |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Helper methods for working with SpatiaLite 557842245 | |
1012230212 | https://github.com/simonw/sqlite-utils/issues/79#issuecomment-1012230212 | https://api.github.com/repos/simonw/sqlite-utils/issues/79 | IC_kwDOCGYnMM48VWhE | eyeseast 25778 | 2022-01-13T15:15:13Z | 2022-01-13T15:15:13Z | CONTRIBUTOR | Some proposals I'd add to sqlite-utils: Some version of this, from geojson-to-sqlite:
Also a function for creating a spatial index:
I don't know the nuances of updating a spatial index, or checking if one already exists. This could be a CLI method like:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Helper methods for working with SpatiaLite 557842245 | |
1012158895 | https://github.com/simonw/sqlite-utils/issues/79#issuecomment-1012158895 | https://api.github.com/repos/simonw/sqlite-utils/issues/79 | IC_kwDOCGYnMM48VFGv | eyeseast 25778 | 2022-01-13T13:55:59Z | 2022-01-13T13:55:59Z | CONTRIBUTOR | Came here to add this. I might pick it up. Would also add a utility to create (and update and delete?) a spatial index. It's not much code but I have to look it up every time. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Helper methods for working with SpatiaLite 557842245 | |
983155079 | https://github.com/simonw/sqlite-utils/issues/348#issuecomment-983155079 | https://api.github.com/repos/simonw/sqlite-utils/issues/348 | IC_kwDOCGYnMM46mcGH | eyeseast 25778 | 2021-12-01T00:28:40Z | 2021-12-01T00:28:40Z | CONTRIBUTOR | I'd use this. Right now, I tend to do |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Command for creating an empty database 1067771698 | |
953911245 | https://github.com/simonw/sqlite-utils/issues/242#issuecomment-953911245 | https://api.github.com/repos/simonw/sqlite-utils/issues/242 | IC_kwDOCGYnMM4424fN | eyeseast 25778 | 2021-10-28T14:37:55Z | 2021-10-28T14:37:55Z | CONTRIBUTOR | I've been thinking about this a bit lately, doing a project that involves moving a lot of data in and out of SQLite files, datasette and GeoJSON. This has me leaning toward the idea that something like I know there's a lot of overlap in sqlite-utils and datasette, and maybe keeping sqlite-utils synchronous would let datasette be entirely async and give a cleaner separation of implementations. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Async support 817989436 | |
869191854 | https://github.com/simonw/datasette/issues/1101#issuecomment-869191854 | https://api.github.com/repos/simonw/datasette/issues/1101 | MDEyOklzc3VlQ29tbWVudDg2OTE5MTg1NA== | eyeseast 25778 | 2021-06-27T16:42:14Z | 2021-06-27T16:42:14Z | CONTRIBUTOR | This would really help with this issue: https://github.com/eyeseast/datasette-geojson/issues/7 |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
register_output_renderer() should support streaming data 749283032 | |
861944202 | https://github.com/simonw/sqlite-utils/issues/272#issuecomment-861944202 | https://api.github.com/repos/simonw/sqlite-utils/issues/272 | MDEyOklzc3VlQ29tbWVudDg2MTk0NDIwMg== | eyeseast 25778 | 2021-06-16T01:41:03Z | 2021-06-16T01:41:03Z | CONTRIBUTOR | So, I do things like this a lot, too. I like the idea of piping in from stdin. Something like this would be nice to do in a makefile:
If you assumed that you're always piping out the same format you're piping in, the option names don't have to change. Depends how much you want to change formats. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Idea: import CSV to memory, run SQL, export in a single command 921878733 | |
857298526 | https://github.com/simonw/datasette/pull/1370#issuecomment-857298526 | https://api.github.com/repos/simonw/datasette/issues/1370 | MDEyOklzc3VlQ29tbWVudDg1NzI5ODUyNg== | eyeseast 25778 | 2021-06-09T01:18:59Z | 2021-06-09T01:18:59Z | CONTRIBUTOR | I'm happy to grab some or all of these in this PR, if you want. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Ensure db.path is a string before trying to insert into internal database 914130834 | |
853895159 | https://github.com/simonw/datasette/issues/1356#issuecomment-853895159 | https://api.github.com/repos/simonw/datasette/issues/1356 | MDEyOklzc3VlQ29tbWVudDg1Mzg5NTE1OQ== | eyeseast 25778 | 2021-06-03T14:03:59Z | 2021-06-03T14:03:59Z | CONTRIBUTOR | (Putting thoughts here to keep the conversation in one place.) I think using datasette for this use-case is the right approach. I usually have both datasette and sqlite-utils installed in the same project, and that's where I'm trying out queries, so it probably makes the most sense to have datasette also manage the output (and maybe the input, too). It seems like both ```sh run an arbitrary querydatasette query covid.db "select * from ny_times_us_counties limit 1" --format yaml run a canned querydatasette get covid.db some-canned-query --format yaml ``` |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Research: syntactic sugar for using --get with SQL queries, maybe "datasette query" 910092577 | |
787121933 | https://github.com/simonw/sqlite-utils/issues/242#issuecomment-787121933 | https://api.github.com/repos/simonw/sqlite-utils/issues/242 | MDEyOklzc3VlQ29tbWVudDc4NzEyMTkzMw== | eyeseast 25778 | 2021-02-27T19:18:57Z | 2021-02-27T19:18:57Z | CONTRIBUTOR | I think HTTPX gets it exactly right, with a clear separation between sync and async clients, each with a basically identical API. (I'm about to switch feed-to-sqlite over to it, from Requests, to eventually make way for async support.) |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Async support 817989436 | |
618758326 | https://github.com/simonw/datasette/issues/731#issuecomment-618758326 | https://api.github.com/repos/simonw/datasette/issues/731 | MDEyOklzc3VlQ29tbWVudDYxODc1ODMyNg== | eyeseast 25778 | 2020-04-24T01:55:00Z | 2020-04-24T01:55:00Z | CONTRIBUTOR | Mounting |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Option to automatically configure based on directory layout 605110015 | |
618126449 | https://github.com/simonw/datasette/issues/731#issuecomment-618126449 | https://api.github.com/repos/simonw/datasette/issues/731 | MDEyOklzc3VlQ29tbWVudDYxODEyNjQ0OQ== | eyeseast 25778 | 2020-04-23T01:38:55Z | 2020-04-23T01:38:55Z | CONTRIBUTOR | I've almost suggested this same thing a couple times. I tend to have Makefile (because I'm doing other |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Option to automatically configure based on directory layout 605110015 | |
489105665 | https://github.com/simonw/datasette/pull/434#issuecomment-489105665 | https://api.github.com/repos/simonw/datasette/issues/434 | MDEyOklzc3VlQ29tbWVudDQ4OTEwNTY2NQ== | eyeseast 25778 | 2019-05-03T14:01:30Z | 2019-05-03T14:01:30Z | CONTRIBUTOR | This is exactly what I needed. Thank you. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
"datasette publish cloudrun" command to publish to Google Cloud Run 434321685 |
Advanced export
JSON shape: default, array, newline-delimited, object
CREATE TABLE [issue_comments] ( [html_url] TEXT, [issue_url] TEXT, [id] INTEGER PRIMARY KEY, [node_id] TEXT, [user] INTEGER REFERENCES [users]([id]), [created_at] TEXT, [updated_at] TEXT, [author_association] TEXT, [body] TEXT, [reactions] TEXT, [issue] INTEGER REFERENCES [issues]([id]) , [performed_via_github_app] TEXT); CREATE INDEX [idx_issue_comments_issue] ON [issue_comments] ([issue]); CREATE INDEX [idx_issue_comments_user] ON [issue_comments] ([user]);
issue 21