issue_comments

39 rows where user = 45057 sorted by updated_at descending

View and edit SQL

Suggested facets: issue_url, created_at (date), updated_at (date)

user

  • russss · 39

author_association

id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions issue
504690927 https://github.com/simonw/datasette/issues/514#issuecomment-504690927 https://api.github.com/repos/simonw/datasette/issues/514 MDEyOklzc3VlQ29tbWVudDUwNDY5MDkyNw== russss 45057 2019-06-22T19:06:07Z 2019-06-22T19:06:07Z CONTRIBUTOR

I'd rather not turn this into a systemd support thread, but you're trying to execute the package directory there. Your datasette executable is probably at /home/chris/Env/datasette/bin/datasette.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Documentation with recommendations on running Datasette in production without using Docker 459397625
504684831 https://github.com/simonw/datasette/issues/514#issuecomment-504684831 https://api.github.com/repos/simonw/datasette/issues/514 MDEyOklzc3VlQ29tbWVudDUwNDY4NDgzMQ== russss 45057 2019-06-22T17:38:23Z 2019-06-22T17:38:23Z CONTRIBUTOR

WorkingDirectory=/path/to/data

@russss, Which directory does this represent?

It's the working directory (cwd) of the spawned process. In this case if you set it to the directory your data is in, you can use relative paths to the db (and metadata/templates/etc) in the ExecStart command.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Documentation with recommendations on running Datasette in production without using Docker 459397625
504663766 https://github.com/simonw/datasette/issues/514#issuecomment-504663766 https://api.github.com/repos/simonw/datasette/issues/514 MDEyOklzc3VlQ29tbWVudDUwNDY2Mzc2Ng== russss 45057 2019-06-22T12:57:59Z 2019-06-22T12:57:59Z CONTRIBUTOR

This example is useful to - I like how it has a Makefile that knows how to set up systemd: https://github.com/pikesley/Queube

I wasn't even aware it was possible to add a systemd service at an arbitrary path, but it seems a little messy to me.

Maybe worth noting that systemd does support per-user services which don't require root access. Cool but probably overkill for most people (especially when you're going to need root to listen on port 80 anyway, directly or via a reverse proxy).

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Documentation with recommendations on running Datasette in production without using Docker 459397625
504662904 https://github.com/simonw/datasette/issues/514#issuecomment-504662904 https://api.github.com/repos/simonw/datasette/issues/514 MDEyOklzc3VlQ29tbWVudDUwNDY2MjkwNA== russss 45057 2019-06-22T12:45:21Z 2019-06-22T12:45:39Z CONTRIBUTOR

On most modern Linux distros, systemd is the easiest answer.

Example systemd unit file (save to /etc/systemd/system/datasette.service):

[Unit]
Description=Datasette
After=network.target

[Service]
Type=simple
User=<username>
WorkingDirectory=/path/to/data
ExecStart=/path/to/datasette serve -h 0.0.0.0 ./my.db
Restart=on-failure

[Install]
WantedBy=multi-user.target

Activate it with:

$ sudo systemctl daemon-reload
$ sudo systemctl enable datasette
$ sudo systemctl start datasette

Logs are best viewed using journalctl -u datasette -f.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Documentation with recommendations on running Datasette in production without using Docker 459397625
489342728 https://github.com/simonw/datasette/pull/450#issuecomment-489342728 https://api.github.com/repos/simonw/datasette/issues/450 MDEyOklzc3VlQ29tbWVudDQ4OTM0MjcyOA== russss 45057 2019-05-04T16:37:35Z 2019-05-04T16:37:35Z CONTRIBUTOR

For a bit more context: this fixes a crash with unsupported operand type(s) for +: 'int' and 'NoneType' on the index page for me.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Coalesce hidden table count to 0 440304714
489222223 https://github.com/simonw/datasette/issues/446#issuecomment-489222223 https://api.github.com/repos/simonw/datasette/issues/446 MDEyOklzc3VlQ29tbWVudDQ4OTIyMjIyMw== russss 45057 2019-05-03T20:01:19Z 2019-05-03T20:01:29Z CONTRIBUTOR

Also I have a slight preference against (ab)using __slots__ to enforce fields, although I have done it myself in the past. It would be possible to do this with __setattr__ instead, although that's an implementation detail and I'm not too fussed about it.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Define mechanism for plugins to return structured data 440134714
489221481 https://github.com/simonw/datasette/issues/446#issuecomment-489221481 https://api.github.com/repos/simonw/datasette/issues/446 MDEyOklzc3VlQ29tbWVudDQ4OTIyMTQ4MQ== russss 45057 2019-05-03T19:58:31Z 2019-05-03T19:58:31Z CONTRIBUTOR

In this particular case I don't think there's an issue making all those required. However, I suspect we might have to allow optional values at some point - my preferred solution to russss/datasette-geo#2 would need one.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Define mechanism for plugins to return structured data 440134714
489060765 https://github.com/simonw/datasette/issues/419#issuecomment-489060765 https://api.github.com/repos/simonw/datasette/issues/419 MDEyOklzc3VlQ29tbWVudDQ4OTA2MDc2NQ== russss 45057 2019-05-03T11:07:42Z 2019-05-03T11:07:42Z CONTRIBUTOR

Are you planning on removing inspect entirely?

I didn't spot this work before I started on datasette-geo, but ironically I think it has a use case which really needs the inspect functionality (or some replacement).

Datasette-geo uses it to store the bounding box of all the geographic features in the table. This is needed when rendering the map because it avoids having to send loads of tile requests for areas which are empty.

Even with relatively small datasets, calculating the bounding box seems to take around 5 seconds, so I don't think it's really feasible to do this on page load.

One possible fix would be to do this on startup, and then in a thread which watches the database for changes.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Default to opening files in mutable mode, special option for immutable files 421551434
488595724 https://github.com/simonw/datasette/pull/432#issuecomment-488595724 https://api.github.com/repos/simonw/datasette/issues/432 MDEyOklzc3VlQ29tbWVudDQ4ODU5NTcyNA== russss 45057 2019-05-02T08:50:53Z 2019-05-02T08:50:53Z CONTRIBUTOR

Can I pull those needs out of the Facet class somehow?

I was thinking that it might be handy for datasette to have a request object which wraps the Sanic Request. This could include the datasette-specific querystring decoding and the special_args parsing from TableView.data.

This would mean that we could expose the request object to plugin hooks without coupling them to Sanic.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Refactor facets to a class and new plugin, refs #427 432893491
488247617 https://github.com/simonw/datasette/pull/441#issuecomment-488247617 https://api.github.com/repos/simonw/datasette/issues/441 MDEyOklzc3VlQ29tbWVudDQ4ODI0NzYxNw== russss 45057 2019-05-01T09:57:50Z 2019-05-01T09:57:50Z CONTRIBUTOR

Just for the record, this PR is now finished and ready to merge from my perspective.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Add register_output_renderer hook 438437973
487859345 https://github.com/simonw/datasette/pull/439#issuecomment-487859345 https://api.github.com/repos/simonw/datasette/issues/439 MDEyOklzc3VlQ29tbWVudDQ4Nzg1OTM0NQ== russss 45057 2019-04-30T08:21:19Z 2019-04-30T08:21:19Z CONTRIBUTOR

I think the best approach to this is to pass through the view_name parameter I added in #441. It's then simple enough for me to add .geojson to the URL in JS - I don't need the pkey.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
[WIP] Add primary key to the extra_body_script hook arguments 438240541
487748271 https://github.com/simonw/datasette/pull/441#issuecomment-487748271 https://api.github.com/repos/simonw/datasette/issues/441 MDEyOklzc3VlQ29tbWVudDQ4Nzc0ODI3MQ== russss 45057 2019-04-29T21:20:17Z 2019-04-29T21:20:17Z CONTRIBUTOR

Also I just pushed a change to add registered output renderers to the templates:

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Add register_output_renderer hook 438437973
487735247 https://github.com/simonw/datasette/pull/441#issuecomment-487735247 https://api.github.com/repos/simonw/datasette/issues/441 MDEyOklzc3VlQ29tbWVudDQ4NzczNTI0Nw== russss 45057 2019-04-29T20:39:43Z 2019-04-29T20:39:43Z CONTRIBUTOR

I updated the hook to pass the datasette object through now.

You can see the working GeoJSON render function here - the hook function is here.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Add register_output_renderer hook 438437973
487724539 https://github.com/simonw/datasette/pull/441#issuecomment-487724539 https://api.github.com/repos/simonw/datasette/issues/441 MDEyOklzc3VlQ29tbWVudDQ4NzcyNDUzOQ== russss 45057 2019-04-29T20:08:32Z 2019-04-29T20:08:32Z CONTRIBUTOR

I also just realised that I should be passing the datasette object into the hook function...as I just found I need it. So hold off merging until I've fixed that.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Add register_output_renderer hook 438437973
487723476 https://github.com/simonw/datasette/pull/441#issuecomment-487723476 https://api.github.com/repos/simonw/datasette/issues/441 MDEyOklzc3VlQ29tbWVudDQ4NzcyMzQ3Ng== russss 45057 2019-04-29T20:05:23Z 2019-04-29T20:05:23Z CONTRIBUTOR

This is the minimal example (I also included it in the docs):

from datasette import hookimpl

def render_test(args, data, view_name):
    return {
       'body': 'Hello World',
       'content_type': 'text/plain'
    }

@hookimpl
def register_output_renderer():
    return {
        'extension': 'test',
        'callback': render_test
    }

I'm working on the GeoJSON one now and it should be ready soon. (I forgot I was going to run into the same problem as before - that Spatialite's stupid binary format isn't WKB and I have no way of altering the query to change that - but I've just managed to write some code to rearrange the bytes from Spatialite blob-geometry into WKB...)

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Add register_output_renderer hook 438437973
487692377 https://github.com/simonw/datasette/pull/424#issuecomment-487692377 https://api.github.com/repos/simonw/datasette/issues/424 MDEyOklzc3VlQ29tbWVudDQ4NzY5MjM3Nw== russss 45057 2019-04-29T18:30:46Z 2019-04-29T18:30:46Z CONTRIBUTOR

Actually no, I ended up not using the inspected column types in my plugin, and the binary column issue can be solved a lot more simply, so I'll close this.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Column types in inspected metadata 427429265
487689477 https://github.com/simonw/datasette/pull/424#issuecomment-487689477 https://api.github.com/repos/simonw/datasette/issues/424 MDEyOklzc3VlQ29tbWVudDQ4NzY4OTQ3Nw== russss 45057 2019-04-29T18:22:40Z 2019-04-29T18:22:40Z CONTRIBUTOR

This is pretty conflicty because I forgot how to use git fetch. If you're interested in merging this I'll rewrite it against an actual modern checkout...

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Column types in inspected metadata 427429265
487686655 https://github.com/simonw/datasette/pull/441#issuecomment-487686655 https://api.github.com/repos/simonw/datasette/issues/441 MDEyOklzc3VlQ29tbWVudDQ4NzY4NjY1NQ== russss 45057 2019-04-29T18:14:25Z 2019-04-29T18:14:25Z CONTRIBUTOR

Subsidiary note which I forgot in the commit message:

I've decided to give each view a short string name to aid in differentiating which view a hook is being called from. Since hooks are functions and not subclasses, and can get called from different places in the URL hierarchy, it's sometimes difficult to distinguish what data you're actually operating on. I think this will come in handy for other hooks as well.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Add register_output_renderer hook 438437973
487542486 https://github.com/simonw/datasette/pull/439#issuecomment-487542486 https://api.github.com/repos/simonw/datasette/issues/439 MDEyOklzc3VlQ29tbWVudDQ4NzU0MjQ4Ng== russss 45057 2019-04-29T11:20:30Z 2019-04-29T11:20:30Z CONTRIBUTOR

Actually I think this is not the whole story because of the rowid issue. I'm going to think about this one a bit more.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
[WIP] Add primary key to the extra_body_script hook arguments 438240541
487537452 https://github.com/simonw/datasette/pull/437#issuecomment-487537452 https://api.github.com/repos/simonw/datasette/issues/437 MDEyOklzc3VlQ29tbWVudDQ4NzUzNzQ1Mg== russss 45057 2019-04-29T10:58:49Z 2019-04-29T10:58:49Z CONTRIBUTOR

I've just spotted that this implements #215.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Add inspect and prepare_sanic hooks 438048318
405026800 https://github.com/simonw/datasette/issues/294#issuecomment-405026800 https://api.github.com/repos/simonw/datasette/issues/294 MDEyOklzc3VlQ29tbWVudDQwNTAyNjgwMA== russss 45057 2018-07-14T14:24:31Z 2018-07-14T14:24:31Z CONTRIBUTOR

I had a quick look at this in relation to #343 and I feel like it might be worth modelling the inspected table metadata internally as an object rather than a dict. (We'd still have to serialise it back to JSON.)

There are a few places where we rely on the structure of this metadata dict for various reasons, including in templates (and potentially also in user templates). It would be nice to have a reasonably well defined API for accessing metadata internally so that it's clearer what we're breaking.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
inspect should record column types 327365110
405026441 https://github.com/simonw/datasette/issues/343#issuecomment-405026441 https://api.github.com/repos/simonw/datasette/issues/343 MDEyOklzc3VlQ29tbWVudDQwNTAyNjQ0MQ== russss 45057 2018-07-14T14:17:14Z 2018-07-14T14:17:14Z CONTRIBUTOR

This probably depends on #294.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Render boolean fields better by default 341228846
405022335 https://github.com/simonw/datasette/issues/344#issuecomment-405022335 https://api.github.com/repos/simonw/datasette/issues/344 MDEyOklzc3VlQ29tbWVudDQwNTAyMjMzNQ== russss 45057 2018-07-14T13:00:48Z 2018-07-14T13:00:48Z CONTRIBUTOR

Looks like this was a red herring actually, and heroku had a blip when I was testing it...

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
datasette publish heroku fails without name provided 341229113
401312981 https://github.com/simonw/datasette/issues/276#issuecomment-401312981 https://api.github.com/repos/simonw/datasette/issues/276 MDEyOklzc3VlQ29tbWVudDQwMTMxMjk4MQ== russss 45057 2018-06-29T10:14:54Z 2018-06-29T10:14:54Z CONTRIBUTOR

@RusSs Different map projections can presumably be handled on the client side using a leaflet plugin to transform the geometry (eg kartena/Proj4Leaflet) although the leaflet side would need to detect or be informed of the original projection?

Well, as @simonw mentioned, GeoJSON only supports WGS84, and GeoJSON (and/or TopoJSON) is the standard we probably want to aim for. On-the-fly reprojection in spatialite is not an issue anyway, and in general I think you want to be serving stuff to web maps in WGS84 or Web Mercator.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Handle spatialite geometry columns better 324835838
393106520 https://github.com/simonw/datasette/issues/276#issuecomment-393106520 https://api.github.com/repos/simonw/datasette/issues/276 MDEyOklzc3VlQ29tbWVudDM5MzEwNjUyMA== russss 45057 2018-05-30T10:09:25Z 2018-05-30T10:09:25Z CONTRIBUTOR

I don't think it's unreasonable to only support spatialite geometries in a coordinate reference system which is at least transformable to WGS84. It would be nice to support different CRSes in the database so conversion to spatialite from the source data is lossless.

I think the working CRS for datasette should be WGS84 though (leaflet requires it, for example) - it's just a case of calling ST_Transform(geom, 4326) on the column while we're loading the data.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Handle spatialite geometry columns better 324835838
392825746 https://github.com/simonw/datasette/issues/276#issuecomment-392825746 https://api.github.com/repos/simonw/datasette/issues/276 MDEyOklzc3VlQ29tbWVudDM5MjgyNTc0Ng== russss 45057 2018-05-29T15:42:53Z 2018-05-29T15:42:53Z CONTRIBUTOR

I haven't had time to look further into this, but if doing this as a plugin results in useful hooks then I think we should do it that way. We could always require the plugin as a standard dependency.

I think this is going to result in quite a bit of refactoring anyway so it's a good time to add hooks regardless.

On the other hand, if we have to add lots of specialist hooks for it then maybe it's worth integrating into the core.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Handle spatialite geometry columns better 324835838
391505930 https://github.com/simonw/datasette/issues/276#issuecomment-391505930 https://api.github.com/repos/simonw/datasette/issues/276 MDEyOklzc3VlQ29tbWVudDM5MTUwNTkzMA== russss 45057 2018-05-23T21:41:37Z 2018-05-23T21:41:37Z CONTRIBUTOR

I'm not keen on anything that modifies the SQLite file itself on startup

Ah I didn't mean that - I meant altering the SELECT query to fetch the data so that it ran a spatialite function to transform that specific column.

I think that's less useful as a general-purpose plugin hook though, and it's not that hard to parse the WKB in Python (my default approach would be to use shapely, which is great, but geomet looks like an interesting pure-python alternative).

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Handle spatialite geometry columns better 324835838
391050113 https://github.com/simonw/datasette/issues/276#issuecomment-391050113 https://api.github.com/repos/simonw/datasette/issues/276 MDEyOklzc3VlQ29tbWVudDM5MTA1MDExMw== russss 45057 2018-05-22T16:13:00Z 2018-05-22T16:13:00Z CONTRIBUTOR

Yup, I'll have a think about it. My current thoughts are for spatialite we'll need to hook into the following places:

  • Inspection, so we can detect which columns are geometry columns. (We also currently ignore spatialite tables during inspection, it may be worth moving that to the plugin as well.)
  • After data load, so we can convert WKB into the correct intermediate format for display. The alternative here is to alter the select SQL itself and get spatialite to do this conversion, but that strikes me as a bit more complex and possibly not as useful.
  • HTML rendering.
  • Querying?

The rendering and querying hooks could also potentially be used to move the units support into a plugin.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Handle spatialite geometry columns better 324835838
390795067 https://github.com/simonw/datasette/issues/276#issuecomment-390795067 https://api.github.com/repos/simonw/datasette/issues/276 MDEyOklzc3VlQ29tbWVudDM5MDc5NTA2Nw== russss 45057 2018-05-21T21:55:57Z 2018-05-21T21:55:57Z CONTRIBUTOR

Well, we do have the capability to detect spatialite so my intention certainly wasn't to require it.

I can see the advantage of having it as a plugin but it does touch a number of points in the code. I think I'm going to attack this by refactoring the necessary bits and seeing where that leads (which was my plan anyway).

I think my main concern is - if I add certain plugin hooks for this, is anything else ever going to use them? I'm not sure I have an answer to that question yet, either way.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Handle spatialite geometry columns better 324835838
381905593 https://github.com/simonw/datasette/pull/209#issuecomment-381905593 https://api.github.com/repos/simonw/datasette/issues/209 MDEyOklzc3VlQ29tbWVudDM4MTkwNTU5Mw== russss 45057 2018-04-17T08:50:28Z 2018-04-17T08:50:28Z CONTRIBUTOR

I've added another commit which puts classes a class on each <td> by default with its column name, and I've also made the PK column bold.

Unfortunately the tests are still failing on 3.6, which is weird. I can't reproduce locally...

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Don't duplicate simple primary keys in the link column 314455877
381763651 https://github.com/simonw/datasette/issues/203#issuecomment-381763651 https://api.github.com/repos/simonw/datasette/issues/203 MDEyOklzc3VlQ29tbWVudDM4MTc2MzY1MQ== russss 45057 2018-04-16T21:59:17Z 2018-04-16T21:59:17Z CONTRIBUTOR

Ah, I had no idea you could bind python functions into sqlite!

I think the primary purpose of this issue has been served now - I'm going to close this and create a new issue for the only bit of this that hasn't been touched yet, which is (optionally) exposing units in the JSON API.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Support for units 313837303
381738137 https://github.com/simonw/datasette/pull/209#issuecomment-381738137 https://api.github.com/repos/simonw/datasette/issues/209 MDEyOklzc3VlQ29tbWVudDM4MTczODEzNw== russss 45057 2018-04-16T20:27:43Z 2018-04-16T20:27:43Z CONTRIBUTOR

Tests now fixed, honest. The failing test on Travis looks like an intermittent sqlite failure which should resolve itself on a retry...

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Don't duplicate simple primary keys in the link column 314455877
381441392 https://github.com/simonw/datasette/pull/209#issuecomment-381441392 https://api.github.com/repos/simonw/datasette/issues/209 MDEyOklzc3VlQ29tbWVudDM4MTQ0MTM5Mg== russss 45057 2018-04-15T21:59:15Z 2018-04-15T21:59:15Z CONTRIBUTOR

I suspected this would cause some test failures, but I'll wait for opinions before attempting to fix them.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Don't duplicate simple primary keys in the link column 314455877
381361734 https://github.com/simonw/datasette/issues/125#issuecomment-381361734 https://api.github.com/repos/simonw/datasette/issues/125 MDEyOklzc3VlQ29tbWVudDM4MTM2MTczNA== russss 45057 2018-04-14T21:26:30Z 2018-04-14T21:26:30Z CONTRIBUTOR

FWIW I am now doing this on my WTR app (instead of silently limiting maps to 1000).

Telefonica now has about 4000 markers and good old BT has 22,000 or so.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Plot rows on a map with Leaflet and Leaflet.markercluster 275135393
381332222 https://github.com/simonw/datasette/pull/205#issuecomment-381332222 https://api.github.com/repos/simonw/datasette/issues/205 MDEyOklzc3VlQ29tbWVudDM4MTMzMjIyMg== russss 45057 2018-04-14T14:16:35Z 2018-04-14T14:16:35Z CONTRIBUTOR

I've added some tests and that docs link.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Support filtering with units and more 314319372
381315675 https://github.com/simonw/datasette/issues/203#issuecomment-381315675 https://api.github.com/repos/simonw/datasette/issues/203 MDEyOklzc3VlQ29tbWVudDM4MTMxNTY3NQ== russss 45057 2018-04-14T09:14:45Z 2018-04-14T09:27:30Z CONTRIBUTOR

I'd like to figure out a sensible opt-in way to expose this in the JSON output as well. Maybe with a &_units=true parameter?

<s>From a machine-readable perspective I'm not sure why it would be useful to decorate the values with units</s>. Edit: Should have had some coffee first. It's clearly useful for stuff like map rendering!

I agree that the unit metadata should definitely be exposed in the JSON.

In #204 you said "I'd like to add support for using units when querying but this is PR is pretty usable as-is." - I'm fascinated to hear more about how this could work.

I'm thinking about a couple of approaches here. I think the simplest one is: if the column has a unit attached, optionally accept units in query fields:

column_units = ureg("Hz") #  Create a unit object for the column's unit
query_variable = ureg("4 GHz") # Supplied query variable

# Now we can convert the query units into column units before querying
supplied_value.to(column_units).magnitude
> 4000000000.0

# If the user doesn't supply units, pint just returns the plain
# number and we can query as usual assuming it's the base unit
query_variable = ureg("50")
query_variable
> 50

isinstance(query_variable, numbers.Number)
> True

This also lets us do some nice unit conversion on querying:

column_units = ureg("m")
query_variable = ureg("50 ft")

supplied_value.to(column_units)
> <Quantity(15.239999999999998, 'meter')>

The alternative would be to provide a dropdown of units next to the query field (so a "Hz" field would give you "kHz", "MHz", "GHz"). Although this would be clearer to the user, it isn't so easy - we'd need to know more about the context of the field to give you sensible SI prefixes (I'm not so interested in nanoHertz, for example).

You also lose the bonus of being able to convert - although pint will happily show you all the compatible units, it again suffers from a lack of context:

ureg("m").compatible_units()
> frozenset({<Unit('angstrom')>,
           <Unit('thou')>,
           <Unit('inch')>,
           <Unit('link')>,
           <Unit('foot')>,
           <Unit('survey_foot')>,
           <Unit('yard')>,
           <Unit('rod')>,
           <Unit('mile')>,
           <Unit('survey_mile')>,
           <Unit('league')>,
           <Unit('light_year')>})
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Support for units 313837303
381237440 https://github.com/simonw/datasette/pull/202#issuecomment-381237440 https://api.github.com/repos/simonw/datasette/issues/202 MDEyOklzc3VlQ29tbWVudDM4MTIzNzQ0MA== russss 45057 2018-04-13T19:22:53Z 2018-04-13T19:22:53Z CONTRIBUTOR

I spotted you'd mentioned that in #184 but only after I'd written the patch!

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Raise 404 on nonexistent table URLs 313785206
380966565 https://github.com/simonw/datasette/issues/203#issuecomment-380966565 https://api.github.com/repos/simonw/datasette/issues/203 MDEyOklzc3VlQ29tbWVudDM4MDk2NjU2NQ== russss 45057 2018-04-12T22:43:08Z 2018-04-12T22:43:08Z CONTRIBUTOR

Looks like pint is pretty good at this.

In [1]: import pint

In [2]: ureg = pint.UnitRegistry()

In [3]: q = 3e6 * ureg('Hz')

In [4]: '{:~P}'.format(q.to_compact())
Out[4]: '3.0 MHz'

In [5]: q = 0.3 * ureg('m')

In [5]: '{:~P}'.format(q.to_compact())
Out[5]: '300.0 mm'

In [6]: q = 5 * ureg('')

In [7]: '{:~P}'.format(q.to_compact())
Out[7]: '5'
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Support for units 313837303
380608372 https://github.com/simonw/datasette/pull/200#issuecomment-380608372 https://api.github.com/repos/simonw/datasette/issues/200 MDEyOklzc3VlQ29tbWVudDM4MDYwODM3Mg== russss 45057 2018-04-11T21:55:46Z 2018-04-11T21:55:46Z CONTRIBUTOR

I think the most reliable way to detect spatialite is to run SELECT AddGeometryColumn(1, 2, 3, 4, 5); against a :memory: database and see if it throws an exception

Or just see if there's a geometry_columns table? I think that's quite unlikely to be added by accident (and it's an OGC standard). It also tells you if Spatialite is installed in the database rather than just loaded.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Hide Spatialite system tables 313494458

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options:

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
                ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
                ON [issue_comments] ([user]);
Powered by Datasette · Query took 30.129ms · About: github-to-sqlite