I'd rather not turn this into a systemd support thread, but you're trying to execute the package directory there. Your datasette executable is probably at /home/chris/Env/datasette/bin/datasette.

WorkingDirectory=/path/to/data

@russss, Which directory does this represent?

It's the working directory (cwd) of the spawned process. In this case if you set it to the directory your data is in, you can use relative paths to the db (and metadata/templates/etc) in the ExecStart command.

This example is useful to - I like how it has a Makefile that knows how to set up systemd: https://github.com/pikesley/Queube

I wasn't even aware it was possible to add a systemd service at an arbitrary path, but it seems a little messy to me.

Maybe worth noting that systemd does support per-user services which don't require root access. Cool but probably overkill for most people (especially when you're going to need root to listen on port 80 anyway, directly or via a reverse proxy).

On most modern Linux distros, systemd is the easiest answer.

Example systemd unit file (save to /etc/systemd/system/datasette.service): ``` [Unit] Description=Datasette After=network.target

[Service] Type=simple User=<username> WorkingDirectory=/path/to/data ExecStart=/path/to/datasette serve -h 0.0.0.0 ./my.db Restart=on-failure

[Install] WantedBy=multi-user.target ```

Activate it with: bash $ sudo systemctl daemon-reload $ sudo systemctl enable datasette $ sudo systemctl start datasette

Logs are best viewed using journalctl -u datasette -f.

For a bit more context: this fixes a crash with unsupported operand type(s) for +: 'int' and 'NoneType' on the index page for me.

Also I have a slight preference against (ab)using __slots__ to enforce fields, although I have done it myself in the past. It would be possible to do this with __setattr__ instead, although that's an implementation detail and I'm not too fussed about it.

In this particular case I don't think there's an issue making all those required. However, I suspect we might have to allow optional values at some point - my preferred solution to russss/datasette-geo#2 would need one.

Are you planning on removing inspect entirely?

I didn't spot this work before I started on datasette-geo, but ironically I think it has a use case which really needs the inspect functionality (or some replacement).

Datasette-geo uses it to store the bounding box of all the geographic features in the table. This is needed when rendering the map because it avoids having to send loads of tile requests for areas which are empty.

Even with relatively small datasets, calculating the bounding box seems to take around 5 seconds, so I don't think it's really feasible to do this on page load.

One possible fix would be to do this on startup, and then in a thread which watches the database for changes.

Can I pull those needs out of the Facet class somehow?

I was thinking that it might be handy for datasette to have a request object which wraps the Sanic Request. This could include the datasette-specific querystring decoding and the special_args parsing from TableView.data.

This would mean that we could expose the request object to plugin hooks without coupling them to Sanic.

Just for the record, this PR is now finished and ready to merge from my perspective.

I think the best approach to this is to pass through the view_name parameter I added in #441. It's then simple enough for me to add .geojson to the URL in JS - I don't need the pkey.

Also I just pushed a change to add registered output renderers to the templates:

I updated the hook to pass the datasette object through now.

You can see the working GeoJSON render function here - the hook function is here.

I also just realised that I should be passing the datasette object into the hook function...as I just found I need it. So hold off merging until I've fixed that.

This is the minimal example (I also included it in the docs):

```python from datasette import hookimpl

def render_test(args, data, view_name): return {   'body': 'Hello World', 'content_type': 'text/plain' }

@hookimpl def register_output_renderer(): return { 'extension': 'test', 'callback': render_test } ```

I'm working on the GeoJSON one now and it should be ready soon. (I forgot I was going to run into the same problem as before - that Spatialite's stupid binary format isn't WKB and I have no way of altering the query to change that - but I've just managed to write some code to rearrange the bytes from Spatialite blob-geometry into WKB...)

Actually no, I ended up not using the inspected column types in my plugin, and the binary column issue can be solved a lot more simply, so I'll close this.

This is pretty conflicty because I forgot how to use git fetch. If you're interested in merging this I'll rewrite it against an actual modern checkout...

Subsidiary note which I forgot in the commit message:

I've decided to give each view a short string name to aid in differentiating which view a hook is being called from. Since hooks are functions and not subclasses, and can get called from different places in the URL hierarchy, it's sometimes difficult to distinguish what data you're actually operating on. I think this will come in handy for other hooks as well.

Actually I think this is not the whole story because of the rowid issue. I'm going to think about this one a bit more.

I've just spotted that this implements #215.

I had a quick look at this in relation to #343 and I feel like it might be worth modelling the inspected table metadata internally as an object rather than a dict. (We'd still have to serialise it back to JSON.)

There are a few places where we rely on the structure of this metadata dict for various reasons, including in templates (and potentially also in user templates). It would be nice to have a reasonably well defined API for accessing metadata internally so that it's clearer what we're breaking.

This probably depends on #294.

Looks like this was a red herring actually, and heroku had a blip when I was testing it...

@RusSs Different map projections can presumably be handled on the client side using a leaflet plugin to transform the geometry (eg kartena/Proj4Leaflet) although the leaflet side would need to detect or be informed of the original projection?

Well, as @simonw mentioned, GeoJSON only supports WGS84, and GeoJSON (and/or TopoJSON) is the standard we probably want to aim for. On-the-fly reprojection in spatialite is not an issue anyway, and in general I think you want to be serving stuff to web maps in WGS84 or Web Mercator.

I don't think it's unreasonable to only support spatialite geometries in a coordinate reference system which is at least transformable to WGS84. It would be nice to support different CRSes in the database so conversion to spatialite from the source data is lossless.

I think the working CRS for datasette should be WGS84 though (leaflet requires it, for example) - it's just a case of calling ST_Transform(geom, 4326) on the column while we're loading the data.

I haven't had time to look further into this, but if doing this as a plugin results in useful hooks then I think we should do it that way. We could always require the plugin as a standard dependency.

I think this is going to result in quite a bit of refactoring anyway so it's a good time to add hooks regardless.

On the other hand, if we have to add lots of specialist hooks for it then maybe it's worth integrating into the core.

I'm not keen on anything that modifies the SQLite file itself on startup

Ah I didn't mean that - I meant altering the SELECT query to fetch the data so that it ran a spatialite function to transform that specific column.

I think that's less useful as a general-purpose plugin hook though, and it's not that hard to parse the WKB in Python (my default approach would be to use shapely, which is great, but geomet looks like an interesting pure-python alternative).

Yup, I'll have a think about it. My current thoughts are for spatialite we'll need to hook into the following places:

• Inspection, so we can detect which columns are geometry columns. (We also currently ignore spatialite tables during inspection, it may be worth moving that to the plugin as well.)
• After data load, so we can convert WKB into the correct intermediate format for display. The alternative here is to alter the select SQL itself and get spatialite to do this conversion, but that strikes me as a bit more complex and possibly not as useful.
• HTML rendering.
• Querying?

The rendering and querying hooks could also potentially be used to move the units support into a plugin.

Well, we do have the capability to detect spatialite so my intention certainly wasn't to require it.

I can see the advantage of having it as a plugin but it does touch a number of points in the code. I think I'm going to attack this by refactoring the necessary bits and seeing where that leads (which was my plan anyway).

I think my main concern is - if I add certain plugin hooks for this, is anything else ever going to use them? I'm not sure I have an answer to that question yet, either way.

I've added another commit which puts classes a class on each <td> by default with its column name, and I've also made the PK column bold.

Unfortunately the tests are still failing on 3.6, which is weird. I can't reproduce locally...

Ah, I had no idea you could bind python functions into sqlite!

I think the primary purpose of this issue has been served now - I'm going to close this and create a new issue for the only bit of this that hasn't been touched yet, which is (optionally) exposing units in the JSON API.

Tests now fixed, honest. The failing test on Travis looks like an intermittent sqlite failure which should resolve itself on a retry...

I suspected this would cause some test failures, but I'll wait for opinions before attempting to fix them.

FWIW I am now doing this on my WTR app (instead of silently limiting maps to 1000).

Telefonica now has about 4000 markers and good old BT has 22,000 or so.

I've added some tests and that docs link.

I'd like to figure out a sensible opt-in way to expose this in the JSON output as well. Maybe with a &_units=true parameter?

<s>From a machine-readable perspective I'm not sure why it would be useful to decorate the values with units</s>. Edit: Should have had some coffee first. It's clearly useful for stuff like map rendering!

I agree that the unit metadata should definitely be exposed in the JSON.

In #204 you said "I'd like to add support for using units when querying but this is PR is pretty usable as-is." - I'm fascinated to hear more about how this could work.

I'm thinking about a couple of approaches here. I think the simplest one is: if the column has a unit attached, optionally accept units in query fields:

```python column_units = ureg("Hz") # Create a unit object for the column's unit query_variable = ureg("4 GHz") # Supplied query variable

Now we can convert the query units into column units before querying

supplied_value.to(column_units).magnitude

4000000000.0

If the user doesn't supply units, pint just returns the plain

number and we can query as usual assuming it's the base unit

query_variable = ureg("50") query_variable

50

isinstance(query_variable, numbers.Number)

True ```

This also lets us do some nice unit conversion on querying:

```python column_units = ureg("m") query_variable = ureg("50 ft")

supplied_value.to(column_units)

<Quantity(15.239999999999998, 'meter')> ```

The alternative would be to provide a dropdown of units next to the query field (so a "Hz" field would give you "kHz", "MHz", "GHz"). Although this would be clearer to the user, it isn't so easy - we'd need to know more about the context of the field to give you sensible SI prefixes (I'm not so interested in nanoHertz, for example).

You also lose the bonus of being able to convert - although pint will happily show you all the compatible units, it again suffers from a lack of context:

```python ureg("m").compatible_units()

frozenset({<Unit('angstrom')>, <Unit('thou')>, <Unit('inch')>, <Unit('link')>, <Unit('foot')>, <Unit('survey_foot')>, <Unit('yard')>, <Unit('rod')>, <Unit('mile')>, <Unit('survey_mile')>, <Unit('league')>, <Unit('light_year')>}) ```

I spotted you'd mentioned that in #184 but only after I'd written the patch!

Looks like pint is pretty good at this.

```python In [1]: import pint

In [2]: ureg = pint.UnitRegistry()

In [3]: q = 3e6 * ureg('Hz')

In [4]: '{:~P}'.format(q.to_compact()) Out[4]: '3.0 MHz'

In [5]: q = 0.3 * ureg('m')

In [5]: '{:~P}'.format(q.to_compact()) Out[5]: '300.0 mm'

In [6]: q = 5 * ureg('')

In [7]: '{:~P}'.format(q.to_compact()) Out[7]: '5' ```

I think the most reliable way to detect spatialite is to run SELECT AddGeometryColumn(1, 2, 3, 4, 5); against a :memory: database and see if it throws an exception

Or just see if there's a geometry_columns table? I think that's quite unlikely to be added by accident (and it's an OGC standard). It also tells you if Spatialite is installed in the database rather than just loaded.