issue_comments

It's a shame pyinstaller can't act as a cross-compiler - so I don't think I can get Travis CI to build packages. But it's fantastic that it's possible to turn the tool into a standalone executable!

It worked!

$ pyinstaller -F \
    --add-data /usr/local/lib/python3.5/site-packages/datasette/templates:datasette/templates \
    --add-data /usr/local/lib/python3.5/site-packages/datasette/static:datasette/static \
    /usr/local/bin/datasette

$ file dist/datasette 
dist/datasette: Mach-O 64-bit executable x86_64
$ dist/datasette --help
Usage: datasette [OPTIONS] COMMAND [ARGS]...

  Datasette!

Options:
  --help  Show this message and exit.

Commands:
  serve*   Serve up specified SQLite database files with...
  build
  package  Package specified SQLite files into a new...
  publish  Publish specified SQLite database files to...

Plugins should be able to interact with the build step. This would give plugins an opportunity to modify the SQL databases and help prepare them for serving - for example, a full-text search plugin might create additional FTS tables, or a mapping plugin might pre-calculate a bunch of geohashes for tables that have latitude/longitude values. Plugins could really take advantage of the immutable nature of the dataset here.

I'm getting an internal server error on http://run.plnkr.co/preview/cj9zlf1qc0003414y90ajkwpk/ at the moment

Sorry about that - here's a working version on Netlify:

https://nhs-england-map.netlify.com

i will look better tomorrow, it's late i surely made some mistake https://asciinema.org/a/ZyAWbetrlriDadwWyVPUWB94H

I'm getting an internal server error on http://run.plnkr.co/preview/cj9zlf1qc0003414y90ajkwpk/ at the moment

Hooray! First dataset that wasn't deployed by me :) https://github.com/simonw/datasette/wiki/Datasettes

That didn't quite work for me. It built me a dist/datasette executable but when I try to run it I get an error:

$ pwd
/Users/simonw/Dropbox/Development/datasette
$ source venv/bin/activate
$ pyinstaller -F --add-data datasette/templates:datasette/templates --add-data datasette/static:datasette/static /Users/simonw/Dropbox/Development/datasette/venv/bin/datasette
$ dist/datasette --help
Traceback (most recent call last):
  File "datasette", line 11, in <module>
  File "site-packages/pkg_resources/__init__.py", line 572, in load_entry_point
  File "site-packages/pkg_resources/__init__.py", line 564, in get_distribution
  File "site-packages/pkg_resources/__init__.py", line 436, in get_provider
  File "site-packages/pkg_resources/__init__.py", line 984, in require
  File "site-packages/pkg_resources/__init__.py", line 870, in resolve
pkg_resources.DistributionNotFound: The 'datasette' distribution was not found and is required by the application
[99117] Failed to execute script datasette

tried quickly, this seems working:

``` ~ pip3 install pyinstaller ~ pyinstaller -F --add-data /usr/local/lib/python3.6/site-packages/datasette/templates:datasette/templates --add-data /usr/local/lib/python3.6/site-packages/datasette/static:datasette/static /usr/local/bin/datasette

~ du -h dist/datasette 6.8M dist/datasette ~ file dist/datasette dist/datasette: Mach-O 64-bit executable x86_64 ```

Looks like we'd need to use this recipe: https://github.com/pyinstaller/pyinstaller/wiki/Recipe-Setuptools-Entry-Point

Even without bundling in the database file itself, I'd love to have a standalone binary version of the core datasette CLI utility.

I think Sanic may have some complex dependencies, but I've never tried pyinstaller so I don't know how easy or hard it would be to get this working.

This is a dupe of #85

Thanks for the explanation! Please do start a pull request.

This isn’t necessary - restarting the server is fast and easy, and I’ve not found myself needing this at all during development.

https://github.com/frappe/charts perhaps

Also requested on Twitter: https://twitter.com/DenubisX/status/930322813864439808

http://charlesleifer.com/blog/going-fast-with-sqlite-and-python/ is useful here too.

Spatial extensions would be really useful too. https://www.gaia-gis.it/spatialite-2.1/SpatiaLite-manual.html

The pattern is called "multi-stage builds". And the result is a svelte 226MB image (201MB for 3.6-slim) vs 700MB+ for the full image. It's possible to get it even smaller, but that takes a lot more work.

I’m intrigued by this pattern:

https://github.com/macropin/datasette/blob/147195c2fdfa2b984d8f9fc1c6cab6634970a056/Dockerfile#L8

What’s the benefit of doing that? Does it result in a smaller image size?

Let me know if you'd like a PR. The image is usable as docker run --rm -t -i -p 9000:8001 -v $(pwd)/db:/db datasette datasette serve /db/chinook.db

I'm happy to contribute this. Just let me know if you want a Dockerfile for development or production purposes, or both.

If it's prod then we can just pip install the source from pypi, otherwise for dev we'll need a requirements.txt to speed up rebuilds.

This is probably a bit too much for the README - I should get readthedocs working.

I managed to do this manually:

datasette package ~/parlgov-db/parlgov.db --metadata=parlgov.json
# Output 8758ec31dda3 as the new image ID
docker save 8758ec31dda3 > /tmp/my-image
# I could have just piped this straight to hyper
cat /tmp/my-image | hyper load
# Now start the container running in hyper
hyper run -d -p 80:8001 --name parlgov 8758ec31dda3
# We need to assign an IP address so we can see it
hyper fip allocate 1
# Outputs 199.245.58.78
hyper fip attach 199.245.58.78 parlgov

At this point, visiting the IP address in a browser showed the parlgov UI.

To clean up...

hyper hyper fip detach parlgov
hyper fip release 199.245.58.78
hyper stop parlgov
hyper rm parlgov

I ended up shipping with https://fivethirtyeight.datasettes.com/ and https://parlgov.datasettes.com/

Oops, if I jumped the gun. I saw the project in my github activity feed and saw some low hanging fruit :)

Did this: https://simonwillison.net/2017/Nov/13/datasette/