issue_comments

Closing this in favour of Datasette Desktop: https://datasette.io/desktop

Next step is to automate this all!

I tested it by running a tmate session on the GitHub macOS machines, and it worked! ``` Mac-1609795972770:tmp runner$ wget 'https://github.com/simonw/datasette/releases/download/0.53/datasette-0.53-macos-binary.zip' --2021-01-04 21:34:10-- https://github.com/simonw/datasette/releases/download/0.53/datasette-0.53-macos-binary.zip Resolving github.com (github.com)... 140.82.114.4 Connecting to github.com (github.com)|140.82.114.4|:443... connected. HTTP request sent, awaiting response... 302 Found Location: https://github-production-release-asset-2e65be.s3.amazonaws.com/107914493/74658700-4e90-11eb-8f3b-ee77e6dfad90?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20210104%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210104T213414Z&X-Amz-Expires=300&X-Amz-Signature=6f3c54211077092553590b33a7c36cd052895c9d4619607ad1df094782f64acf&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=107914493&response-content-disposition=attachment%3B%20filename%3Ddatasette-0.53-macos-binary.zip&response-content-type=application%2Foctet-stream [following] --2021-01-04 21:34:14-- https://github-production-release-asset-2e65be.s3.amazonaws.com/107914493/74658700-4e90-11eb-8f3b-ee77e6dfad90?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20210104%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210104T213414Z&X-Amz-Expires=300&X-Amz-Signature=6f3c54211077092553590b33a7c36cd052895c9d4619607ad1df094782f64acf&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=107914493&response-content-disposition=attachment%3B%20filename%3Ddatasette-0.53-macos-binary.zip&response-content-type=application%2Foctet-stream Resolving github-production-release-asset-2e65be.s3.amazonaws.com (github-production-release-asset-2e65be.s3.amazonaws.com)... 52.217.43.164 Connecting to github-production-release-asset-2e65be.s3.amazonaws.com (github-production-release-asset-2e65be.s3.amazonaws.com)|52.217.43.164|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 8297283 (7.9M) [application/octet-stream] Saving to: ‘datasette-0.53-macos-binary.zip’

datasette-0.53-maco 100%[===================>] 7.91M --.-KB/s in 0.1s

2021-01-04 21:34:14 (73.4 MB/s) - ‘datasette-0.53-macos-binary.zip’ saved [8297283/8297283]

Mac-1609795972770:tmp runner$ unzip datasette-0.53-macos-binary.zip Archive: datasette-0.53-macos-binary.zip creating: datasette-0.53-macos-binary/ inflating: datasette-0.53-macos-binary/datasette
Mac-1609795972770:tmp runner$ datasette-0.53-macos-binary/datasette --help Usage: datasette [OPTIONS] COMMAND [ARGS]...

Datasette!

Options: --version Show the version and exit. --help Show this message and exit.

Commands: serve* Serve up specified SQLite database files with a web UI inspect install Install Python packages - e.g. package Package specified SQLite files into a new datasette Docker... plugins List currently available plugins publish Publish specified SQLite database files to the internet along... uninstall Uninstall Python packages (e.g. Mac-1609795972770:tmp runner$ datasette-0.53-macos-binary/datasette --get /-/versions.json {"python": {"version": "3.9.1", "full": "3.9.1 (default, Dec 10 2020, 10:36:35) \n[Clang 12.0.0 (clang-1200.0.32.27)]"}, "datasette": {"version": "0.53"}, "asgi": "3.0", "uvicorn": "0.13.3", "sqlite": {"version": "3.34.0", "fts_versions": ["FTS5", "FTS4", "FTS3"], "extensions": {"json1": null}, "compile_options": ["COMPILER=clang-12.0.0", "ENABLE_COLUMN_METADATA", "ENABLE_FTS3", "ENABLE_FTS3_PARENTHESIS", "ENABLE_FTS4", "ENABLE_FTS5", "ENABLE_GEOPOLY", "ENABLE_JSON1", "ENABLE_PREUPDATE_HOOK", "ENABLE_RTREE", "ENABLE_SESSION", "MAX_VARIABLE_NUMBER=250000", "THREADSAFE=1"]}} Mac-1609795972770:tmp runner$ ```

As an experiment, I put the macOS one in a zip file and attached it to the latest release:

mkdir datasette-0.53-macos-binary cp dist/datasette datasette-0.53-macos-binary zip -r datasette-0.53-macos-binary.zip datasette-0.53-macos-binary

It's available here: https://github.com/simonw/datasette/releases/tag/0.53 - download URL is https://github.com/simonw/datasette/releases/download/0.53/datasette-0.53-macos-binary.zip

(pyinstaller-venv) root@dogsheep:/tmp/pyinstaller-venv# dist/datasette --get /-/databases.json [{"name": ":memory:", "path": null, "size": 0, "is_mutable": true, "is_memory": true, "hash": null}] (pyinstaller-venv) root@dogsheep:/tmp/pyinstaller-venv# ls -lah dist/datasette -rwxr-xr-x 1 root root 8.9M Jan 4 21:05 dist/datasette

Works on Linux/Ubuntu too, except I had to do export BASE= on a separate line. I also did this: ``` apt-get install python3 python3-venv python3 -m venv pyinstaller-venv source pyinstaller-venv/bin/activate pip install wheel pip install datasette pyinstaller

export DATASETTE_BASE=$(python -c 'import os; print(os.path.dirname(import("datasette").file))')

pyinstaller -F \ --add-data "$DATASETTE_BASE/templates:datasette/templates" \ --add-data "$DATASETTE_BASE/static:datasette/static" \ --hidden-import datasette.publish \ --hidden-import datasette.publish.heroku \ --hidden-import datasette.publish.cloudrun \ --hidden-import datasette.facets \ --hidden-import datasette.sql_functions \ --hidden-import datasette.actor_auth_cookie \ --hidden-import datasette.default_permissions \ --hidden-import datasette.default_magic_parameters \ --hidden-import datasette.blob_renderer \ --hidden-import datasette.default_menu_links \ --hidden-import uvicorn \ --hidden-import uvicorn.logging \ --hidden-import uvicorn.loops \ --hidden-import uvicorn.loops.auto \ --hidden-import uvicorn.protocols \ --hidden-import uvicorn.protocols.http \ --hidden-import uvicorn.protocols.http.auto \ --hidden-import uvicorn.protocols.websockets \ --hidden-import uvicorn.protocols.websockets.auto \ --hidden-import uvicorn.lifespan \ --hidden-import uvicorn.lifespan.on \ $(which datasette) ```

That BASE= trick seems to work with zsh but not with bash.

Updated pyinstaller recipe - lots of hidden imports needed now: ``` pip install wheel pip install datasette pyinstaller

BASE=$(python -c 'import os; print(os.path.dirname(import("datasette").file))') \ pyinstaller -F \ --add-data "$BASE/templates:datasette/templates" \ --add-data "$BASE/static:datasette/static" \ --hidden-import datasette.publish \ --hidden-import datasette.publish.heroku \ --hidden-import datasette.publish.cloudrun \ --hidden-import datasette.facets \ --hidden-import datasette.sql_functions \ --hidden-import datasette.actor_auth_cookie \ --hidden-import datasette.default_permissions \ --hidden-import datasette.default_magic_parameters \ --hidden-import datasette.blob_renderer \ --hidden-import datasette.default_menu_links \ --hidden-import uvicorn \ --hidden-import uvicorn.logging \ --hidden-import uvicorn.loops \ --hidden-import uvicorn.loops.auto \ --hidden-import uvicorn.protocols \ --hidden-import uvicorn.protocols.http \ --hidden-import uvicorn.protocols.http.auto \ --hidden-import uvicorn.protocols.websockets \ --hidden-import uvicorn.protocols.websockets.auto \ --hidden-import uvicorn.lifespan \ --hidden-import uvicorn.lifespan.on \ $(which datasette) ```

(pyinstaller-datasette) pyinstaller-datasette % file dist/datasette dist/datasette: Mach-O 64-bit executable x86_64 (pyinstaller-datasette) pyinstaller-datasette % ls -lah dist/datasette -rwxr-xr-x 1 simon wheel 8.0M Jan 4 12:58 dist/datasette I'm surprised it's only 8MB!

Relevant conversation on Twitter: https://twitter.com/simonw/status/1180866651962560512?s=21

It's a shame pyinstaller can't act as a cross-compiler - so I don't think I can get Travis CI to build packages. But it's fantastic that it's possible to turn the tool into a standalone executable!

It worked!

$ pyinstaller -F \
    --add-data /usr/local/lib/python3.5/site-packages/datasette/templates:datasette/templates \
    --add-data /usr/local/lib/python3.5/site-packages/datasette/static:datasette/static \
    /usr/local/bin/datasette

$ file dist/datasette 
dist/datasette: Mach-O 64-bit executable x86_64
$ dist/datasette --help
Usage: datasette [OPTIONS] COMMAND [ARGS]...

  Datasette!

Options:
  --help  Show this message and exit.

Commands:
  serve*   Serve up specified SQLite database files with...
  build
  package  Package specified SQLite files into a new...
  publish  Publish specified SQLite database files to...

That didn't quite work for me. It built me a dist/datasette executable but when I try to run it I get an error:

$ pwd
/Users/simonw/Dropbox/Development/datasette
$ source venv/bin/activate
$ pyinstaller -F --add-data datasette/templates:datasette/templates --add-data datasette/static:datasette/static /Users/simonw/Dropbox/Development/datasette/venv/bin/datasette
$ dist/datasette --help
Traceback (most recent call last):
  File "datasette", line 11, in <module>
  File "site-packages/pkg_resources/__init__.py", line 572, in load_entry_point
  File "site-packages/pkg_resources/__init__.py", line 564, in get_distribution
  File "site-packages/pkg_resources/__init__.py", line 436, in get_provider
  File "site-packages/pkg_resources/__init__.py", line 984, in require
  File "site-packages/pkg_resources/__init__.py", line 870, in resolve
pkg_resources.DistributionNotFound: The 'datasette' distribution was not found and is required by the application
[99117] Failed to execute script datasette

Looks like we'd need to use this recipe: https://github.com/pyinstaller/pyinstaller/wiki/Recipe-Setuptools-Entry-Point

Even without bundling in the database file itself, I'd love to have a standalone binary version of the core datasette CLI utility.

I think Sanic may have some complex dependencies, but I've never tried pyinstaller so I don't know how easy or hard it would be to get this working.