{"html_url": "https://github.com/simonw/datasette/issues/1249#issuecomment-804347152", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1249", "id": 804347152, "node_id": "MDEyOklzc3VlQ29tbWVudDgwNDM0NzE1Mg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-03-22T19:47:56Z", "updated_at": "2021-03-22T19:48:03Z", "author_association": "OWNER", "body": "I wrote a bunch of tips on creating smaller Docker images here: https://simonwillison.net/2018/Nov/19/smaller-python-docker-images/", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 824064069, "label": "Updated Dockerfile with SpatiaLite version 5.0"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1249#issuecomment-804344553", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1249", "id": 804344553, "node_id": "MDEyOklzc3VlQ29tbWVudDgwNDM0NDU1Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-03-22T19:43:25Z", "updated_at": "2021-03-22T19:43:25Z", "author_association": "OWNER", "body": "Does `--no-install-recommends` make a difference?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 824064069, "label": "Updated Dockerfile with SpatiaLite version 5.0"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1249#issuecomment-804338678", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1249", "id": 804338678, "node_id": "MDEyOklzc3VlQ29tbWVudDgwNDMzODY3OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-03-22T19:33:43Z", "updated_at": "2021-03-22T19:33:43Z", "author_association": "OWNER", "body": "Replacing `rm -rf /var/lib/{apt,dpkg,cache,log}/` with \r\n```\r\n    rm -rf /var/lib/apt && \\\r\n    rm -rf /var/lib/dpkg\r\n```\r\nGot the size down to 305MB.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 824064069, "label": "Updated Dockerfile with SpatiaLite version 5.0"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1249#issuecomment-804318314", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1249", "id": 804318314, "node_id": "MDEyOklzc3VlQ29tbWVudDgwNDMxODMxNA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-03-22T19:04:30Z", "updated_at": "2021-03-22T19:04:30Z", "author_association": "OWNER", "body": "Considering the image on Docker Hub right now is `383MB` this is actually an improvement.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 824064069, "label": "Updated Dockerfile with SpatiaLite version 5.0"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1249#issuecomment-804317545", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1249", "id": 804317545, "node_id": "MDEyOklzc3VlQ29tbWVudDgwNDMxNzU0NQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-03-22T19:03:22Z", "updated_at": "2021-03-22T19:03:22Z", "author_association": "OWNER", "body": "This Dockerfile:\r\n```dockerfile\r\nFROM python:3.9.2-slim-buster as build\r\n\r\n# software-properties-common provides add-apt-repository\r\nRUN apt-get update && \\\r\n    apt-get -y install software-properties-common && \\\r\n    add-apt-repository \"deb http://httpredir.debian.org/debian sid main\" && \\\r\n    apt-get update && \\\r\n    apt-get -t sid install -y libsqlite3-mod-spatialite && \\\r\n    apt clean && \\\r\n    rm -rf /var/lib/{apt,dpkg,cache,log}/\r\n\r\nRUN pip install datasette\r\n\r\nEXPOSE 8001\r\nCMD [\"datasette\"]\r\n```\r\nProduces a 344MB image that includes a working SpatiaLite 5.0 module. And weirdly... it doesn't exhibit the hanging bug!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 824064069, "label": "Updated Dockerfile with SpatiaLite version 5.0"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1249#issuecomment-804310353", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1249", "id": 804310353, "node_id": "MDEyOklzc3VlQ29tbWVudDgwNDMxMDM1Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-03-22T18:52:12Z", "updated_at": "2021-03-22T18:52:12Z", "author_association": "OWNER", "body": "This Dockerfile:\r\n```dockerfile\r\nFROM python:3.9.2-slim-buster as build\r\n\r\n# Setup build dependencies\r\nRUN apt update \\\r\n    && apt install -y python3-dev build-essential wget libxml2-dev libproj-dev \\\r\n    libminizip-dev libgeos-dev libsqlite3-dev zlib1g-dev pkg-config git \\\r\n    && apt clean\r\n\r\nRUN wget \"https://www.sqlite.org/2021/sqlite-autoconf-3340100.tar.gz\" && tar xzf sqlite-autoconf-3340100.tar.gz \\\r\n    && cd sqlite-autoconf-3340100 && ./configure --disable-static --enable-fts5 --enable-json1 \\\r\n    CFLAGS=\"-g -O2 -DSQLITE_ENABLE_FTS3=1 -DSQLITE_ENABLE_FTS3_PARENTHESIS -DSQLITE_ENABLE_FTS4=1 -DSQLITE_ENABLE_RTREE=1 -DSQLITE_ENABLE_JSON1\" \\\r\n    && make && make install\r\n\r\nRUN wget \"http://www.gaia-gis.it/gaia-sins/freexl-1.0.6.tar.gz\" && tar zxf freexl-1.0.6.tar.gz \\\r\n    && cd freexl-1.0.6 && ./configure && make && make install\r\n\r\nRUN wget \"http://www.gaia-gis.it/gaia-sins/libspatialite-5.0.1.tar.gz\" && tar zxf libspatialite-5.0.1.tar.gz \\\r\n    && cd libspatialite-5.0.1 && ./configure --disable-rttopo && make && make install\r\n\r\nRUN wget \"http://www.gaia-gis.it/gaia-sins/readosm-sources/readosm-1.1.0.tar.gz\" && tar zxf readosm-1.1.0.tar.gz && cd readosm-1.1.0 && ./configure && make && make install\r\n\r\nRUN wget \"http://www.gaia-gis.it/gaia-sins/spatialite-tools-5.0.0.tar.gz\" && tar zxf spatialite-tools-5.0.0.tar.gz \\\r\n    && cd spatialite-tools-5.0.0 && ./configure --disable-rttopo && make && make install\r\n\r\n# Add local code to the image instead of fetching from pypi.\r\n#COPY . /datasette\r\n#RUN pip install /datasette\r\nRUN pip install datasette\r\n\r\nFROM python:3.9.2-slim-buster\r\n\r\n# Copy python dependencies and spatialite libraries\r\nCOPY --from=build /usr/local/lib/ /usr/local/lib/\r\n# Copy executables\r\nCOPY --from=build /usr/local/bin /usr/local/bin\r\n# Copy spatial extensions\r\nCOPY --from=build /usr/lib/x86_64-linux-gnu /usr/lib/x86_64-linux-gnu\r\n\r\nENV LD_LIBRARY_PATH=/usr/local/lib\r\n\r\nEXPOSE 8001\r\nCMD [\"datasette\"]\r\n```\r\nProduced a 448MB image.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 824064069, "label": "Updated Dockerfile with SpatiaLite version 5.0"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1249#issuecomment-804309510", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1249", "id": 804309510, "node_id": "MDEyOklzc3VlQ29tbWVudDgwNDMwOTUxMA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-03-22T18:50:50Z", "updated_at": "2021-03-22T18:50:50Z", "author_association": "OWNER", "body": "Ideally I'd like to use the Debian stable `python:3.9.2-slim-buster` base image but install SpatiaLite from Debian unstable here: https://packages.debian.org/sid/libspatialite7\r\n\r\nThis pattern might let me do that: https://github.com/helmesjo/cpp_bash_utils/blob/f031e926249f8e2d7f260f22dc8974c6d5be11fe/docker/images/linux-gcc.dockerfile#L20-L24", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 824064069, "label": "Updated Dockerfile with SpatiaLite version 5.0"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1249#issuecomment-804263434", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1249", "id": 804263434, "node_id": "MDEyOklzc3VlQ29tbWVudDgwNDI2MzQzNA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-03-22T17:43:25Z", "updated_at": "2021-03-22T17:43:25Z", "author_association": "OWNER", "body": "I figured out the cause of the hang in #1268 - it was caused by `select count(*) from SpatialIndex` interacting badly with the `set_progress_handler()` mechanism I was using to implement query time limits. #1271 has a replacement for that using `asyncio.wait_for()` and `conn.interrupt()` which should resolve the SpatiaLite issue too.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 824064069, "label": "Updated Dockerfile with SpatiaLite version 5.0"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1249#issuecomment-803755698", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1249", "id": 803755698, "node_id": "MDEyOklzc3VlQ29tbWVudDgwMzc1NTY5OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-03-22T04:43:02Z", "updated_at": "2021-03-22T04:43:02Z", "author_association": "OWNER", "body": "I'll spin off a separate ticket to investigate the hang.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 824064069, "label": "Updated Dockerfile with SpatiaLite version 5.0"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1249#issuecomment-803753388", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1249", "id": 803753388, "node_id": "MDEyOklzc3VlQ29tbWVudDgwMzc1MzM4OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-03-22T04:34:20Z", "updated_at": "2021-03-22T04:35:10Z", "author_association": "OWNER", "body": "Well this is frustrating. I finally found a Dockerfile that worked and installed an Ubuntu pre-compiled SpatiaLite module that would load...\r\n```dockerfile\r\nFROM ubuntu:20.10 as install_spatialite\r\n\r\nRUN apt update && \\\r\n    apt install -y libsqlite3-mod-spatialite && \\\r\n    apt clean && \\\r\n    rm -rf /var/lib/{apt,dpkg,cache,log}/\r\n\r\nFROM ubuntu:20.10\r\n\r\nRUN apt update && \\\r\n    apt install -y python3-pip && \\\r\n    apt clean && \\\r\n    rm -rf /var/lib/{apt,dpkg,cache,log}/\r\n\r\nRUN pip install datasette\r\n\r\n# Copy spatial extensions\r\nCOPY --from=install_spatialite /usr/lib/x86_64-linux-gnu /usr/lib/x86_64-linux-gnu/\r\n\r\nENV LD_LIBRARY_PATH=/usr/local/lib\r\n\r\nEXPOSE 8001\r\nCMD [\"datasette\"]\r\n```\r\n(Which produced a 550MB image)\r\n\r\nAnd when I ran Datasette I got that same error where the database listing page hangs!\r\n```\r\ndocker run -p 8001:8001 -v `pwd`:/mnt datasette-spatialite:latest datasette -p 8001 -h 0.0.0.0 /mnt/tuscany_housenumbers.sqlite --load-extension=spatialite\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 824064069, "label": "Updated Dockerfile with SpatiaLite version 5.0"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1249#issuecomment-803751068", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1249", "id": 803751068, "node_id": "MDEyOklzc3VlQ29tbWVudDgwMzc1MTA2OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-03-22T04:26:45Z", "updated_at": "2021-03-22T04:26:45Z", "author_association": "OWNER", "body": "Here's why:\r\n```\r\ndatasette % docker run -it -p 8001:8001 -v `pwd`:/mnt datasette-spatialite:latest bash                                                      \r\nroot@3430352ff378:/# datasette\r\nbash: /usr/local/bin/datasette: /usr/bin/python3: bad interpreter: No such file or directory\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 824064069, "label": "Updated Dockerfile with SpatiaLite version 5.0"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1249#issuecomment-803750617", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1249", "id": 803750617, "node_id": "MDEyOklzc3VlQ29tbWVudDgwMzc1MDYxNw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-03-22T04:25:14Z", "updated_at": "2021-03-22T04:25:14Z", "author_association": "OWNER", "body": "Got this error attempting to run Datasette (with or without SpatiaLite):\r\n```\r\nstandard_init_linux.go:219: exec user process caused: no such file or directory\r\n```\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 824064069, "label": "Updated Dockerfile with SpatiaLite version 5.0"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1249#issuecomment-803750399", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1249", "id": 803750399, "node_id": "MDEyOklzc3VlQ29tbWVudDgwMzc1MDM5OQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-03-22T04:24:25Z", "updated_at": "2021-03-22T04:24:25Z", "author_association": "OWNER", "body": "I'll try using `ubuntu:20.10` for everything:\r\n```dockerfile\r\nFROM ubuntu:20.10 as install_spatialite\r\n\r\nRUN apt update && \\\r\n    apt install -y libsqlite3-mod-spatialite && \\\r\n    apt clean && \\\r\n    rm -rf /var/lib/{apt,dpkg,cache,log}/\r\n\r\nFROM ubuntu:20.10 as build\r\n\r\nRUN apt update && \\\r\n    apt install -y python3-pip && \\\r\n    apt clean && \\\r\n    rm -rf /var/lib/{apt,dpkg,cache,log}/\r\n\r\nRUN pip install datasette\r\n\r\n#COPY . /datasette\r\n#RUN pip install /datasette\r\n\r\nFROM ubuntu:20.10\r\n\r\n# Copy python dependencies and spatialite libraries\r\nCOPY --from=build /usr/local/lib/ /usr/local/lib/\r\n# Copy executables\r\nCOPY --from=build /usr/local/bin /usr/local/bin\r\n# Copy spatial extensions\r\nCOPY --from=install_spatialite /usr/lib/x86_64-linux-gnu/mod_spatialite.so /usr/lib/x86_64-linux-gnu/mod_spatialite.so\r\n\r\nENV LD_LIBRARY_PATH=/usr/local/lib\r\n\r\nEXPOSE 8001\r\nCMD [\"datasette\"]\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 824064069, "label": "Updated Dockerfile with SpatiaLite version 5.0"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1249#issuecomment-803749831", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1249", "id": 803749831, "node_id": "MDEyOklzc3VlQ29tbWVudDgwMzc0OTgzMQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-03-22T04:22:35Z", "updated_at": "2021-03-22T04:22:35Z", "author_association": "OWNER", "body": "I tried copying just the `mod_spatialite.so` file:\r\n```dockerfile\r\nFROM ubuntu:20.10 as install_spatialite\r\n\r\nRUN apt update && \\\r\n    apt install -y libsqlite3-mod-spatialite && \\\r\n    apt clean && \\\r\n    rm -rf /var/lib/{apt,dpkg,cache,log}/\r\n\r\nFROM python:3.9.2-slim as build\r\n\r\nRUN pip install datasette\r\n\r\n#COPY . /datasette\r\n#RUN pip install /datasette\r\n\r\nFROM python:3.9.2-slim\r\n\r\n# Copy python dependencies and spatialite libraries\r\nCOPY --from=build /usr/local/lib/ /usr/local/lib/\r\n# Copy executables\r\nCOPY --from=build /usr/local/bin /usr/local/bin\r\n# Copy spatial extensions\r\nCOPY --from=install_spatialite /usr/lib/x86_64-linux-gnu/mod_spatialite.so /usr/lib/x86_64-linux-gnu/mod_spatialite.so\r\n\r\nENV LD_LIBRARY_PATH=/usr/local/lib\r\n\r\nEXPOSE 8001\r\nCMD [\"datasette\"]\r\n```\r\nBut when I ran Datasette with `--load-extension=spatialite` I got this:\r\n```\r\n  File \"/usr/local/lib/python3.9/site-packages/datasette/database.py\", line 151, in in_thread\r\n    self.ds._prepare_connection(conn, self.name)\r\n  File \"/usr/local/lib/python3.9/site-packages/datasette/app.py\", line 502, in _prepare_connection\r\n    conn.execute(f\"SELECT load_extension('{extension}')\")\r\nsqlite3.OperationalError: /usr/lib/x86_64-linux-gnu/mod_spatialite.so.so: cannot open shared object file: No such file or directory\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 824064069, "label": "Updated Dockerfile with SpatiaLite version 5.0"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1249#issuecomment-803748469", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1249", "id": 803748469, "node_id": "MDEyOklzc3VlQ29tbWVudDgwMzc0ODQ2OQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-03-22T04:17:51Z", "updated_at": "2021-03-22T04:17:51Z", "author_association": "OWNER", "body": "... except my clever image using SpatiaLite installed for Ubuntu doesn't actually work:\r\n\r\n```\r\ndatasette % docker run -p 8001:8001 -v `pwd`:/mnt datasette-spatialite:latest datasette -p 8001 -h 0.0.0.0 /mnt/fixtures.db\r\n  File \"/usr/local/lib/python3.9/sqlite3/dbapi2.py\", line 27, in <module>\r\n    from _sqlite3 import *\r\nImportError: /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.29' not found (required by /usr/lib/x86_64-linux-gnu/libsqlite3.so.0)\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 824064069, "label": "Updated Dockerfile with SpatiaLite version 5.0"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1249#issuecomment-803748158", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1249", "id": 803748158, "node_id": "MDEyOklzc3VlQ29tbWVudDgwMzc0ODE1OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-03-22T04:16:57Z", "updated_at": "2021-03-22T04:16:57Z", "author_association": "OWNER", "body": "Which is great, because the image on Docker Hub right now is 383MB.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 824064069, "label": "Updated Dockerfile with SpatiaLite version 5.0"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1249#issuecomment-803747701", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1249", "id": 803747701, "node_id": "MDEyOklzc3VlQ29tbWVudDgwMzc0NzcwMQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-03-22T04:15:40Z", "updated_at": "2021-03-22T04:15:40Z", "author_association": "OWNER", "body": "Here's a trick: install SpatiaLite in `ubuntu:20.10` and then copy it into the final `python:3.9.2-slim` image.\r\n\r\n```dockerfile\r\nFROM ubuntu:20.10 as install_spatialite\r\n\r\nRUN apt update && \\\r\n    apt install -y libsqlite3-mod-spatialite && \\\r\n    apt clean && \\\r\n    rm -rf /var/lib/{apt,dpkg,cache,log}/\r\n\r\nFROM python:3.9.2-slim as build\r\n\r\nRUN pip install datasette\r\n\r\n#COPY . /datasette\r\n#RUN pip install /datasette\r\n\r\nFROM python:3.9.2-slim\r\n\r\n# Copy python dependencies and spatialite libraries\r\nCOPY --from=build /usr/local/lib/ /usr/local/lib/\r\n# Copy executables\r\nCOPY --from=build /usr/local/bin /usr/local/bin\r\n# Copy spatial extensions\r\nCOPY --from=install_spatialite /usr/lib/x86_64-linux-gnu /usr/lib/x86_64-linux-gnu\r\n\r\nENV LD_LIBRARY_PATH=/usr/local/lib\r\n\r\nEXPOSE 8001\r\nCMD [\"datasette\"]\r\n```\r\nThat produced a 265MB image.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 824064069, "label": "Updated Dockerfile with SpatiaLite version 5.0"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1249#issuecomment-803700940", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1249", "id": 803700940, "node_id": "MDEyOklzc3VlQ29tbWVudDgwMzcwMDk0MA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-03-22T01:14:24Z", "updated_at": "2021-03-22T01:14:24Z", "author_association": "OWNER", "body": "I tried that with just `python3-pip` (removing `libsqlite3-mod-spatialite`) and got 435MB.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 824064069, "label": "Updated Dockerfile with SpatiaLite version 5.0"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1249#issuecomment-803700626", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1249", "id": 803700626, "node_id": "MDEyOklzc3VlQ29tbWVudDgwMzcwMDYyNg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-03-22T01:13:04Z", "updated_at": "2021-03-22T01:13:04Z", "author_association": "OWNER", "body": "Building a Dockerfile containing just `FROM ubuntu:20.10` gave me `79.5MB`.\r\n\r\nBuilding this one:\r\n```dockerfile\r\nFROM ubuntu:20.10\r\n\r\n# Setup build dependencies\r\nRUN apt update && \\\r\n    apt install -y python3-pip libsqlite3-mod-spatialite && \\\r\n    apt clean && \\\r\n    rm -rf /var/lib/{apt,dpkg,cache,log}/\r\n```\r\nResulted in a 515MB image.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 824064069, "label": "Updated Dockerfile with SpatiaLite version 5.0"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1249#issuecomment-803698983", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1249", "id": 803698983, "node_id": "MDEyOklzc3VlQ29tbWVudDgwMzY5ODk4Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-03-22T01:05:36Z", "updated_at": "2021-03-22T01:06:23Z", "author_association": "OWNER", "body": "It's pretty big though. I tried this version which avoids copying junk from my laptop in:\r\n\r\n```dockerfile\r\nFROM ubuntu:20.10\r\n\r\n# Setup build dependencies\r\nRUN apt update && apt install -y python3-pip libsqlite3-mod-spatialite && apt clean\r\n\r\nRUN pip install datasette\r\n\r\nEXPOSE 8001\r\nCMD [\"datasette\"]\r\n```\r\nAnd got this:\r\n```\r\ndatasette % docker images datasette-spatialite    \r\nREPOSITORY             TAG       IMAGE ID       CREATED         SIZE\r\ndatasette-spatialite   latest    0796950653c2   2 seconds ago   528MB\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 824064069, "label": "Updated Dockerfile with SpatiaLite version 5.0"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1249#issuecomment-803698168", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1249", "id": 803698168, "node_id": "MDEyOklzc3VlQ29tbWVudDgwMzY5ODE2OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-03-22T01:02:02Z", "updated_at": "2021-03-22T01:02:30Z", "author_association": "OWNER", "body": "This is the shortest Dockerfile that appeared to give me a working SpatiaLite module:\r\n```dockerfile\r\nFROM ubuntu:20.10\r\n\r\n# Setup build dependencies\r\nRUN apt update && apt install -y python3-pip libsqlite3-mod-spatialite && apt clean\r\n\r\n# Add local code to the image instead of fetching from pypi.\r\nCOPY . /datasette\r\n\r\nRUN pip install /datasette\r\n\r\nRUN rm -rf /datasette\r\n\r\nEXPOSE 8001\r\nCMD [\"datasette\"]\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 824064069, "label": "Updated Dockerfile with SpatiaLite version 5.0"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1249#issuecomment-803697546", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1249", "id": 803697546, "node_id": "MDEyOklzc3VlQ29tbWVudDgwMzY5NzU0Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-03-22T00:59:47Z", "updated_at": "2021-03-22T00:59:47Z", "author_association": "OWNER", "body": "To debug I'm running:\r\n\r\n    docker run -it -p 8001:8001 -v `pwd`:/mnt datasette-spatialite:latest bash\r\n\r\nThis gets me a shell I can use.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 824064069, "label": "Updated Dockerfile with SpatiaLite version 5.0"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1249#issuecomment-803697211", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1249", "id": 803697211, "node_id": "MDEyOklzc3VlQ29tbWVudDgwMzY5NzIxMQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-03-22T00:58:01Z", "updated_at": "2021-03-22T00:58:01Z", "author_association": "OWNER", "body": "I'm messing around with the `Dockerfile` and after each change I'm running:\r\n\r\n    docker build . -t datasette-spatialite\r\n\r\nAnd then:\r\n\r\n    docker run -p 8001:8001 -v `pwd`:/mnt datasette-spatialite:latest datasette -p 8001 -h 0.0.0.0 /mnt/fixtures.db\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 824064069, "label": "Updated Dockerfile with SpatiaLite version 5.0"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1249#issuecomment-803694661", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1249", "id": 803694661, "node_id": "MDEyOklzc3VlQ29tbWVudDgwMzY5NDY2MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-03-22T00:46:49Z", "updated_at": "2021-03-22T00:46:49Z", "author_association": "OWNER", "body": "Actually for the loadable module I think I need https://packages.ubuntu.com/groovy/libsqlite3-mod-spatialite", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 824064069, "label": "Updated Dockerfile with SpatiaLite version 5.0"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1249#issuecomment-803694436", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1249", "id": 803694436, "node_id": "MDEyOklzc3VlQ29tbWVudDgwMzY5NDQzNg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-03-22T00:46:00Z", "updated_at": "2021-03-22T00:46:00Z", "author_association": "OWNER", "body": "So I'm going to try `20.10` and see where that gets me.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 824064069, "label": "Updated Dockerfile with SpatiaLite version 5.0"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1249#issuecomment-803694359", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1249", "id": 803694359, "node_id": "MDEyOklzc3VlQ29tbWVudDgwMzY5NDM1OQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-03-22T00:45:47Z", "updated_at": "2021-03-22T00:45:47Z", "author_association": "OWNER", "body": "https://pythonspeed.com/articles/base-image-python-docker-images/ suggests using `python:3.9-slim-buster` or `ubuntu:20.04` - but 20.04 is focal which still has SpatiaLite `4.3.0a-6build1` - It's `20.10` that has 5.0: https://packages.ubuntu.com/groovy/libspatialite-dev", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 824064069, "label": "Updated Dockerfile with SpatiaLite version 5.0"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1249#issuecomment-803693181", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1249", "id": 803693181, "node_id": "MDEyOklzc3VlQ29tbWVudDgwMzY5MzE4MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-03-22T00:41:02Z", "updated_at": "2021-03-22T00:41:02Z", "author_association": "OWNER", "body": "Debian sid has it too: https://packages.debian.org/sid/libspatialite-dev", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 824064069, "label": "Updated Dockerfile with SpatiaLite version 5.0"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1249#issuecomment-803692673", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1249", "id": 803692673, "node_id": "MDEyOklzc3VlQ29tbWVudDgwMzY5MjY3Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-03-22T00:38:42Z", "updated_at": "2021-03-22T00:38:42Z", "author_association": "OWNER", "body": "Ubuntu Groovy has a package for SpatiaLite 5 - I could try using that instead: https://packages.ubuntu.com/groovy/libspatialite-dev", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 824064069, "label": "Updated Dockerfile with SpatiaLite version 5.0"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1249#issuecomment-803691236", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1249", "id": 803691236, "node_id": "MDEyOklzc3VlQ29tbWVudDgwMzY5MTIzNg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-03-22T00:32:03Z", "updated_at": "2021-03-22T00:32:03Z", "author_association": "OWNER", "body": "Here's something odd: when I run `datasette tuscany_housenumbers.sqlite --load-extension=spatialite` on macOS against SpatiaLite installed using Homebrew (which reports `\"spatialite\": \"5.0.0\"` on the `/-/versions` page) I don't get any weird errors at all, everything works fine.\r\n\r\nBut when I tried compiling SpatiaLite inside the Docker container I had hanging errors on some pages.\r\n\r\nThis is using https://www.gaia-gis.it/gaia-sins/knn/tuscany_housenumbers.7z from the SpatiaLite KNN tutorial at https://www.gaia-gis.it/fossil/libspatialite/wiki?name=KNN", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 824064069, "label": "Updated Dockerfile with SpatiaLite version 5.0"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1249#issuecomment-792384854", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1249", "id": 792384854, "node_id": "MDEyOklzc3VlQ29tbWVudDc5MjM4NDg1NA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-03-08T00:23:38Z", "updated_at": "2021-03-08T00:23:38Z", "author_association": "OWNER", "body": "One reason to prioritize this issue: Homebrew upgraded to SpatiaLite 5.0 recently https://formulae.brew.sh/formula/spatialite-tools and as a result SpatiaLite database created on my laptop don't appear to be compatible with Datasette when published using `datasette publish`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 824064069, "label": "Updated Dockerfile with SpatiaLite version 5.0"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1249#issuecomment-792384382", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1249", "id": 792384382, "node_id": "MDEyOklzc3VlQ29tbWVudDc5MjM4NDM4Mg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-03-08T00:22:02Z", "updated_at": "2021-03-08T00:22:02Z", "author_association": "OWNER", "body": "I tried this patch against `Dockerfile`:\r\n```diff\r\ndiff --git a/Dockerfile b/Dockerfile\r\nindex f4b1414..dd659e1 100644\r\n--- a/Dockerfile\r\n+++ b/Dockerfile\r\n@@ -1,25 +1,26 @@\r\n-FROM python:3.7.10-slim-stretch as build\r\n+FROM python:3.9.2-slim-buster as build\r\n \r\n # Setup build dependencies\r\n RUN apt update \\\r\n-&& apt install -y python3-dev build-essential wget libxml2-dev libproj-dev libgeos-dev libsqlite3-dev zlib1g-dev pkg-config git \\\r\n- && apt clean\r\n+  && apt install -y python3-dev build-essential wget libxml2-dev libproj-dev \\\r\n+  libminizip-dev libgeos-dev libsqlite3-dev zlib1g-dev pkg-config git \\\r\n+  && apt clean\r\n \r\n-\r\n-RUN wget \"https://www.sqlite.org/2020/sqlite-autoconf-3310100.tar.gz\" && tar xzf sqlite-autoconf-3310100.tar.gz \\\r\n-    && cd sqlite-autoconf-3310100 && ./configure --disable-static --enable-fts5 --enable-json1 CFLAGS=\"-g -O2 -DSQLITE_ENABLE_FTS3=1 -DSQLITE_ENABLE_FTS3_PARENTHESIS -DSQLITE_ENABLE_FTS4=1 -DSQLITE_ENABLE_RTREE=1 -DSQLITE_ENABLE_JSON1\" \\\r\n+RUN wget \"https://www.sqlite.org/2021/sqlite-autoconf-3340100.tar.gz\" && tar xzf sqlite-autoconf-3340100.tar.gz \\\r\n+    && cd sqlite-autoconf-3340100 && ./configure --disable-static --enable-fts5 --enable-json1 \\\r\n+    CFLAGS=\"-g -O2 -DSQLITE_ENABLE_FTS3=1 -DSQLITE_ENABLE_FTS3_PARENTHESIS -DSQLITE_ENABLE_FTS4=1 -DSQLITE_ENABLE_RTREE=1 -DSQLITE_ENABLE_JSON1\" \\\r\n     && make && make install\r\n \r\n-RUN wget \"http://www.gaia-gis.it/gaia-sins/freexl-sources/freexl-1.0.5.tar.gz\" && tar zxf freexl-1.0.5.tar.gz \\\r\n-    && cd freexl-1.0.5 && ./configure && make && make install\r\n+RUN wget \"http://www.gaia-gis.it/gaia-sins/freexl-1.0.6.tar.gz\" && tar zxf freexl-1.0.6.tar.gz \\\r\n+    && cd freexl-1.0.6 && ./configure && make && make install\r\n \r\n-RUN wget \"http://www.gaia-gis.it/gaia-sins/libspatialite-sources/libspatialite-4.4.0-RC0.tar.gz\" && tar zxf libspatialite-4.4.0-RC0.tar.gz \\\r\n-    && cd libspatialite-4.4.0-RC0 && ./configure && make && make install\r\n+RUN wget \"http://www.gaia-gis.it/gaia-sins/libspatialite-5.0.1.tar.gz\" && tar zxf libspatialite-5.0.1.tar.gz \\\r\n+    && cd libspatialite-5.0.1 && ./configure --disable-rttopo && make && make install\r\n \r\n RUN wget \"http://www.gaia-gis.it/gaia-sins/readosm-sources/readosm-1.1.0.tar.gz\" && tar zxf readosm-1.1.0.tar.gz && cd readosm-1.1.0 && ./configure && make && make install\r\n \r\n-RUN wget \"http://www.gaia-gis.it/gaia-sins/spatialite-tools-sources/spatialite-tools-4.4.0-RC0.tar.gz\" && tar zxf spatialite-tools-4.4.0-RC0.tar.gz \\\r\n-    && cd spatialite-tools-4.4.0-RC0 && ./configure && make && make install\r\n+RUN wget \"http://www.gaia-gis.it/gaia-sins/spatialite-tools-5.0.0.tar.gz\" && tar zxf spatialite-tools-5.0.0.tar.gz \\\r\n+    && cd spatialite-tools-5.0.0 && ./configure --disable-rttopo && make && make install\r\n \r\n \r\n # Add local code to the image instead of fetching from pypi.\r\n@@ -27,7 +28,7 @@ COPY . /datasette\r\n \r\n RUN pip install /datasette\r\n \r\n-FROM python:3.7.10-slim-stretch\r\n+FROM python:3.9.2-slim-buster\r\n \r\n # Copy python dependencies and spatialite libraries\r\n COPY --from=build /usr/local/lib/ /usr/local/lib/\r\n```\r\n\r\nI had to use `--disable-rttopo` from the tip in https://github.com/OSGeo/gdal/pull/3443 and also needed to install `libminizip-dev`.\r\n\r\nThis works, sort of... I'm getting a weird issue where the `/dbname` page is hanging some of the time instead of loading correctly. Other than that it seems to work, but a hanging page is bad!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 824064069, "label": "Updated Dockerfile with SpatiaLite version 5.0"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1249#issuecomment-792383956", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1249", "id": 792383956, "node_id": "MDEyOklzc3VlQ29tbWVudDc5MjM4Mzk1Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-03-08T00:20:09Z", "updated_at": "2021-03-08T00:20:09Z", "author_association": "OWNER", "body": "Worth noting that the Docker image used by `datasette publish cloudrun` doesn't actually use that Datasette docker image - it does this:\r\n\r\nhttps://github.com/simonw/datasette/blob/d0fd833b8cdd97e1b91d0f97a69b494895d82bee/datasette/utils/__init__.py#L349-L353\r\n\r\nWhere the apt extras for SpatiaLite are: https://github.com/simonw/datasette/blob/d0fd833b8cdd97e1b91d0f97a69b494895d82bee/datasette/utils/__init__.py#L344-L345\r\n\r\n`libsqlite3-mod-spatialite` against that official `python:3.8` image doesn't appear to install SpatiaLite 5.0.\r\n\r\n\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 824064069, "label": "Updated Dockerfile with SpatiaLite version 5.0"}, "performed_via_github_app": null}