html_url,issue_url,id,node_id,user,user_label,created_at,updated_at,author_association,body,reactions,issue,issue_label,performed_via_github_app
https://github.com/simonw/datasette/issues/1249#issuecomment-804347152,https://api.github.com/repos/simonw/datasette/issues/1249,804347152,MDEyOklzc3VlQ29tbWVudDgwNDM0NzE1Mg==,9599,simonw,2021-03-22T19:47:56Z,2021-03-22T19:48:03Z,OWNER,I wrote a bunch of tips on creating smaller Docker images here: https://simonwillison.net/2018/Nov/19/smaller-python-docker-images/,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",824064069,Updated Dockerfile with SpatiaLite version 5.0,
https://github.com/simonw/datasette/issues/1249#issuecomment-804344553,https://api.github.com/repos/simonw/datasette/issues/1249,804344553,MDEyOklzc3VlQ29tbWVudDgwNDM0NDU1Mw==,9599,simonw,2021-03-22T19:43:25Z,2021-03-22T19:43:25Z,OWNER,Does `--no-install-recommends` make a difference?,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",824064069,Updated Dockerfile with SpatiaLite version 5.0,
https://github.com/simonw/datasette/issues/1249#issuecomment-804338678,https://api.github.com/repos/simonw/datasette/issues/1249,804338678,MDEyOklzc3VlQ29tbWVudDgwNDMzODY3OA==,9599,simonw,2021-03-22T19:33:43Z,2021-03-22T19:33:43Z,OWNER,"Replacing `rm -rf /var/lib/{apt,dpkg,cache,log}/` with
```
rm -rf /var/lib/apt && \
rm -rf /var/lib/dpkg
```
Got the size down to 305MB.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",824064069,Updated Dockerfile with SpatiaLite version 5.0,
https://github.com/simonw/datasette/issues/1249#issuecomment-804318314,https://api.github.com/repos/simonw/datasette/issues/1249,804318314,MDEyOklzc3VlQ29tbWVudDgwNDMxODMxNA==,9599,simonw,2021-03-22T19:04:30Z,2021-03-22T19:04:30Z,OWNER,Considering the image on Docker Hub right now is `383MB` this is actually an improvement.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",824064069,Updated Dockerfile with SpatiaLite version 5.0,
https://github.com/simonw/datasette/issues/1249#issuecomment-804317545,https://api.github.com/repos/simonw/datasette/issues/1249,804317545,MDEyOklzc3VlQ29tbWVudDgwNDMxNzU0NQ==,9599,simonw,2021-03-22T19:03:22Z,2021-03-22T19:03:22Z,OWNER,"This Dockerfile:
```dockerfile
FROM python:3.9.2-slim-buster as build
# software-properties-common provides add-apt-repository
RUN apt-get update && \
apt-get -y install software-properties-common && \
add-apt-repository ""deb http://httpredir.debian.org/debian sid main"" && \
apt-get update && \
apt-get -t sid install -y libsqlite3-mod-spatialite && \
apt clean && \
rm -rf /var/lib/{apt,dpkg,cache,log}/
RUN pip install datasette
EXPOSE 8001
CMD [""datasette""]
```
Produces a 344MB image that includes a working SpatiaLite 5.0 module. And weirdly... it doesn't exhibit the hanging bug!","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",824064069,Updated Dockerfile with SpatiaLite version 5.0,
https://github.com/simonw/datasette/issues/1249#issuecomment-804310353,https://api.github.com/repos/simonw/datasette/issues/1249,804310353,MDEyOklzc3VlQ29tbWVudDgwNDMxMDM1Mw==,9599,simonw,2021-03-22T18:52:12Z,2021-03-22T18:52:12Z,OWNER,"This Dockerfile:
```dockerfile
FROM python:3.9.2-slim-buster as build
# Setup build dependencies
RUN apt update \
&& apt install -y python3-dev build-essential wget libxml2-dev libproj-dev \
libminizip-dev libgeos-dev libsqlite3-dev zlib1g-dev pkg-config git \
&& apt clean
RUN wget ""https://www.sqlite.org/2021/sqlite-autoconf-3340100.tar.gz"" && tar xzf sqlite-autoconf-3340100.tar.gz \
&& cd sqlite-autoconf-3340100 && ./configure --disable-static --enable-fts5 --enable-json1 \
CFLAGS=""-g -O2 -DSQLITE_ENABLE_FTS3=1 -DSQLITE_ENABLE_FTS3_PARENTHESIS -DSQLITE_ENABLE_FTS4=1 -DSQLITE_ENABLE_RTREE=1 -DSQLITE_ENABLE_JSON1"" \
&& make && make install
RUN wget ""http://www.gaia-gis.it/gaia-sins/freexl-1.0.6.tar.gz"" && tar zxf freexl-1.0.6.tar.gz \
&& cd freexl-1.0.6 && ./configure && make && make install
RUN wget ""http://www.gaia-gis.it/gaia-sins/libspatialite-5.0.1.tar.gz"" && tar zxf libspatialite-5.0.1.tar.gz \
&& cd libspatialite-5.0.1 && ./configure --disable-rttopo && make && make install
RUN wget ""http://www.gaia-gis.it/gaia-sins/readosm-sources/readosm-1.1.0.tar.gz"" && tar zxf readosm-1.1.0.tar.gz && cd readosm-1.1.0 && ./configure && make && make install
RUN wget ""http://www.gaia-gis.it/gaia-sins/spatialite-tools-5.0.0.tar.gz"" && tar zxf spatialite-tools-5.0.0.tar.gz \
&& cd spatialite-tools-5.0.0 && ./configure --disable-rttopo && make && make install
# Add local code to the image instead of fetching from pypi.
#COPY . /datasette
#RUN pip install /datasette
RUN pip install datasette
FROM python:3.9.2-slim-buster
# Copy python dependencies and spatialite libraries
COPY --from=build /usr/local/lib/ /usr/local/lib/
# Copy executables
COPY --from=build /usr/local/bin /usr/local/bin
# Copy spatial extensions
COPY --from=build /usr/lib/x86_64-linux-gnu /usr/lib/x86_64-linux-gnu
ENV LD_LIBRARY_PATH=/usr/local/lib
EXPOSE 8001
CMD [""datasette""]
```
Produced a 448MB image.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",824064069,Updated Dockerfile with SpatiaLite version 5.0,
https://github.com/simonw/datasette/issues/1249#issuecomment-804309510,https://api.github.com/repos/simonw/datasette/issues/1249,804309510,MDEyOklzc3VlQ29tbWVudDgwNDMwOTUxMA==,9599,simonw,2021-03-22T18:50:50Z,2021-03-22T18:50:50Z,OWNER,"Ideally I'd like to use the Debian stable `python:3.9.2-slim-buster` base image but install SpatiaLite from Debian unstable here: https://packages.debian.org/sid/libspatialite7
This pattern might let me do that: https://github.com/helmesjo/cpp_bash_utils/blob/f031e926249f8e2d7f260f22dc8974c6d5be11fe/docker/images/linux-gcc.dockerfile#L20-L24","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",824064069,Updated Dockerfile with SpatiaLite version 5.0,
https://github.com/simonw/datasette/issues/1249#issuecomment-804263434,https://api.github.com/repos/simonw/datasette/issues/1249,804263434,MDEyOklzc3VlQ29tbWVudDgwNDI2MzQzNA==,9599,simonw,2021-03-22T17:43:25Z,2021-03-22T17:43:25Z,OWNER,I figured out the cause of the hang in #1268 - it was caused by `select count(*) from SpatialIndex` interacting badly with the `set_progress_handler()` mechanism I was using to implement query time limits. #1271 has a replacement for that using `asyncio.wait_for()` and `conn.interrupt()` which should resolve the SpatiaLite issue too.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",824064069,Updated Dockerfile with SpatiaLite version 5.0,
https://github.com/simonw/datasette/issues/1249#issuecomment-803755698,https://api.github.com/repos/simonw/datasette/issues/1249,803755698,MDEyOklzc3VlQ29tbWVudDgwMzc1NTY5OA==,9599,simonw,2021-03-22T04:43:02Z,2021-03-22T04:43:02Z,OWNER,I'll spin off a separate ticket to investigate the hang.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",824064069,Updated Dockerfile with SpatiaLite version 5.0,
https://github.com/simonw/datasette/issues/1249#issuecomment-803753388,https://api.github.com/repos/simonw/datasette/issues/1249,803753388,MDEyOklzc3VlQ29tbWVudDgwMzc1MzM4OA==,9599,simonw,2021-03-22T04:34:20Z,2021-03-22T04:35:10Z,OWNER,"Well this is frustrating. I finally found a Dockerfile that worked and installed an Ubuntu pre-compiled SpatiaLite module that would load...
```dockerfile
FROM ubuntu:20.10 as install_spatialite
RUN apt update && \
apt install -y libsqlite3-mod-spatialite && \
apt clean && \
rm -rf /var/lib/{apt,dpkg,cache,log}/
FROM ubuntu:20.10
RUN apt update && \
apt install -y python3-pip && \
apt clean && \
rm -rf /var/lib/{apt,dpkg,cache,log}/
RUN pip install datasette
# Copy spatial extensions
COPY --from=install_spatialite /usr/lib/x86_64-linux-gnu /usr/lib/x86_64-linux-gnu/
ENV LD_LIBRARY_PATH=/usr/local/lib
EXPOSE 8001
CMD [""datasette""]
```
(Which produced a 550MB image)
And when I ran Datasette I got that same error where the database listing page hangs!
```
docker run -p 8001:8001 -v `pwd`:/mnt datasette-spatialite:latest datasette -p 8001 -h 0.0.0.0 /mnt/tuscany_housenumbers.sqlite --load-extension=spatialite
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",824064069,Updated Dockerfile with SpatiaLite version 5.0,
https://github.com/simonw/datasette/issues/1249#issuecomment-803751068,https://api.github.com/repos/simonw/datasette/issues/1249,803751068,MDEyOklzc3VlQ29tbWVudDgwMzc1MTA2OA==,9599,simonw,2021-03-22T04:26:45Z,2021-03-22T04:26:45Z,OWNER,"Here's why:
```
datasette % docker run -it -p 8001:8001 -v `pwd`:/mnt datasette-spatialite:latest bash
root@3430352ff378:/# datasette
bash: /usr/local/bin/datasette: /usr/bin/python3: bad interpreter: No such file or directory
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",824064069,Updated Dockerfile with SpatiaLite version 5.0,
https://github.com/simonw/datasette/issues/1249#issuecomment-803750617,https://api.github.com/repos/simonw/datasette/issues/1249,803750617,MDEyOklzc3VlQ29tbWVudDgwMzc1MDYxNw==,9599,simonw,2021-03-22T04:25:14Z,2021-03-22T04:25:14Z,OWNER,"Got this error attempting to run Datasette (with or without SpatiaLite):
```
standard_init_linux.go:219: exec user process caused: no such file or directory
```
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",824064069,Updated Dockerfile with SpatiaLite version 5.0,
https://github.com/simonw/datasette/issues/1249#issuecomment-803750399,https://api.github.com/repos/simonw/datasette/issues/1249,803750399,MDEyOklzc3VlQ29tbWVudDgwMzc1MDM5OQ==,9599,simonw,2021-03-22T04:24:25Z,2021-03-22T04:24:25Z,OWNER,"I'll try using `ubuntu:20.10` for everything:
```dockerfile
FROM ubuntu:20.10 as install_spatialite
RUN apt update && \
apt install -y libsqlite3-mod-spatialite && \
apt clean && \
rm -rf /var/lib/{apt,dpkg,cache,log}/
FROM ubuntu:20.10 as build
RUN apt update && \
apt install -y python3-pip && \
apt clean && \
rm -rf /var/lib/{apt,dpkg,cache,log}/
RUN pip install datasette
#COPY . /datasette
#RUN pip install /datasette
FROM ubuntu:20.10
# Copy python dependencies and spatialite libraries
COPY --from=build /usr/local/lib/ /usr/local/lib/
# Copy executables
COPY --from=build /usr/local/bin /usr/local/bin
# Copy spatial extensions
COPY --from=install_spatialite /usr/lib/x86_64-linux-gnu/mod_spatialite.so /usr/lib/x86_64-linux-gnu/mod_spatialite.so
ENV LD_LIBRARY_PATH=/usr/local/lib
EXPOSE 8001
CMD [""datasette""]
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",824064069,Updated Dockerfile with SpatiaLite version 5.0,
https://github.com/simonw/datasette/issues/1249#issuecomment-803749831,https://api.github.com/repos/simonw/datasette/issues/1249,803749831,MDEyOklzc3VlQ29tbWVudDgwMzc0OTgzMQ==,9599,simonw,2021-03-22T04:22:35Z,2021-03-22T04:22:35Z,OWNER,"I tried copying just the `mod_spatialite.so` file:
```dockerfile
FROM ubuntu:20.10 as install_spatialite
RUN apt update && \
apt install -y libsqlite3-mod-spatialite && \
apt clean && \
rm -rf /var/lib/{apt,dpkg,cache,log}/
FROM python:3.9.2-slim as build
RUN pip install datasette
#COPY . /datasette
#RUN pip install /datasette
FROM python:3.9.2-slim
# Copy python dependencies and spatialite libraries
COPY --from=build /usr/local/lib/ /usr/local/lib/
# Copy executables
COPY --from=build /usr/local/bin /usr/local/bin
# Copy spatial extensions
COPY --from=install_spatialite /usr/lib/x86_64-linux-gnu/mod_spatialite.so /usr/lib/x86_64-linux-gnu/mod_spatialite.so
ENV LD_LIBRARY_PATH=/usr/local/lib
EXPOSE 8001
CMD [""datasette""]
```
But when I ran Datasette with `--load-extension=spatialite` I got this:
```
File ""/usr/local/lib/python3.9/site-packages/datasette/database.py"", line 151, in in_thread
self.ds._prepare_connection(conn, self.name)
File ""/usr/local/lib/python3.9/site-packages/datasette/app.py"", line 502, in _prepare_connection
conn.execute(f""SELECT load_extension('{extension}')"")
sqlite3.OperationalError: /usr/lib/x86_64-linux-gnu/mod_spatialite.so.so: cannot open shared object file: No such file or directory
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",824064069,Updated Dockerfile with SpatiaLite version 5.0,
https://github.com/simonw/datasette/issues/1249#issuecomment-803748469,https://api.github.com/repos/simonw/datasette/issues/1249,803748469,MDEyOklzc3VlQ29tbWVudDgwMzc0ODQ2OQ==,9599,simonw,2021-03-22T04:17:51Z,2021-03-22T04:17:51Z,OWNER,"... except my clever image using SpatiaLite installed for Ubuntu doesn't actually work:
```
datasette % docker run -p 8001:8001 -v `pwd`:/mnt datasette-spatialite:latest datasette -p 8001 -h 0.0.0.0 /mnt/fixtures.db
File ""/usr/local/lib/python3.9/sqlite3/dbapi2.py"", line 27, in
from _sqlite3 import *
ImportError: /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.29' not found (required by /usr/lib/x86_64-linux-gnu/libsqlite3.so.0)
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",824064069,Updated Dockerfile with SpatiaLite version 5.0,
https://github.com/simonw/datasette/issues/1249#issuecomment-803748158,https://api.github.com/repos/simonw/datasette/issues/1249,803748158,MDEyOklzc3VlQ29tbWVudDgwMzc0ODE1OA==,9599,simonw,2021-03-22T04:16:57Z,2021-03-22T04:16:57Z,OWNER,"Which is great, because the image on Docker Hub right now is 383MB.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",824064069,Updated Dockerfile with SpatiaLite version 5.0,
https://github.com/simonw/datasette/issues/1249#issuecomment-803747701,https://api.github.com/repos/simonw/datasette/issues/1249,803747701,MDEyOklzc3VlQ29tbWVudDgwMzc0NzcwMQ==,9599,simonw,2021-03-22T04:15:40Z,2021-03-22T04:15:40Z,OWNER,"Here's a trick: install SpatiaLite in `ubuntu:20.10` and then copy it into the final `python:3.9.2-slim` image.
```dockerfile
FROM ubuntu:20.10 as install_spatialite
RUN apt update && \
apt install -y libsqlite3-mod-spatialite && \
apt clean && \
rm -rf /var/lib/{apt,dpkg,cache,log}/
FROM python:3.9.2-slim as build
RUN pip install datasette
#COPY . /datasette
#RUN pip install /datasette
FROM python:3.9.2-slim
# Copy python dependencies and spatialite libraries
COPY --from=build /usr/local/lib/ /usr/local/lib/
# Copy executables
COPY --from=build /usr/local/bin /usr/local/bin
# Copy spatial extensions
COPY --from=install_spatialite /usr/lib/x86_64-linux-gnu /usr/lib/x86_64-linux-gnu
ENV LD_LIBRARY_PATH=/usr/local/lib
EXPOSE 8001
CMD [""datasette""]
```
That produced a 265MB image.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",824064069,Updated Dockerfile with SpatiaLite version 5.0,
https://github.com/simonw/datasette/issues/1249#issuecomment-803700940,https://api.github.com/repos/simonw/datasette/issues/1249,803700940,MDEyOklzc3VlQ29tbWVudDgwMzcwMDk0MA==,9599,simonw,2021-03-22T01:14:24Z,2021-03-22T01:14:24Z,OWNER,I tried that with just `python3-pip` (removing `libsqlite3-mod-spatialite`) and got 435MB.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",824064069,Updated Dockerfile with SpatiaLite version 5.0,
https://github.com/simonw/datasette/issues/1249#issuecomment-803700626,https://api.github.com/repos/simonw/datasette/issues/1249,803700626,MDEyOklzc3VlQ29tbWVudDgwMzcwMDYyNg==,9599,simonw,2021-03-22T01:13:04Z,2021-03-22T01:13:04Z,OWNER,"Building a Dockerfile containing just `FROM ubuntu:20.10` gave me `79.5MB`.
Building this one:
```dockerfile
FROM ubuntu:20.10
# Setup build dependencies
RUN apt update && \
apt install -y python3-pip libsqlite3-mod-spatialite && \
apt clean && \
rm -rf /var/lib/{apt,dpkg,cache,log}/
```
Resulted in a 515MB image.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",824064069,Updated Dockerfile with SpatiaLite version 5.0,
https://github.com/simonw/datasette/issues/1249#issuecomment-803698983,https://api.github.com/repos/simonw/datasette/issues/1249,803698983,MDEyOklzc3VlQ29tbWVudDgwMzY5ODk4Mw==,9599,simonw,2021-03-22T01:05:36Z,2021-03-22T01:06:23Z,OWNER,"It's pretty big though. I tried this version which avoids copying junk from my laptop in:
```dockerfile
FROM ubuntu:20.10
# Setup build dependencies
RUN apt update && apt install -y python3-pip libsqlite3-mod-spatialite && apt clean
RUN pip install datasette
EXPOSE 8001
CMD [""datasette""]
```
And got this:
```
datasette % docker images datasette-spatialite
REPOSITORY TAG IMAGE ID CREATED SIZE
datasette-spatialite latest 0796950653c2 2 seconds ago 528MB
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",824064069,Updated Dockerfile with SpatiaLite version 5.0,
https://github.com/simonw/datasette/issues/1249#issuecomment-803698168,https://api.github.com/repos/simonw/datasette/issues/1249,803698168,MDEyOklzc3VlQ29tbWVudDgwMzY5ODE2OA==,9599,simonw,2021-03-22T01:02:02Z,2021-03-22T01:02:30Z,OWNER,"This is the shortest Dockerfile that appeared to give me a working SpatiaLite module:
```dockerfile
FROM ubuntu:20.10
# Setup build dependencies
RUN apt update && apt install -y python3-pip libsqlite3-mod-spatialite && apt clean
# Add local code to the image instead of fetching from pypi.
COPY . /datasette
RUN pip install /datasette
RUN rm -rf /datasette
EXPOSE 8001
CMD [""datasette""]
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",824064069,Updated Dockerfile with SpatiaLite version 5.0,
https://github.com/simonw/datasette/issues/1249#issuecomment-803697546,https://api.github.com/repos/simonw/datasette/issues/1249,803697546,MDEyOklzc3VlQ29tbWVudDgwMzY5NzU0Ng==,9599,simonw,2021-03-22T00:59:47Z,2021-03-22T00:59:47Z,OWNER,"To debug I'm running:
docker run -it -p 8001:8001 -v `pwd`:/mnt datasette-spatialite:latest bash
This gets me a shell I can use.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",824064069,Updated Dockerfile with SpatiaLite version 5.0,
https://github.com/simonw/datasette/issues/1249#issuecomment-803697211,https://api.github.com/repos/simonw/datasette/issues/1249,803697211,MDEyOklzc3VlQ29tbWVudDgwMzY5NzIxMQ==,9599,simonw,2021-03-22T00:58:01Z,2021-03-22T00:58:01Z,OWNER,"I'm messing around with the `Dockerfile` and after each change I'm running:
docker build . -t datasette-spatialite
And then:
docker run -p 8001:8001 -v `pwd`:/mnt datasette-spatialite:latest datasette -p 8001 -h 0.0.0.0 /mnt/fixtures.db
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",824064069,Updated Dockerfile with SpatiaLite version 5.0,
https://github.com/simonw/datasette/issues/1249#issuecomment-803694661,https://api.github.com/repos/simonw/datasette/issues/1249,803694661,MDEyOklzc3VlQ29tbWVudDgwMzY5NDY2MQ==,9599,simonw,2021-03-22T00:46:49Z,2021-03-22T00:46:49Z,OWNER,Actually for the loadable module I think I need https://packages.ubuntu.com/groovy/libsqlite3-mod-spatialite,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",824064069,Updated Dockerfile with SpatiaLite version 5.0,
https://github.com/simonw/datasette/issues/1249#issuecomment-803694436,https://api.github.com/repos/simonw/datasette/issues/1249,803694436,MDEyOklzc3VlQ29tbWVudDgwMzY5NDQzNg==,9599,simonw,2021-03-22T00:46:00Z,2021-03-22T00:46:00Z,OWNER,So I'm going to try `20.10` and see where that gets me.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",824064069,Updated Dockerfile with SpatiaLite version 5.0,
https://github.com/simonw/datasette/issues/1249#issuecomment-803694359,https://api.github.com/repos/simonw/datasette/issues/1249,803694359,MDEyOklzc3VlQ29tbWVudDgwMzY5NDM1OQ==,9599,simonw,2021-03-22T00:45:47Z,2021-03-22T00:45:47Z,OWNER,https://pythonspeed.com/articles/base-image-python-docker-images/ suggests using `python:3.9-slim-buster` or `ubuntu:20.04` - but 20.04 is focal which still has SpatiaLite `4.3.0a-6build1` - It's `20.10` that has 5.0: https://packages.ubuntu.com/groovy/libspatialite-dev,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",824064069,Updated Dockerfile with SpatiaLite version 5.0,
https://github.com/simonw/datasette/issues/1249#issuecomment-803693181,https://api.github.com/repos/simonw/datasette/issues/1249,803693181,MDEyOklzc3VlQ29tbWVudDgwMzY5MzE4MQ==,9599,simonw,2021-03-22T00:41:02Z,2021-03-22T00:41:02Z,OWNER,Debian sid has it too: https://packages.debian.org/sid/libspatialite-dev,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",824064069,Updated Dockerfile with SpatiaLite version 5.0,
https://github.com/simonw/datasette/issues/1249#issuecomment-803692673,https://api.github.com/repos/simonw/datasette/issues/1249,803692673,MDEyOklzc3VlQ29tbWVudDgwMzY5MjY3Mw==,9599,simonw,2021-03-22T00:38:42Z,2021-03-22T00:38:42Z,OWNER,Ubuntu Groovy has a package for SpatiaLite 5 - I could try using that instead: https://packages.ubuntu.com/groovy/libspatialite-dev,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",824064069,Updated Dockerfile with SpatiaLite version 5.0,
https://github.com/simonw/datasette/issues/1249#issuecomment-803691236,https://api.github.com/repos/simonw/datasette/issues/1249,803691236,MDEyOklzc3VlQ29tbWVudDgwMzY5MTIzNg==,9599,simonw,2021-03-22T00:32:03Z,2021-03-22T00:32:03Z,OWNER,"Here's something odd: when I run `datasette tuscany_housenumbers.sqlite --load-extension=spatialite` on macOS against SpatiaLite installed using Homebrew (which reports `""spatialite"": ""5.0.0""` on the `/-/versions` page) I don't get any weird errors at all, everything works fine.
But when I tried compiling SpatiaLite inside the Docker container I had hanging errors on some pages.
This is using https://www.gaia-gis.it/gaia-sins/knn/tuscany_housenumbers.7z from the SpatiaLite KNN tutorial at https://www.gaia-gis.it/fossil/libspatialite/wiki?name=KNN","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",824064069,Updated Dockerfile with SpatiaLite version 5.0,
https://github.com/simonw/datasette/issues/1249#issuecomment-792384854,https://api.github.com/repos/simonw/datasette/issues/1249,792384854,MDEyOklzc3VlQ29tbWVudDc5MjM4NDg1NA==,9599,simonw,2021-03-08T00:23:38Z,2021-03-08T00:23:38Z,OWNER,One reason to prioritize this issue: Homebrew upgraded to SpatiaLite 5.0 recently https://formulae.brew.sh/formula/spatialite-tools and as a result SpatiaLite database created on my laptop don't appear to be compatible with Datasette when published using `datasette publish`.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",824064069,Updated Dockerfile with SpatiaLite version 5.0,
https://github.com/simonw/datasette/issues/1249#issuecomment-792384382,https://api.github.com/repos/simonw/datasette/issues/1249,792384382,MDEyOklzc3VlQ29tbWVudDc5MjM4NDM4Mg==,9599,simonw,2021-03-08T00:22:02Z,2021-03-08T00:22:02Z,OWNER,"I tried this patch against `Dockerfile`:
```diff
diff --git a/Dockerfile b/Dockerfile
index f4b1414..dd659e1 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -1,25 +1,26 @@
-FROM python:3.7.10-slim-stretch as build
+FROM python:3.9.2-slim-buster as build
# Setup build dependencies
RUN apt update \
-&& apt install -y python3-dev build-essential wget libxml2-dev libproj-dev libgeos-dev libsqlite3-dev zlib1g-dev pkg-config git \
- && apt clean
+ && apt install -y python3-dev build-essential wget libxml2-dev libproj-dev \
+ libminizip-dev libgeos-dev libsqlite3-dev zlib1g-dev pkg-config git \
+ && apt clean
-
-RUN wget ""https://www.sqlite.org/2020/sqlite-autoconf-3310100.tar.gz"" && tar xzf sqlite-autoconf-3310100.tar.gz \
- && cd sqlite-autoconf-3310100 && ./configure --disable-static --enable-fts5 --enable-json1 CFLAGS=""-g -O2 -DSQLITE_ENABLE_FTS3=1 -DSQLITE_ENABLE_FTS3_PARENTHESIS -DSQLITE_ENABLE_FTS4=1 -DSQLITE_ENABLE_RTREE=1 -DSQLITE_ENABLE_JSON1"" \
+RUN wget ""https://www.sqlite.org/2021/sqlite-autoconf-3340100.tar.gz"" && tar xzf sqlite-autoconf-3340100.tar.gz \
+ && cd sqlite-autoconf-3340100 && ./configure --disable-static --enable-fts5 --enable-json1 \
+ CFLAGS=""-g -O2 -DSQLITE_ENABLE_FTS3=1 -DSQLITE_ENABLE_FTS3_PARENTHESIS -DSQLITE_ENABLE_FTS4=1 -DSQLITE_ENABLE_RTREE=1 -DSQLITE_ENABLE_JSON1"" \
&& make && make install
-RUN wget ""http://www.gaia-gis.it/gaia-sins/freexl-sources/freexl-1.0.5.tar.gz"" && tar zxf freexl-1.0.5.tar.gz \
- && cd freexl-1.0.5 && ./configure && make && make install
+RUN wget ""http://www.gaia-gis.it/gaia-sins/freexl-1.0.6.tar.gz"" && tar zxf freexl-1.0.6.tar.gz \
+ && cd freexl-1.0.6 && ./configure && make && make install
-RUN wget ""http://www.gaia-gis.it/gaia-sins/libspatialite-sources/libspatialite-4.4.0-RC0.tar.gz"" && tar zxf libspatialite-4.4.0-RC0.tar.gz \
- && cd libspatialite-4.4.0-RC0 && ./configure && make && make install
+RUN wget ""http://www.gaia-gis.it/gaia-sins/libspatialite-5.0.1.tar.gz"" && tar zxf libspatialite-5.0.1.tar.gz \
+ && cd libspatialite-5.0.1 && ./configure --disable-rttopo && make && make install
RUN wget ""http://www.gaia-gis.it/gaia-sins/readosm-sources/readosm-1.1.0.tar.gz"" && tar zxf readosm-1.1.0.tar.gz && cd readosm-1.1.0 && ./configure && make && make install
-RUN wget ""http://www.gaia-gis.it/gaia-sins/spatialite-tools-sources/spatialite-tools-4.4.0-RC0.tar.gz"" && tar zxf spatialite-tools-4.4.0-RC0.tar.gz \
- && cd spatialite-tools-4.4.0-RC0 && ./configure && make && make install
+RUN wget ""http://www.gaia-gis.it/gaia-sins/spatialite-tools-5.0.0.tar.gz"" && tar zxf spatialite-tools-5.0.0.tar.gz \
+ && cd spatialite-tools-5.0.0 && ./configure --disable-rttopo && make && make install
# Add local code to the image instead of fetching from pypi.
@@ -27,7 +28,7 @@ COPY . /datasette
RUN pip install /datasette
-FROM python:3.7.10-slim-stretch
+FROM python:3.9.2-slim-buster
# Copy python dependencies and spatialite libraries
COPY --from=build /usr/local/lib/ /usr/local/lib/
```
I had to use `--disable-rttopo` from the tip in https://github.com/OSGeo/gdal/pull/3443 and also needed to install `libminizip-dev`.
This works, sort of... I'm getting a weird issue where the `/dbname` page is hanging some of the time instead of loading correctly. Other than that it seems to work, but a hanging page is bad!","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",824064069,Updated Dockerfile with SpatiaLite version 5.0,
https://github.com/simonw/datasette/issues/1249#issuecomment-792383956,https://api.github.com/repos/simonw/datasette/issues/1249,792383956,MDEyOklzc3VlQ29tbWVudDc5MjM4Mzk1Ng==,9599,simonw,2021-03-08T00:20:09Z,2021-03-08T00:20:09Z,OWNER,"Worth noting that the Docker image used by `datasette publish cloudrun` doesn't actually use that Datasette docker image - it does this:
https://github.com/simonw/datasette/blob/d0fd833b8cdd97e1b91d0f97a69b494895d82bee/datasette/utils/__init__.py#L349-L353
Where the apt extras for SpatiaLite are: https://github.com/simonw/datasette/blob/d0fd833b8cdd97e1b91d0f97a69b494895d82bee/datasette/utils/__init__.py#L344-L345
`libsqlite3-mod-spatialite` against that official `python:3.8` image doesn't appear to install SpatiaLite 5.0.
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",824064069,Updated Dockerfile with SpatiaLite version 5.0,