issue_comments
8,358 rows where author_association = "OWNER" and user = 9599 sorted by updated_at descending
This data as json, CSV (advanced)
user 1
- simonw · 4,309 ✖
id | html_url | issue_url | node_id | user | created_at | updated_at ▲ | author_association | body | reactions | issue | performed_via_github_app |
---|---|---|---|---|---|---|---|---|---|---|---|
804347152 | https://github.com/simonw/datasette/issues/1249#issuecomment-804347152 | https://api.github.com/repos/simonw/datasette/issues/1249 | MDEyOklzc3VlQ29tbWVudDgwNDM0NzE1Mg== | simonw 9599 | 2021-03-22T19:47:56Z | 2021-03-22T19:48:03Z | OWNER | I wrote a bunch of tips on creating smaller Docker images here: https://simonwillison.net/2018/Nov/19/smaller-python-docker-images/ |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Updated Dockerfile with SpatiaLite version 5.0 824064069 | |
804344553 | https://github.com/simonw/datasette/issues/1249#issuecomment-804344553 | https://api.github.com/repos/simonw/datasette/issues/1249 | MDEyOklzc3VlQ29tbWVudDgwNDM0NDU1Mw== | simonw 9599 | 2021-03-22T19:43:25Z | 2021-03-22T19:43:25Z | OWNER | Does |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Updated Dockerfile with SpatiaLite version 5.0 824064069 | |
804338678 | https://github.com/simonw/datasette/issues/1249#issuecomment-804338678 | https://api.github.com/repos/simonw/datasette/issues/1249 | MDEyOklzc3VlQ29tbWVudDgwNDMzODY3OA== | simonw 9599 | 2021-03-22T19:33:43Z | 2021-03-22T19:33:43Z | OWNER | Replacing |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Updated Dockerfile with SpatiaLite version 5.0 824064069 | |
804318314 | https://github.com/simonw/datasette/issues/1249#issuecomment-804318314 | https://api.github.com/repos/simonw/datasette/issues/1249 | MDEyOklzc3VlQ29tbWVudDgwNDMxODMxNA== | simonw 9599 | 2021-03-22T19:04:30Z | 2021-03-22T19:04:30Z | OWNER | Considering the image on Docker Hub right now is |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Updated Dockerfile with SpatiaLite version 5.0 824064069 | |
804317545 | https://github.com/simonw/datasette/issues/1249#issuecomment-804317545 | https://api.github.com/repos/simonw/datasette/issues/1249 | MDEyOklzc3VlQ29tbWVudDgwNDMxNzU0NQ== | simonw 9599 | 2021-03-22T19:03:22Z | 2021-03-22T19:03:22Z | OWNER | This Dockerfile: ```dockerfile FROM python:3.9.2-slim-buster as build software-properties-common provides add-apt-repositoryRUN apt-get update && \ apt-get -y install software-properties-common && \ add-apt-repository "deb http://httpredir.debian.org/debian sid main" && \ apt-get update && \ apt-get -t sid install -y libsqlite3-mod-spatialite && \ apt clean && \ rm -rf /var/lib/{apt,dpkg,cache,log}/ RUN pip install datasette EXPOSE 8001 CMD ["datasette"] ``` Produces a 344MB image that includes a working SpatiaLite 5.0 module. And weirdly... it doesn't exhibit the hanging bug! |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Updated Dockerfile with SpatiaLite version 5.0 824064069 | |
804310353 | https://github.com/simonw/datasette/issues/1249#issuecomment-804310353 | https://api.github.com/repos/simonw/datasette/issues/1249 | MDEyOklzc3VlQ29tbWVudDgwNDMxMDM1Mw== | simonw 9599 | 2021-03-22T18:52:12Z | 2021-03-22T18:52:12Z | OWNER | This Dockerfile: ```dockerfile FROM python:3.9.2-slim-buster as build Setup build dependenciesRUN apt update \ && apt install -y python3-dev build-essential wget libxml2-dev libproj-dev \ libminizip-dev libgeos-dev libsqlite3-dev zlib1g-dev pkg-config git \ && apt clean RUN wget "https://www.sqlite.org/2021/sqlite-autoconf-3340100.tar.gz" && tar xzf sqlite-autoconf-3340100.tar.gz \ && cd sqlite-autoconf-3340100 && ./configure --disable-static --enable-fts5 --enable-json1 \ CFLAGS="-g -O2 -DSQLITE_ENABLE_FTS3=1 -DSQLITE_ENABLE_FTS3_PARENTHESIS -DSQLITE_ENABLE_FTS4=1 -DSQLITE_ENABLE_RTREE=1 -DSQLITE_ENABLE_JSON1" \ && make && make install RUN wget "http://www.gaia-gis.it/gaia-sins/freexl-1.0.6.tar.gz" && tar zxf freexl-1.0.6.tar.gz \ && cd freexl-1.0.6 && ./configure && make && make install RUN wget "http://www.gaia-gis.it/gaia-sins/libspatialite-5.0.1.tar.gz" && tar zxf libspatialite-5.0.1.tar.gz \ && cd libspatialite-5.0.1 && ./configure --disable-rttopo && make && make install RUN wget "http://www.gaia-gis.it/gaia-sins/readosm-sources/readosm-1.1.0.tar.gz" && tar zxf readosm-1.1.0.tar.gz && cd readosm-1.1.0 && ./configure && make && make install RUN wget "http://www.gaia-gis.it/gaia-sins/spatialite-tools-5.0.0.tar.gz" && tar zxf spatialite-tools-5.0.0.tar.gz \ && cd spatialite-tools-5.0.0 && ./configure --disable-rttopo && make && make install Add local code to the image instead of fetching from pypi.COPY . /datasetteRUN pip install /datasetteRUN pip install datasette FROM python:3.9.2-slim-buster Copy python dependencies and spatialite librariesCOPY --from=build /usr/local/lib/ /usr/local/lib/ Copy executablesCOPY --from=build /usr/local/bin /usr/local/bin Copy spatial extensionsCOPY --from=build /usr/lib/x86_64-linux-gnu /usr/lib/x86_64-linux-gnu ENV LD_LIBRARY_PATH=/usr/local/lib EXPOSE 8001 CMD ["datasette"] ``` Produced a 448MB image. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Updated Dockerfile with SpatiaLite version 5.0 824064069 | |
804309510 | https://github.com/simonw/datasette/issues/1249#issuecomment-804309510 | https://api.github.com/repos/simonw/datasette/issues/1249 | MDEyOklzc3VlQ29tbWVudDgwNDMwOTUxMA== | simonw 9599 | 2021-03-22T18:50:50Z | 2021-03-22T18:50:50Z | OWNER | Ideally I'd like to use the Debian stable This pattern might let me do that: https://github.com/helmesjo/cpp_bash_utils/blob/f031e926249f8e2d7f260f22dc8974c6d5be11fe/docker/images/linux-gcc.dockerfile#L20-L24 |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Updated Dockerfile with SpatiaLite version 5.0 824064069 | |
804265042 | https://github.com/simonw/datasette/pull/1271#issuecomment-804265042 | https://api.github.com/repos/simonw/datasette/issues/1271 | MDEyOklzc3VlQ29tbWVudDgwNDI2NTA0Mg== | simonw 9599 | 2021-03-22T17:45:45Z | 2021-03-22T17:45:45Z | OWNER | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Use SQLite conn.interrupt() instead of sqlite_timelimit() 837956424 | ||
804263434 | https://github.com/simonw/datasette/issues/1249#issuecomment-804263434 | https://api.github.com/repos/simonw/datasette/issues/1249 | MDEyOklzc3VlQ29tbWVudDgwNDI2MzQzNA== | simonw 9599 | 2021-03-22T17:43:25Z | 2021-03-22T17:43:25Z | OWNER | I figured out the cause of the hang in #1268 - it was caused by |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Updated Dockerfile with SpatiaLite version 5.0 824064069 | |
804261915 | https://github.com/simonw/datasette/issues/1268#issuecomment-804261915 | https://api.github.com/repos/simonw/datasette/issues/1268 | MDEyOklzc3VlQ29tbWVudDgwNDI2MTkxNQ== | simonw 9599 | 2021-03-22T17:41:12Z | 2021-03-22T17:41:12Z | OWNER | Closing this because I've figured out the root of the problem now, and I have a potential solution. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Figure out why SpatiaLite 5.0 hangs the database page on Linux 837308703 | |
804261610 | https://github.com/simonw/datasette/issues/1269#issuecomment-804261610 | https://api.github.com/repos/simonw/datasette/issues/1269 | MDEyOklzc3VlQ29tbWVudDgwNDI2MTYxMA== | simonw 9599 | 2021-03-22T17:40:41Z | 2021-03-22T17:40:41Z | OWNER | 1270 looks promising, and I don't want to leave open a security hole where someone could potentially hang Datasette with a nasty |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Don't attempt to run count(*) against virtual tables 837348479 | |
804255633 | https://github.com/simonw/datasette/issues/1270#issuecomment-804255633 | https://api.github.com/repos/simonw/datasette/issues/1270 | MDEyOklzc3VlQ29tbWVudDgwNDI1NTYzMw== | simonw 9599 | 2021-03-22T17:32:02Z | 2021-03-22T17:32:08Z | OWNER | Confirmed that the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Try implementing SQLite timeouts using .interrupt() instead of using .set_progress_handler() 837350092 | |
803834784 | https://github.com/simonw/datasette/issues/1270#issuecomment-803834784 | https://api.github.com/repos/simonw/datasette/issues/1270 | MDEyOklzc3VlQ29tbWVudDgwMzgzNDc4NA== | simonw 9599 | 2021-03-22T07:31:57Z | 2021-03-22T16:22:19Z | OWNER | I think the implementation for this goes here: https://github.com/simonw/datasette/blob/6f41c8a2bef309a66588b2875c3e24d26adb4850/datasette/database.py#L146-L157 I figured out a similar pattern in |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Try implementing SQLite timeouts using .interrupt() instead of using .set_progress_handler() 837350092 | |
803802957 | https://github.com/simonw/datasette/issues/1268#issuecomment-803802957 | https://api.github.com/repos/simonw/datasette/issues/1268 | MDEyOklzc3VlQ29tbWVudDgwMzgwMjk1Nw== | simonw 9599 | 2021-03-22T06:38:14Z | 2021-03-22T06:38:14Z | OWNER | Also worth trying is to change this code:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Figure out why SpatiaLite 5.0 hangs the database page on Linux 837308703 | |
803785808 | https://github.com/simonw/datasette/issues/1269#issuecomment-803785808 | https://api.github.com/repos/simonw/datasette/issues/1269 | MDEyOklzc3VlQ29tbWVudDgwMzc4NTgwOA== | simonw 9599 | 2021-03-22T06:00:53Z | 2021-03-22T06:00:53Z | OWNER | This may not be necessary if using `.interrupt() for SQLite timeouts in #1270 works. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Don't attempt to run count(*) against virtual tables 837348479 | |
803784902 | https://github.com/simonw/datasette/issues/1268#issuecomment-803784902 | https://api.github.com/repos/simonw/datasette/issues/1268 | MDEyOklzc3VlQ29tbWVudDgwMzc4NDkwMg== | simonw 9599 | 2021-03-22T05:59:06Z | 2021-03-22T05:59:06Z | OWNER | Even if I implement that workaround in #1269 I'm concerned that this could still allow users to deliberately crash Datasette (if it's running SpatiaLite 5.0) by executing That |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Figure out why SpatiaLite 5.0 hangs the database page on Linux 837308703 | |
803782705 | https://github.com/simonw/datasette/issues/1268#issuecomment-803782705 | https://api.github.com/repos/simonw/datasette/issues/1268 | MDEyOklzc3VlQ29tbWVudDgwMzc4MjcwNQ== | simonw 9599 | 2021-03-22T05:54:19Z | 2021-03-22T05:54:19Z | OWNER | Got two new TILs out of this: |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Figure out why SpatiaLite 5.0 hangs the database page on Linux 837308703 | |
803777724 | https://github.com/simonw/datasette/issues/1268#issuecomment-803777724 | https://api.github.com/repos/simonw/datasette/issues/1268 | MDEyOklzc3VlQ29tbWVudDgwMzc3NzcyNA== | simonw 9599 | 2021-03-22T05:42:50Z | 2021-03-22T05:43:23Z | OWNER | If I want to avoid counting virtual tables, I need to detect which tables are virtual tables. The safest way to do this is probably to pull the This would catch things like |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Figure out why SpatiaLite 5.0 hangs the database page on Linux 837308703 | |
803775121 | https://github.com/simonw/datasette/issues/1268#issuecomment-803775121 | https://api.github.com/repos/simonw/datasette/issues/1268 | MDEyOklzc3VlQ29tbWVudDgwMzc3NTEyMQ== | simonw 9599 | 2021-03-22T05:36:26Z | 2021-03-22T05:36:26Z | OWNER | So one fix could be to avoid running counts for anything that turns out to be a virtual table. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Figure out why SpatiaLite 5.0 hangs the database page on Linux 837308703 | |
803774926 | https://github.com/simonw/datasette/issues/1268#issuecomment-803774926 | https://api.github.com/repos/simonw/datasette/issues/1268 | MDEyOklzc3VlQ29tbWVudDgwMzc3NDkyNg== | simonw 9599 | 2021-03-22T05:35:56Z | 2021-03-22T05:35:56Z | OWNER | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Figure out why SpatiaLite 5.0 hangs the database page on Linux 837308703 | ||
803774518 | https://github.com/simonw/datasette/issues/1268#issuecomment-803774518 | https://api.github.com/repos/simonw/datasette/issues/1268 | MDEyOklzc3VlQ29tbWVudDgwMzc3NDUxOA== | simonw 9599 | 2021-03-22T05:34:57Z | 2021-03-22T05:34:57Z | OWNER | ... and sure enough, adding this code fixed the problem:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Figure out why SpatiaLite 5.0 hangs the database page on Linux 837308703 | |
803773484 | https://github.com/simonw/datasette/issues/1268#issuecomment-803773484 | https://api.github.com/repos/simonw/datasette/issues/1268 | MDEyOklzc3VlQ29tbWVudDgwMzc3MzQ4NA== | simonw 9599 | 2021-03-22T05:32:29Z | 2021-03-22T05:32:29Z | OWNER | To figure out which SQL query triggers the problem I added this code to write to a log file:
With the log enabled, I used
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Figure out why SpatiaLite 5.0 hangs the database page on Linux 837308703 | |
803764919 | https://github.com/simonw/datasette/issues/1268#issuecomment-803764919 | https://api.github.com/repos/simonw/datasette/issues/1268 | MDEyOklzc3VlQ29tbWVudDgwMzc2NDkxOQ== | simonw 9599 | 2021-03-22T05:11:11Z | 2021-03-22T05:11:11Z | OWNER | Maybe I could implement SQLite query timeouts using the https://stackoverflow.com/questions/43240496/python-sqlite3-how-to-quickly-and-cleanly-interrupt-long-running-query-with-e has some tips. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Figure out why SpatiaLite 5.0 hangs the database page on Linux 837308703 | |
803764200 | https://github.com/simonw/datasette/issues/1268#issuecomment-803764200 | https://api.github.com/repos/simonw/datasette/issues/1268 | MDEyOklzc3VlQ29tbWVudDgwMzc2NDIwMA== | simonw 9599 | 2021-03-22T05:09:13Z | 2021-03-22T05:09:13Z | OWNER | I tried building a container where the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Figure out why SpatiaLite 5.0 hangs the database page on Linux 837308703 | |
803762969 | https://github.com/simonw/datasette/issues/1268#issuecomment-803762969 | https://api.github.com/repos/simonw/datasette/issues/1268 | MDEyOklzc3VlQ29tbWVudDgwMzc2Mjk2OQ== | simonw 9599 | 2021-03-22T05:05:51Z | 2021-03-22T05:05:51Z | OWNER | I had to run |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Figure out why SpatiaLite 5.0 hangs the database page on Linux 837308703 | |
803762609 | https://github.com/simonw/datasette/issues/1268#issuecomment-803762609 | https://api.github.com/repos/simonw/datasette/issues/1268 | MDEyOklzc3VlQ29tbWVudDgwMzc2MjYwOQ== | simonw 9599 | 2021-03-22T05:05:00Z | 2021-03-22T05:05:00Z | OWNER | Using https://til.simonwillison.net/docker/attach-bash-to-running-container - I figured out how to run apt-get install gdb python3-dbggdb /usr/bin/python3 -p 20
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Figure out why SpatiaLite 5.0 hangs the database page on Linux 837308703 | |
803759051 | https://github.com/simonw/datasette/issues/1268#issuecomment-803759051 | https://api.github.com/repos/simonw/datasette/issues/1268 | MDEyOklzc3VlQ29tbWVudDgwMzc1OTA1MQ== | simonw 9599 | 2021-03-22T04:55:22Z | 2021-03-22T04:55:22Z | OWNER | So I think there's a bug in the way the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Figure out why SpatiaLite 5.0 hangs the database page on Linux 837308703 | |
803758793 | https://github.com/simonw/datasette/issues/1268#issuecomment-803758793 | https://api.github.com/repos/simonw/datasette/issues/1268 | MDEyOklzc3VlQ29tbWVudDgwMzc1ODc5Mw== | simonw 9599 | 2021-03-22T04:54:32Z | 2021-03-22T04:54:32Z | OWNER | Hitting http://localhost:8001/tuscany_housenumbers triggers the bug. It gets stuck in a loop that looks like this: Which looks to me like this code: https://github.com/simonw/datasette/blob/8e18c7943181f228ce5ebcea48deb59ce50bee1f/datasette/utils/init.py#L139-L158 |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Figure out why SpatiaLite 5.0 hangs the database page on Linux 837308703 | |
803758182 | https://github.com/simonw/datasette/issues/1268#issuecomment-803758182 | https://api.github.com/repos/simonw/datasette/issues/1268 | MDEyOklzc3VlQ29tbWVudDgwMzc1ODE4Mg== | simonw 9599 | 2021-03-22T04:52:15Z | 2021-03-22T04:52:15Z | OWNER | Hitting http://localhost:8001/ successfully shows the homepage (after a lot more scrolling). |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Figure out why SpatiaLite 5.0 hangs the database page on Linux 837308703 | |
803757746 | https://github.com/simonw/datasette/issues/1268#issuecomment-803757746 | https://api.github.com/repos/simonw/datasette/issues/1268 | MDEyOklzc3VlQ29tbWVudDgwMzc1Nzc0Ng== | simonw 9599 | 2021-03-22T04:50:40Z | 2021-03-22T04:51:52Z | OWNER | Here's a fun debugging trick:
A huge amount of stuff scrolls past as Datasette starts up, since we are tracing every executed line of Python. After about a minute it's finished starting and gets to this point:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Figure out why SpatiaLite 5.0 hangs the database page on Linux 837308703 | |
803756495 | https://github.com/simonw/datasette/issues/1268#issuecomment-803756495 | https://api.github.com/repos/simonw/datasette/issues/1268 | MDEyOklzc3VlQ29tbWVudDgwMzc1NjQ5NQ== | simonw 9599 | 2021-03-22T04:46:04Z | 2021-03-22T04:46:04Z | OWNER |
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Figure out why SpatiaLite 5.0 hangs the database page on Linux 837308703 | |
803755698 | https://github.com/simonw/datasette/issues/1249#issuecomment-803755698 | https://api.github.com/repos/simonw/datasette/issues/1249 | MDEyOklzc3VlQ29tbWVudDgwMzc1NTY5OA== | simonw 9599 | 2021-03-22T04:43:02Z | 2021-03-22T04:43:02Z | OWNER | I'll spin off a separate ticket to investigate the hang. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Updated Dockerfile with SpatiaLite version 5.0 824064069 | |
803754226 | https://github.com/simonw/datasette/issues/1267#issuecomment-803754226 | https://api.github.com/repos/simonw/datasette/issues/1267 | MDEyOklzc3VlQ29tbWVudDgwMzc1NDIyNg== | simonw 9599 | 2021-03-22T04:37:26Z | 2021-03-22T04:37:26Z | OWNER | Thanks for doing this - I've used alternativeto.net a bunch in the past, it's great to see Datasette listed there. This does raise some interesting philosophical questions: three years into the project I'm still not entirely sure what Datasette competes with! Could be SQLite desktop packages, could be visualization software like Tableau, could even be something like Airtable (given a few more plugins). It will be interesting to see how the alternativeto listing evolves, maybe it will help me answer that question! |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Update Datasette alternativeto listening with details 837208901 | |
803753388 | https://github.com/simonw/datasette/issues/1249#issuecomment-803753388 | https://api.github.com/repos/simonw/datasette/issues/1249 | MDEyOklzc3VlQ29tbWVudDgwMzc1MzM4OA== | simonw 9599 | 2021-03-22T04:34:20Z | 2021-03-22T04:35:10Z | OWNER | Well this is frustrating. I finally found a Dockerfile that worked and installed an Ubuntu pre-compiled SpatiaLite module that would load... ```dockerfile FROM ubuntu:20.10 as install_spatialite RUN apt update && \ apt install -y libsqlite3-mod-spatialite && \ apt clean && \ rm -rf /var/lib/{apt,dpkg,cache,log}/ FROM ubuntu:20.10 RUN apt update && \ apt install -y python3-pip && \ apt clean && \ rm -rf /var/lib/{apt,dpkg,cache,log}/ RUN pip install datasette Copy spatial extensionsCOPY --from=install_spatialite /usr/lib/x86_64-linux-gnu /usr/lib/x86_64-linux-gnu/ ENV LD_LIBRARY_PATH=/usr/local/lib EXPOSE 8001 CMD ["datasette"] ``` (Which produced a 550MB image) And when I ran Datasette I got that same error where the database listing page hangs!
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Updated Dockerfile with SpatiaLite version 5.0 824064069 | |
803751068 | https://github.com/simonw/datasette/issues/1249#issuecomment-803751068 | https://api.github.com/repos/simonw/datasette/issues/1249 | MDEyOklzc3VlQ29tbWVudDgwMzc1MTA2OA== | simonw 9599 | 2021-03-22T04:26:45Z | 2021-03-22T04:26:45Z | OWNER | Here's why:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Updated Dockerfile with SpatiaLite version 5.0 824064069 | |
803750617 | https://github.com/simonw/datasette/issues/1249#issuecomment-803750617 | https://api.github.com/repos/simonw/datasette/issues/1249 | MDEyOklzc3VlQ29tbWVudDgwMzc1MDYxNw== | simonw 9599 | 2021-03-22T04:25:14Z | 2021-03-22T04:25:14Z | OWNER | Got this error attempting to run Datasette (with or without SpatiaLite):
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Updated Dockerfile with SpatiaLite version 5.0 824064069 | |
803750399 | https://github.com/simonw/datasette/issues/1249#issuecomment-803750399 | https://api.github.com/repos/simonw/datasette/issues/1249 | MDEyOklzc3VlQ29tbWVudDgwMzc1MDM5OQ== | simonw 9599 | 2021-03-22T04:24:25Z | 2021-03-22T04:24:25Z | OWNER | I'll try using RUN apt update && \ apt install -y libsqlite3-mod-spatialite && \ apt clean && \ rm -rf /var/lib/{apt,dpkg,cache,log}/ FROM ubuntu:20.10 as build RUN apt update && \ apt install -y python3-pip && \ apt clean && \ rm -rf /var/lib/{apt,dpkg,cache,log}/ RUN pip install datasette COPY . /datasetteRUN pip install /datasetteFROM ubuntu:20.10 Copy python dependencies and spatialite librariesCOPY --from=build /usr/local/lib/ /usr/local/lib/ Copy executablesCOPY --from=build /usr/local/bin /usr/local/bin Copy spatial extensionsCOPY --from=install_spatialite /usr/lib/x86_64-linux-gnu/mod_spatialite.so /usr/lib/x86_64-linux-gnu/mod_spatialite.so ENV LD_LIBRARY_PATH=/usr/local/lib EXPOSE 8001 CMD ["datasette"] ``` |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Updated Dockerfile with SpatiaLite version 5.0 824064069 | |
803749831 | https://github.com/simonw/datasette/issues/1249#issuecomment-803749831 | https://api.github.com/repos/simonw/datasette/issues/1249 | MDEyOklzc3VlQ29tbWVudDgwMzc0OTgzMQ== | simonw 9599 | 2021-03-22T04:22:35Z | 2021-03-22T04:22:35Z | OWNER | I tried copying just the RUN apt update && \ apt install -y libsqlite3-mod-spatialite && \ apt clean && \ rm -rf /var/lib/{apt,dpkg,cache,log}/ FROM python:3.9.2-slim as build RUN pip install datasette COPY . /datasetteRUN pip install /datasetteFROM python:3.9.2-slim Copy python dependencies and spatialite librariesCOPY --from=build /usr/local/lib/ /usr/local/lib/ Copy executablesCOPY --from=build /usr/local/bin /usr/local/bin Copy spatial extensionsCOPY --from=install_spatialite /usr/lib/x86_64-linux-gnu/mod_spatialite.so /usr/lib/x86_64-linux-gnu/mod_spatialite.so ENV LD_LIBRARY_PATH=/usr/local/lib EXPOSE 8001
CMD ["datasette"]
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Updated Dockerfile with SpatiaLite version 5.0 824064069 | |
803748469 | https://github.com/simonw/datasette/issues/1249#issuecomment-803748469 | https://api.github.com/repos/simonw/datasette/issues/1249 | MDEyOklzc3VlQ29tbWVudDgwMzc0ODQ2OQ== | simonw 9599 | 2021-03-22T04:17:51Z | 2021-03-22T04:17:51Z | OWNER | ... except my clever image using SpatiaLite installed for Ubuntu doesn't actually work:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Updated Dockerfile with SpatiaLite version 5.0 824064069 | |
803748158 | https://github.com/simonw/datasette/issues/1249#issuecomment-803748158 | https://api.github.com/repos/simonw/datasette/issues/1249 | MDEyOklzc3VlQ29tbWVudDgwMzc0ODE1OA== | simonw 9599 | 2021-03-22T04:16:57Z | 2021-03-22T04:16:57Z | OWNER | Which is great, because the image on Docker Hub right now is 383MB. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Updated Dockerfile with SpatiaLite version 5.0 824064069 | |
803747701 | https://github.com/simonw/datasette/issues/1249#issuecomment-803747701 | https://api.github.com/repos/simonw/datasette/issues/1249 | MDEyOklzc3VlQ29tbWVudDgwMzc0NzcwMQ== | simonw 9599 | 2021-03-22T04:15:40Z | 2021-03-22T04:15:40Z | OWNER | Here's a trick: install SpatiaLite in ```dockerfile FROM ubuntu:20.10 as install_spatialite RUN apt update && \ apt install -y libsqlite3-mod-spatialite && \ apt clean && \ rm -rf /var/lib/{apt,dpkg,cache,log}/ FROM python:3.9.2-slim as build RUN pip install datasette COPY . /datasetteRUN pip install /datasetteFROM python:3.9.2-slim Copy python dependencies and spatialite librariesCOPY --from=build /usr/local/lib/ /usr/local/lib/ Copy executablesCOPY --from=build /usr/local/bin /usr/local/bin Copy spatial extensionsCOPY --from=install_spatialite /usr/lib/x86_64-linux-gnu /usr/lib/x86_64-linux-gnu ENV LD_LIBRARY_PATH=/usr/local/lib EXPOSE 8001 CMD ["datasette"] ``` That produced a 265MB image. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Updated Dockerfile with SpatiaLite version 5.0 824064069 | |
803700940 | https://github.com/simonw/datasette/issues/1249#issuecomment-803700940 | https://api.github.com/repos/simonw/datasette/issues/1249 | MDEyOklzc3VlQ29tbWVudDgwMzcwMDk0MA== | simonw 9599 | 2021-03-22T01:14:24Z | 2021-03-22T01:14:24Z | OWNER | I tried that with just |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Updated Dockerfile with SpatiaLite version 5.0 824064069 | |
803700626 | https://github.com/simonw/datasette/issues/1249#issuecomment-803700626 | https://api.github.com/repos/simonw/datasette/issues/1249 | MDEyOklzc3VlQ29tbWVudDgwMzcwMDYyNg== | simonw 9599 | 2021-03-22T01:13:04Z | 2021-03-22T01:13:04Z | OWNER | Building a Dockerfile containing just Building this one: ```dockerfile FROM ubuntu:20.10 Setup build dependenciesRUN apt update && \ apt install -y python3-pip libsqlite3-mod-spatialite && \ apt clean && \ rm -rf /var/lib/{apt,dpkg,cache,log}/ ``` Resulted in a 515MB image. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Updated Dockerfile with SpatiaLite version 5.0 824064069 | |
803698983 | https://github.com/simonw/datasette/issues/1249#issuecomment-803698983 | https://api.github.com/repos/simonw/datasette/issues/1249 | MDEyOklzc3VlQ29tbWVudDgwMzY5ODk4Mw== | simonw 9599 | 2021-03-22T01:05:36Z | 2021-03-22T01:06:23Z | OWNER | It's pretty big though. I tried this version which avoids copying junk from my laptop in: ```dockerfile FROM ubuntu:20.10 Setup build dependenciesRUN apt update && apt install -y python3-pip libsqlite3-mod-spatialite && apt clean RUN pip install datasette EXPOSE 8001
CMD ["datasette"]
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Updated Dockerfile with SpatiaLite version 5.0 824064069 | |
803698168 | https://github.com/simonw/datasette/issues/1249#issuecomment-803698168 | https://api.github.com/repos/simonw/datasette/issues/1249 | MDEyOklzc3VlQ29tbWVudDgwMzY5ODE2OA== | simonw 9599 | 2021-03-22T01:02:02Z | 2021-03-22T01:02:30Z | OWNER | This is the shortest Dockerfile that appeared to give me a working SpatiaLite module: ```dockerfile FROM ubuntu:20.10 Setup build dependenciesRUN apt update && apt install -y python3-pip libsqlite3-mod-spatialite && apt clean Add local code to the image instead of fetching from pypi.COPY . /datasette RUN pip install /datasette RUN rm -rf /datasette EXPOSE 8001 CMD ["datasette"] ``` |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Updated Dockerfile with SpatiaLite version 5.0 824064069 | |
803697546 | https://github.com/simonw/datasette/issues/1249#issuecomment-803697546 | https://api.github.com/repos/simonw/datasette/issues/1249 | MDEyOklzc3VlQ29tbWVudDgwMzY5NzU0Ng== | simonw 9599 | 2021-03-22T00:59:47Z | 2021-03-22T00:59:47Z | OWNER | To debug I'm running:
This gets me a shell I can use. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Updated Dockerfile with SpatiaLite version 5.0 824064069 | |
803697211 | https://github.com/simonw/datasette/issues/1249#issuecomment-803697211 | https://api.github.com/repos/simonw/datasette/issues/1249 | MDEyOklzc3VlQ29tbWVudDgwMzY5NzIxMQ== | simonw 9599 | 2021-03-22T00:58:01Z | 2021-03-22T00:58:01Z | OWNER | I'm messing around with the
And then:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Updated Dockerfile with SpatiaLite version 5.0 824064069 | |
803694661 | https://github.com/simonw/datasette/issues/1249#issuecomment-803694661 | https://api.github.com/repos/simonw/datasette/issues/1249 | MDEyOklzc3VlQ29tbWVudDgwMzY5NDY2MQ== | simonw 9599 | 2021-03-22T00:46:49Z | 2021-03-22T00:46:49Z | OWNER | Actually for the loadable module I think I need https://packages.ubuntu.com/groovy/libsqlite3-mod-spatialite |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Updated Dockerfile with SpatiaLite version 5.0 824064069 | |
803694436 | https://github.com/simonw/datasette/issues/1249#issuecomment-803694436 | https://api.github.com/repos/simonw/datasette/issues/1249 | MDEyOklzc3VlQ29tbWVudDgwMzY5NDQzNg== | simonw 9599 | 2021-03-22T00:46:00Z | 2021-03-22T00:46:00Z | OWNER | So I'm going to try |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Updated Dockerfile with SpatiaLite version 5.0 824064069 | |
803694359 | https://github.com/simonw/datasette/issues/1249#issuecomment-803694359 | https://api.github.com/repos/simonw/datasette/issues/1249 | MDEyOklzc3VlQ29tbWVudDgwMzY5NDM1OQ== | simonw 9599 | 2021-03-22T00:45:47Z | 2021-03-22T00:45:47Z | OWNER | https://pythonspeed.com/articles/base-image-python-docker-images/ suggests using |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Updated Dockerfile with SpatiaLite version 5.0 824064069 | |
803693181 | https://github.com/simonw/datasette/issues/1249#issuecomment-803693181 | https://api.github.com/repos/simonw/datasette/issues/1249 | MDEyOklzc3VlQ29tbWVudDgwMzY5MzE4MQ== | simonw 9599 | 2021-03-22T00:41:02Z | 2021-03-22T00:41:02Z | OWNER | Debian sid has it too: https://packages.debian.org/sid/libspatialite-dev |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Updated Dockerfile with SpatiaLite version 5.0 824064069 | |
803692673 | https://github.com/simonw/datasette/issues/1249#issuecomment-803692673 | https://api.github.com/repos/simonw/datasette/issues/1249 | MDEyOklzc3VlQ29tbWVudDgwMzY5MjY3Mw== | simonw 9599 | 2021-03-22T00:38:42Z | 2021-03-22T00:38:42Z | OWNER | Ubuntu Groovy has a package for SpatiaLite 5 - I could try using that instead: https://packages.ubuntu.com/groovy/libspatialite-dev |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Updated Dockerfile with SpatiaLite version 5.0 824064069 | |
803691236 | https://github.com/simonw/datasette/issues/1249#issuecomment-803691236 | https://api.github.com/repos/simonw/datasette/issues/1249 | MDEyOklzc3VlQ29tbWVudDgwMzY5MTIzNg== | simonw 9599 | 2021-03-22T00:32:03Z | 2021-03-22T00:32:03Z | OWNER | Here's something odd: when I run But when I tried compiling SpatiaLite inside the Docker container I had hanging errors on some pages. This is using https://www.gaia-gis.it/gaia-sins/knn/tuscany_housenumbers.7z from the SpatiaLite KNN tutorial at https://www.gaia-gis.it/fossil/libspatialite/wiki?name=KNN |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Updated Dockerfile with SpatiaLite version 5.0 824064069 | |
803674728 | https://github.com/simonw/datasette/issues/1259#issuecomment-803674728 | https://api.github.com/repos/simonw/datasette/issues/1259 | MDEyOklzc3VlQ29tbWVudDgwMzY3NDcyOA== | simonw 9599 | 2021-03-21T22:55:31Z | 2021-03-21T22:55:31Z | OWNER | CTEs were added in 2014-02-03 SQLite 3.8.3 - so I think it's OK to depend on them for Datasette. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Research using CTEs for faster facet counts 830567275 | |
803673225 | https://github.com/simonw/datasette/issues/647#issuecomment-803673225 | https://api.github.com/repos/simonw/datasette/issues/647 | MDEyOklzc3VlQ29tbWVudDgwMzY3MzIyNQ== | simonw 9599 | 2021-03-21T22:44:19Z | 2021-03-21T22:44:19Z | OWNER | Now that I'm looking at refactoring how views work in #878 it's clear that the gnarliest, most convoluted code I need to deal with relates to this old feature. I'm going to remove it entirely. Any performance enhancement or provides can be achieved just as well by using regular URLs and a caching proxy. I may provide a 404 handling plugin that attempts to rewrite old URLs that used this mechanism, but I won't do any more than that. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Move hashed URL mode out to a plugin 531755959 | |
803501756 | https://github.com/simonw/sqlite-utils/issues/249#issuecomment-803501756 | https://api.github.com/repos/simonw/sqlite-utils/issues/249 | MDEyOklzc3VlQ29tbWVudDgwMzUwMTc1Ng== | simonw 9599 | 2021-03-21T02:33:45Z | 2021-03-21T02:33:45Z | OWNER | Did you run If so you'll need to run A better solution may be to add |
{ "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Full text search possibly broken? 836963850 | |
803473015 | https://github.com/simonw/datasette/issues/878#issuecomment-803473015 | https://api.github.com/repos/simonw/datasette/issues/878 | MDEyOklzc3VlQ29tbWVudDgwMzQ3MzAxNQ== | simonw 9599 | 2021-03-20T22:33:05Z | 2021-03-20T22:33:05Z | OWNER | Things this mechanism needs to be able to support:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
New pattern for views that return either JSON or HTML, available for plugins 648435885 | |
803472595 | https://github.com/simonw/datasette/issues/878#issuecomment-803472595 | https://api.github.com/repos/simonw/datasette/issues/878 | MDEyOklzc3VlQ29tbWVudDgwMzQ3MjU5NQ== | simonw 9599 | 2021-03-20T22:28:12Z | 2021-03-20T22:28:12Z | OWNER | Another idea I had: a view is a class that takes the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
New pattern for views that return either JSON or HTML, available for plugins 648435885 | |
803472278 | https://github.com/simonw/datasette/issues/878#issuecomment-803472278 | https://api.github.com/repos/simonw/datasette/issues/878 | MDEyOklzc3VlQ29tbWVudDgwMzQ3MjI3OA== | simonw 9599 | 2021-03-20T22:25:04Z | 2021-03-20T22:25:04Z | OWNER | I came up with a slightly wild idea for this that would involve pytest-style dependency injection. Prototype here: https://gist.github.com/simonw/496b24fdad44f6f8b7237fe394a0ced7 Copying from my private notes:
|
{ "total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 1 } |
New pattern for views that return either JSON or HTML, available for plugins 648435885 | |
803471917 | https://github.com/simonw/datasette/issues/878#issuecomment-803471917 | https://api.github.com/repos/simonw/datasette/issues/878 | MDEyOklzc3VlQ29tbWVudDgwMzQ3MTkxNw== | simonw 9599 | 2021-03-20T22:21:33Z | 2021-03-20T22:21:33Z | OWNER | This has been blocking things for too long. If this becomes a documented pattern, things like adding a JSON output to https://github.com/dogsheep/dogsheep-beta becomes easier too. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
New pattern for views that return either JSON or HTML, available for plugins 648435885 | |
803471702 | https://github.com/simonw/datasette/issues/1258#issuecomment-803471702 | https://api.github.com/repos/simonw/datasette/issues/1258 | MDEyOklzc3VlQ29tbWVudDgwMzQ3MTcwMg== | simonw 9599 | 2021-03-20T22:19:39Z | 2021-03-20T22:19:39Z | OWNER | This is a good idea. I avoided this initially because it should be possible to run a canned query with a parameter set to the empty string, but that view could definitely be smart enough to differentiate between |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Allow canned query params to specify default values 828858421 | |
803469623 | https://github.com/simonw/datasette/issues/782#issuecomment-803469623 | https://api.github.com/repos/simonw/datasette/issues/782 | MDEyOklzc3VlQ29tbWVudDgwMzQ2OTYyMw== | simonw 9599 | 2021-03-20T22:01:23Z | 2021-03-20T22:01:23Z | OWNER | I'm going to keep |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Redesign default .json format 627794879 | |
803468314 | https://github.com/simonw/datasette/issues/1261#issuecomment-803468314 | https://api.github.com/repos/simonw/datasette/issues/1261 | MDEyOklzc3VlQ29tbWVudDgwMzQ2ODMxNA== | simonw 9599 | 2021-03-20T21:48:48Z | 2021-03-20T21:48:48Z | OWNER | That's fixed in this release of |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Some links aren't properly URL encoded. 832092321 | |
803466868 | https://github.com/simonw/datasette/issues/1261#issuecomment-803466868 | https://api.github.com/repos/simonw/datasette/issues/1261 | MDEyOklzc3VlQ29tbWVudDgwMzQ2Njg2OA== | simonw 9599 | 2021-03-20T21:36:06Z | 2021-03-20T21:36:06Z | OWNER | This isn't a Datasette bug - it's a Vercel bug: https://github.com/simonw/datasette-publish-vercel/issues/28 I'm looking at a fix for that now, so watch that issue for updates. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Some links aren't properly URL encoded. 832092321 | |
803466730 | https://github.com/simonw/datasette/issues/1266#issuecomment-803466730 | https://api.github.com/repos/simonw/datasette/issues/1266 | MDEyOklzc3VlQ29tbWVudDgwMzQ2NjczMA== | simonw 9599 | 2021-03-20T21:35:00Z | 2021-03-20T21:35:00Z | OWNER | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Documentation for Response.asgi_send(send) method 836273891 | ||
803130332 | https://github.com/simonw/datasette/issues/1265#issuecomment-803130332 | https://api.github.com/repos/simonw/datasette/issues/1265 | MDEyOklzc3VlQ29tbWVudDgwMzEzMDMzMg== | simonw 9599 | 2021-03-19T21:03:09Z | 2021-03-19T21:03:09Z | OWNER | This is now available in |
{ "total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 1, "eyes": 0 } |
Support for HTTP Basic Authentication 836123030 | |
802099264 | https://github.com/simonw/datasette/issues/1262#issuecomment-802099264 | https://api.github.com/repos/simonw/datasette/issues/1262 | MDEyOklzc3VlQ29tbWVudDgwMjA5OTI2NA== | simonw 9599 | 2021-03-18T16:43:09Z | 2021-03-18T16:43:09Z | OWNER | I often find myself wanting this too, when I'm exploring a new dataset. i agree with Bob that this is a good candidate for a plugin. The plugin system isn't quite setup for this yet though - there isn't an obvious mechanism for adding extra sort orders or other interface elements that manipulate the query used by the table view in some way. I'm going to promote this issue to status of a plugin hook feature request - I have a hunch that a plugin hook that enables |
{ "total_count": 2, "+1": 2, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Plugin hook that could support 'order by random()' for table view 834602299 | |
799479175 | https://github.com/simonw/sqlite-utils/issues/246#issuecomment-799479175 | https://api.github.com/repos/simonw/sqlite-utils/issues/246 | MDEyOklzc3VlQ29tbWVudDc5OTQ3OTE3NQ== | simonw 9599 | 2021-03-15T14:47:31Z | 2021-03-15T14:47:31Z | OWNER | This is a smart feature. I have something that does this in Datasette, extracting it out to https://github.com/simonw/datasette/blob/8e18c7943181f228ce5ebcea48deb59ce50bee1f/datasette/utils/init.py#L818-L829 |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Escaping FTS search strings 831751367 | |
799066252 | https://github.com/simonw/datasette/issues/236#issuecomment-799066252 | https://api.github.com/repos/simonw/datasette/issues/236 | MDEyOklzc3VlQ29tbWVudDc5OTA2NjI1Mg== | simonw 9599 | 2021-03-15T03:34:52Z | 2021-03-15T03:34:52Z | OWNER | Yeah the Lambda Docker stuff is pretty odd - you still don't get to speak HTTP, you have to speak their custom event protocol instead. https://github.com/glassechidna/serverlessish looks interesting here - it adds a proxy inside the container which allows your existing HTTP Docker image to run within Docker-on-Lambda. I've not tried it out yet though. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
datasette publish lambda plugin 317001500 | |
797827038 | https://github.com/simonw/datasette/issues/1259#issuecomment-797827038 | https://api.github.com/repos/simonw/datasette/issues/1259 | MDEyOklzc3VlQ29tbWVudDc5NzgyNzAzOA== | simonw 9599 | 2021-03-13T00:15:40Z | 2021-03-13T00:15:40Z | OWNER | If all of the facets were being calculated in a single query, I'd be willing to bump the facet time limit up to something a lot higher, maybe even a full second. There's a chance that could work amazingly well with a materialized CTE. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Research using CTEs for faster facet counts 830567275 | |
797804869 | https://github.com/simonw/datasette/issues/1259#issuecomment-797804869 | https://api.github.com/repos/simonw/datasette/issues/1259 | MDEyOklzc3VlQ29tbWVudDc5NzgwNDg2OQ== | simonw 9599 | 2021-03-12T23:05:05Z | 2021-03-12T23:05:05Z | OWNER | I wonder if I could optimize facet suggestion in the same way? One challenge: the query time limit will apply to the full CTE query, not to the individual columns. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Research using CTEs for faster facet counts 830567275 | |
797801075 | https://github.com/simonw/datasette/issues/1259#issuecomment-797801075 | https://api.github.com/repos/simonw/datasette/issues/1259 | MDEyOklzc3VlQ29tbWVudDc5NzgwMTA3NQ== | simonw 9599 | 2021-03-12T22:53:56Z | 2021-03-12T22:55:16Z | OWNER | OK, a better comparison: https://global-power-plants.datasettes.com/global-power-plants?sql=WITH+data+as+%28%0D%0A++select%0D%0A++++%0D%0A++from%0D%0A++++%5Bglobal-power-plants%5D%0D%0A%29%2C%0D%0Acountry_long+as+%28select+%0D%0A++%27country_long%27+as+col%2C+country_long+as+value%2C+count%28%29+as+c+from+data+group+by+country_long%0D%0A++order+by+c+desc+limit+31%0D%0A%29%2C%0D%0Aprimary_fuel+as+%28%0D%0Aselect%0D%0A++%27primary_fuel%27+as+col%2C+primary_fuel+as+value%2C+count%28%29+as+c+from+data+group+by+primary_fuel%0D%0A++order+by+c+desc+limit+31%0D%0A%29%2C%0D%0Aowner+as+%28%0D%0Aselect%0D%0A++%27owner%27+as+col%2C+owner+as+value%2C+count%28%29+as+c+from+data+group+by+owner%0D%0A++order+by+c+desc+limit+31%0D%0A%29%0D%0Aselect++from+primary_fuel+union+select++from+country_long%0D%0Aunion+select++from+owner+order+by+col%2C+c+desc calculates facets against three columns. It takes 78.5ms* (and 34.5ms when I refreshed it, presumably after warming some SQLite caches of some sort). https://global-power-plants.datasettes.com/global-power-plants/global-power-plants?_facet=country_long&_facet=primary_fuel&_trace=1&_size=0 shows those facets with size=0 on the SQL query - and shows a SQL trace at the bottom of the page. The country_long facet query takes 45.36ms, owner takes 38.45ms, primary_fuel takes 49.04ms - so a total of 132.85ms That's against https://global-power-plants.datasettes.com/-/versions says SQLite 3.27.3 - so even on a SQLite version that doesn't materialize the CTEs there's a significant performance boost to doing all three facets in a single CTE query. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Research using CTEs for faster facet counts 830567275 | |
797790017 | https://github.com/simonw/datasette/issues/1259#issuecomment-797790017 | https://api.github.com/repos/simonw/datasette/issues/1259 | MDEyOklzc3VlQ29tbWVudDc5Nzc5MDAxNw== | simonw 9599 | 2021-03-12T22:22:12Z | 2021-03-12T22:22:12Z | OWNER | https://sqlite.org/lang_with.html
It looks like this optimization is completely unavailable on SQLite prior to 3.35.0 (released 12th March 2021). But I could still rewrite the faceting to work in this way, using the exact same SQL - it would just be significantly faster on 3.35.0+ (assuming it's actually faster in practice - would need to benchmark). |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Research using CTEs for faster facet counts 830567275 | |
797159434 | https://github.com/simonw/datasette/issues/1193#issuecomment-797159434 | https://api.github.com/repos/simonw/datasette/issues/1193 | MDEyOklzc3VlQ29tbWVudDc5NzE1OTQzNA== | simonw 9599 | 2021-03-12T01:01:54Z | 2021-03-12T01:01:54Z | OWNER | DuckDB has a read-only mechanism: https://duckdb.org/docs/api/python
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Research plugin hook for alternative database backends 787173276 | |
797159221 | https://github.com/simonw/datasette/issues/1250#issuecomment-797159221 | https://api.github.com/repos/simonw/datasette/issues/1250 | MDEyOklzc3VlQ29tbWVudDc5NzE1OTIyMQ== | simonw 9599 | 2021-03-12T01:01:17Z | 2021-03-12T01:01:17Z | OWNER | This is a duplicate of #1193. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Research: Plugin hook for alternative database connections 824067604 | |
797158641 | https://github.com/simonw/datasette/issues/670#issuecomment-797158641 | https://api.github.com/repos/simonw/datasette/issues/670 | MDEyOklzc3VlQ29tbWVudDc5NzE1ODY0MQ== | simonw 9599 | 2021-03-12T00:59:49Z | 2021-03-12T00:59:49Z | OWNER |
It looks like the answer to this is yes - I'll need users to setup read-only credentials. Here's a TIL about that: https://til.simonwillison.net/postgresql/read-only-postgresql-user |
{ "total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 1, "rocket": 0, "eyes": 0 } |
Prototoype for Datasette on PostgreSQL 564833696 | |
796854370 | https://github.com/simonw/datasette/pull/1211#issuecomment-796854370 | https://api.github.com/repos/simonw/datasette/issues/1211 | MDEyOklzc3VlQ29tbWVudDc5Njg1NDM3MA== | simonw 9599 | 2021-03-11T16:15:29Z | 2021-03-11T16:15:29Z | OWNER | Thanks very much for this - it's really comprehensive. I need to bake some of these patterns into my coding habits better! |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Use context manager instead of plain open 797649915 | |
795918377 | https://github.com/simonw/datasette/issues/838#issuecomment-795918377 | https://api.github.com/repos/simonw/datasette/issues/838 | MDEyOklzc3VlQ29tbWVudDc5NTkxODM3Nw== | simonw 9599 | 2021-03-10T19:01:48Z | 2021-03-10T19:01:48Z | OWNER | The biggest challenge here I think is to replicate the exact situation here this happens in a Python unit test. The fix should be easy once we have a test in place. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Incorrect URLs when served behind a proxy with base_url set 637395097 | |
795895436 | https://github.com/simonw/datasette/issues/838#issuecomment-795895436 | https://api.github.com/repos/simonw/datasette/issues/838 | MDEyOklzc3VlQ29tbWVudDc5NTg5NTQzNg== | simonw 9599 | 2021-03-10T18:44:46Z | 2021-03-10T18:44:57Z | OWNER | Let's reopen this. |
{ "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Incorrect URLs when served behind a proxy with base_url set 637395097 | |
795870524 | https://github.com/simonw/datasette/pull/1254#issuecomment-795870524 | https://api.github.com/repos/simonw/datasette/issues/1254 | MDEyOklzc3VlQ29tbWVudDc5NTg3MDUyNA== | simonw 9599 | 2021-03-10T18:27:45Z | 2021-03-10T18:27:45Z | OWNER | What other breaks did you spot? |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Update Docker Spatialite version to 5.0.1 + add support for Spatialite topology functions 826613352 | |
795869144 | https://github.com/simonw/datasette/pull/1256#issuecomment-795869144 | https://api.github.com/repos/simonw/datasette/issues/1256 | MDEyOklzc3VlQ29tbWVudDc5NTg2OTE0NA== | simonw 9599 | 2021-03-10T18:26:46Z | 2021-03-10T18:26:46Z | OWNER | Thanks! |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Minor type in IP adress 827341657 | |
794439632 | https://github.com/simonw/datasette/pull/1254#issuecomment-794439632 | https://api.github.com/repos/simonw/datasette/issues/1254 | MDEyOklzc3VlQ29tbWVudDc5NDQzOTYzMg== | simonw 9599 | 2021-03-09T20:53:02Z | 2021-03-09T20:53:02Z | OWNER | Thanks for catching that documentation update! |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Update Docker Spatialite version to 5.0.1 + add support for Spatialite topology functions 826613352 | |
794437715 | https://github.com/simonw/datasette/pull/1254#issuecomment-794437715 | https://api.github.com/repos/simonw/datasette/issues/1254 | MDEyOklzc3VlQ29tbWVudDc5NDQzNzcxNQ== | simonw 9599 | 2021-03-09T20:51:19Z | 2021-03-09T20:51:19Z | OWNER | Did you see my note on https://github.com/simonw/datasette/issues/1249#issuecomment-792384382 about a weird issue I was having with the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Update Docker Spatialite version to 5.0.1 + add support for Spatialite topology functions 826613352 | |
792386484 | https://github.com/simonw/datasette/issues/1250#issuecomment-792386484 | https://api.github.com/repos/simonw/datasette/issues/1250 | MDEyOklzc3VlQ29tbWVudDc5MjM4NjQ4NA== | simonw 9599 | 2021-03-08T00:29:06Z | 2021-03-08T00:29:06Z | OWNER | DuckDB has a read-only mechanism: https://duckdb.org/docs/api/python
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Research: Plugin hook for alternative database connections 824067604 | |
792385274 | https://github.com/simonw/datasette/issues/1248#issuecomment-792385274 | https://api.github.com/repos/simonw/datasette/issues/1248 | MDEyOklzc3VlQ29tbWVudDc5MjM4NTI3NA== | simonw 9599 | 2021-03-08T00:25:10Z | 2021-03-08T00:25:10Z | OWNER | It's not possible yet, unfortunately. This came up on the forums recently: https://github.com/simonw/datasette/discussions/968 I'm leaning further towards making the database connection layer itself work via a plugin hook, which would open up the possibility of supporting DuckDB and other databases as well. I've not committed to doing this yet though. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
duckdb database (very low performance in SQLite) 823035080 | |
792384854 | https://github.com/simonw/datasette/issues/1249#issuecomment-792384854 | https://api.github.com/repos/simonw/datasette/issues/1249 | MDEyOklzc3VlQ29tbWVudDc5MjM4NDg1NA== | simonw 9599 | 2021-03-08T00:23:38Z | 2021-03-08T00:23:38Z | OWNER | One reason to prioritize this issue: Homebrew upgraded to SpatiaLite 5.0 recently https://formulae.brew.sh/formula/spatialite-tools and as a result SpatiaLite database created on my laptop don't appear to be compatible with Datasette when published using |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Updated Dockerfile with SpatiaLite version 5.0 824064069 | |
792384382 | https://github.com/simonw/datasette/issues/1249#issuecomment-792384382 | https://api.github.com/repos/simonw/datasette/issues/1249 | MDEyOklzc3VlQ29tbWVudDc5MjM4NDM4Mg== | simonw 9599 | 2021-03-08T00:22:02Z | 2021-03-08T00:22:02Z | OWNER | I tried this patch against # Setup build dependencies RUN apt update \ -&& apt install -y python3-dev build-essential wget libxml2-dev libproj-dev libgeos-dev libsqlite3-dev zlib1g-dev pkg-config git \ - && apt clean + && apt install -y python3-dev build-essential wget libxml2-dev libproj-dev \ + libminizip-dev libgeos-dev libsqlite3-dev zlib1g-dev pkg-config git \ + && apt clean - -RUN wget "https://www.sqlite.org/2020/sqlite-autoconf-3310100.tar.gz" && tar xzf sqlite-autoconf-3310100.tar.gz \ - && cd sqlite-autoconf-3310100 && ./configure --disable-static --enable-fts5 --enable-json1 CFLAGS="-g -O2 -DSQLITE_ENABLE_FTS3=1 -DSQLITE_ENABLE_FTS3_PARENTHESIS -DSQLITE_ENABLE_FTS4=1 -DSQLITE_ENABLE_RTREE=1 -DSQLITE_ENABLE_JSON1" \ +RUN wget "https://www.sqlite.org/2021/sqlite-autoconf-3340100.tar.gz" && tar xzf sqlite-autoconf-3340100.tar.gz \ + && cd sqlite-autoconf-3340100 && ./configure --disable-static --enable-fts5 --enable-json1 \ + CFLAGS="-g -O2 -DSQLITE_ENABLE_FTS3=1 -DSQLITE_ENABLE_FTS3_PARENTHESIS -DSQLITE_ENABLE_FTS4=1 -DSQLITE_ENABLE_RTREE=1 -DSQLITE_ENABLE_JSON1" \ && make && make install -RUN wget "http://www.gaia-gis.it/gaia-sins/freexl-sources/freexl-1.0.5.tar.gz" && tar zxf freexl-1.0.5.tar.gz \ - && cd freexl-1.0.5 && ./configure && make && make install +RUN wget "http://www.gaia-gis.it/gaia-sins/freexl-1.0.6.tar.gz" && tar zxf freexl-1.0.6.tar.gz \ + && cd freexl-1.0.6 && ./configure && make && make install -RUN wget "http://www.gaia-gis.it/gaia-sins/libspatialite-sources/libspatialite-4.4.0-RC0.tar.gz" && tar zxf libspatialite-4.4.0-RC0.tar.gz \ - && cd libspatialite-4.4.0-RC0 && ./configure && make && make install +RUN wget "http://www.gaia-gis.it/gaia-sins/libspatialite-5.0.1.tar.gz" && tar zxf libspatialite-5.0.1.tar.gz \ + && cd libspatialite-5.0.1 && ./configure --disable-rttopo && make && make install RUN wget "http://www.gaia-gis.it/gaia-sins/readosm-sources/readosm-1.1.0.tar.gz" && tar zxf readosm-1.1.0.tar.gz && cd readosm-1.1.0 && ./configure && make && make install -RUN wget "http://www.gaia-gis.it/gaia-sins/spatialite-tools-sources/spatialite-tools-4.4.0-RC0.tar.gz" && tar zxf spatialite-tools-4.4.0-RC0.tar.gz \ - && cd spatialite-tools-4.4.0-RC0 && ./configure && make && make install +RUN wget "http://www.gaia-gis.it/gaia-sins/spatialite-tools-5.0.0.tar.gz" && tar zxf spatialite-tools-5.0.0.tar.gz \ + && cd spatialite-tools-5.0.0 && ./configure --disable-rttopo && make && make install # Add local code to the image instead of fetching from pypi. @@ -27,7 +28,7 @@ COPY . /datasette RUN pip install /datasette -FROM python:3.7.10-slim-stretch +FROM python:3.9.2-slim-buster # Copy python dependencies and spatialite libraries COPY --from=build /usr/local/lib/ /usr/local/lib/ ``` I had to use This works, sort of... I'm getting a weird issue where the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Updated Dockerfile with SpatiaLite version 5.0 824064069 | |
792383956 | https://github.com/simonw/datasette/issues/1249#issuecomment-792383956 | https://api.github.com/repos/simonw/datasette/issues/1249 | MDEyOklzc3VlQ29tbWVudDc5MjM4Mzk1Ng== | simonw 9599 | 2021-03-08T00:20:09Z | 2021-03-08T00:20:09Z | OWNER | Worth noting that the Docker image used by https://github.com/simonw/datasette/blob/d0fd833b8cdd97e1b91d0f97a69b494895d82bee/datasette/utils/init.py#L349-L353 Where the apt extras for SpatiaLite are: https://github.com/simonw/datasette/blob/d0fd833b8cdd97e1b91d0f97a69b494895d82bee/datasette/utils/init.py#L344-L345
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Updated Dockerfile with SpatiaLite version 5.0 824064069 | |
792233255 | https://github.com/simonw/datasette/pull/1223#issuecomment-792233255 | https://api.github.com/repos/simonw/datasette/issues/1223 | MDEyOklzc3VlQ29tbWVudDc5MjIzMzI1NQ== | simonw 9599 | 2021-03-07T07:41:01Z | 2021-03-07T07:41:01Z | OWNER | This is fantastic, thanks so much for tracking this down. |
{ "total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 1, "rocket": 0, "eyes": 0 } |
Add compile option to Dockerfile to fix failing test (fixes #696) 806918878 | |
790311215 | https://github.com/simonw/datasette/pull/1243#issuecomment-790311215 | https://api.github.com/repos/simonw/datasette/issues/1243 | MDEyOklzc3VlQ29tbWVudDc5MDMxMTIxNQ== | simonw 9599 | 2021-03-04T05:45:57Z | 2021-03-04T05:45:57Z | OWNER | Thanks! |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
fix small typo 815955014 | |
787616446 | https://github.com/simonw/datasette/issues/1247#issuecomment-787616446 | https://api.github.com/repos/simonw/datasette/issues/1247 | MDEyOklzc3VlQ29tbWVudDc4NzYxNjQ0Ng== | simonw 9599 | 2021-03-01T03:50:37Z | 2021-03-01T03:50:37Z | OWNER | I like the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
datasette.add_memory_database() method 818430405 | |
787616158 | https://github.com/simonw/datasette/issues/1247#issuecomment-787616158 | https://api.github.com/repos/simonw/datasette/issues/1247 | MDEyOklzc3VlQ29tbWVudDc4NzYxNjE1OA== | simonw 9599 | 2021-03-01T03:49:27Z | 2021-03-01T03:49:27Z | OWNER | A couple of options: ```python datasette.add_memory_database("test_json_array") or make that first argument to add_database() optional and support:datasette.add_database(memory_name="test_json_array") ``` |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
datasette.add_memory_database() method 818430405 | |
787611153 | https://github.com/simonw/datasette/issues/1246#issuecomment-787611153 | https://api.github.com/repos/simonw/datasette/issues/1246 | MDEyOklzc3VlQ29tbWVudDc4NzYxMTE1Mw== | simonw 9599 | 2021-03-01T03:30:57Z | 2021-03-01T03:30:57Z | OWNER | I'm going to try a new pattern for testing this, enabled by #1151 - the test will create a new named in-memory database, write some records to it and then run some test facets against that. This will save me from having to add yet another fixtures table for this. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Suggest for ArrayFacet possibly confused by blank values 817597268 | |
787536267 | https://github.com/simonw/datasette/issues/1005#issuecomment-787536267 | https://api.github.com/repos/simonw/datasette/issues/1005 | MDEyOklzc3VlQ29tbWVudDc4NzUzNjI2Nw== | simonw 9599 | 2021-02-28T22:30:37Z | 2021-02-28T22:30:37Z | OWNER | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Remove xfail tests when new httpx is released 718259202 | ||
787532279 | https://github.com/simonw/sqlite-utils/issues/242#issuecomment-787532279 | https://api.github.com/repos/simonw/sqlite-utils/issues/242 | MDEyOklzc3VlQ29tbWVudDc4NzUzMjI3OQ== | simonw 9599 | 2021-02-28T22:09:37Z | 2021-02-28T22:09:37Z | OWNER | Microsoft's playwright Python library solves this problem by code generating both their sync AND their async libraries https://github.com/microsoft/playwright-python/tree/master/scripts |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Async support 817989436 | |
787198202 | https://github.com/simonw/sqlite-utils/issues/242#issuecomment-787198202 | https://api.github.com/repos/simonw/sqlite-utils/issues/242 | MDEyOklzc3VlQ29tbWVudDc4NzE5ODIwMg== | simonw 9599 | 2021-02-27T22:33:58Z | 2021-02-27T22:33:58Z | OWNER | Hah or use this trick, which genuinely rewrites the code at runtime using a class decorator! https://github.com/python-happybase/aiohappybase/blob/0990ef45cfdb720dc987afdb4957a0fac591cb99/aiohappybase/sync/_util.py#L19-L32 |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Async support 817989436 | |
787195536 | https://github.com/simonw/sqlite-utils/issues/242#issuecomment-787195536 | https://api.github.com/repos/simonw/sqlite-utils/issues/242 | MDEyOklzc3VlQ29tbWVudDc4NzE5NTUzNg== | simonw 9599 | 2021-02-27T22:13:24Z | 2021-02-27T22:13:24Z | OWNER | Some other interesting background reading: https://docs.sqlalchemy.org/en/14/orm/extensions/asyncio.html - in particular see how SQLALchemy has a |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Async support 817989436 | |
787190562 | https://github.com/simonw/sqlite-utils/issues/242#issuecomment-787190562 | https://api.github.com/repos/simonw/sqlite-utils/issues/242 | MDEyOklzc3VlQ29tbWVudDc4NzE5MDU2Mg== | simonw 9599 | 2021-02-27T22:04:00Z | 2021-02-27T22:04:00Z | OWNER | From the poster here: https://github.com/sethmlarson/pycon-async-sync-poster/blob/master/poster.pdf |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Async support 817989436 | |
787186826 | https://github.com/simonw/sqlite-utils/issues/242#issuecomment-787186826 | https://api.github.com/repos/simonw/sqlite-utils/issues/242 | MDEyOklzc3VlQ29tbWVudDc4NzE4NjgyNg== | simonw 9599 | 2021-02-27T22:01:54Z | 2021-02-27T22:01:54Z | OWNER |
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Async support 817989436 | |
787175126 | https://github.com/simonw/sqlite-utils/issues/242#issuecomment-787175126 | https://api.github.com/repos/simonw/sqlite-utils/issues/242 | MDEyOklzc3VlQ29tbWVudDc4NzE3NTEyNg== | simonw 9599 | 2021-02-27T21:55:05Z | 2021-02-27T21:55:05Z | OWNER | "how to use some new tools to more easily maintain a codebase that supports both async and synchronous I/O and multiple async libraries" - yeah that's exactly what I need, thank you! |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Async support 817989436 |
Advanced export
JSON shape: default, array, newline-delimited, object
CREATE TABLE [issue_comments] ( [html_url] TEXT, [issue_url] TEXT, [id] INTEGER PRIMARY KEY, [node_id] TEXT, [user] INTEGER REFERENCES [users]([id]), [created_at] TEXT, [updated_at] TEXT, [author_association] TEXT, [body] TEXT, [reactions] TEXT, [issue] INTEGER REFERENCES [issues]([id]) , [performed_via_github_app] TEXT); CREATE INDEX [idx_issue_comments_issue] ON [issue_comments] ([issue]); CREATE INDEX [idx_issue_comments_user] ON [issue_comments] ([user]);
issue >1000