issue_comments

This should support multiple columns, e.g. ?_group_count=precinct&_group_count=candidate

Should this support sum/avg/etc as well?

Consider for example https://fivethirtyeight.datasettes.com/fivethirtyeight/inconvenient-sequel%2Fratings

The idea here is to be able to support querystring parameters like this:

• ?timestamp___date=2017-07-17 - return every item where the timestamp falls on that date
• ?timestamp___year=2017 - return every item where the timestamp falls within 2017
• ?timestamp___month=1 - return every item where the month component is January
• ?timestamp___day=10 - return every item where the day-of-the-month component is 10

This is similar to #64 but a fair bit more complicated.

SQLite date functions are documented here: https://sqlite.org/lang_datefunc.html

This was fixed by ed2b3f25beac720f14869350baacc5f62b065194 in #107 - thanks @raynae!

OK, I can confirm that the version in the new docker container supports FTS5, JSON and spatialite!

Notes on how I built the container and tested the spatialite extension are here: https://github.com/simonw/datasette/issues/112#issuecomment-345255655

To confirm that JSON and FTS5 are working, I ran the following:

$ docker run -it -p 8001:8001 6c9ca7e29181 python
Python 3.6.3 (default, Nov  4 2017, 14:24:48) 
[GCC 6.3.0 20170516] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sqlite3
>>> sqlite3.connect(':memory:').execute('CREATE VIRTUAL TABLE email USING fts5(sender, title, body);')
<sqlite3.Cursor object at 0x7f2d90839960>
>>> list(sqlite3.connect(':memory:').execute('''SELECT json(' { "this" : "is", "a": [ "test" ] } ') '''))
[('{"this":"is","a":["test"]}',)]

If I do the same thing in python3 on my OS X laptop directly, I get this:

$ python3
Python 3.5.1 (default, Apr 18 2016, 11:46:32) 
[GCC 4.2.1 Compatible Apple LLVM 7.3.0 (clang-703.0.29)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sqlite3
>>> sqlite3.connect(':memory:').execute('CREATE VIRTUAL TABLE email USING fts5(sender, title, body);')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
sqlite3.OperationalError: no such module: fts5
>>> list(sqlite3.connect(':memory:').execute('''SELECT json(' { "this" : "is", "a": [ "test" ] } ') '''))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
sqlite3.OperationalError: no such function: json

This is great - I've been frustrated by how CodeMirror prevents me from hitting tab-enter to activate the "Run SQL" button.

I tesed this by first building and running a container using the new Dockerfile from #114:

docker build .
docker run -it -p 8001:8001 6c9ca7e29181 /bin/sh

Then I ran this inside the container itself:

apt update && apt-get install wget -y \
    && wget http://www.gaia-gis.it/spatialite-2.3.1/test-2.3.sqlite.gz \
    && gunzip test-2.3.sqlite.gz \
    && mv test-2.3.sqlite test23.sqlite \
    && datasette -h 0.0.0.0 test23.sqlite

I visited this URL to confirm I got an error due to spatialite not being loaded:

http://localhost:8001/test23-c88bc35?sql=select+ST_AsText%28Geometry%29+from+HighWays+limit+1

Then I checked that loading it with --load-extension worked correctly:

datasette -h 0.0.0.0 test23.sqlite \
    --load-extension=/usr/lib/x86_64-linux-gnu/mod_spatialite.so

Then, finally, I tested it with the new environment variable option:

SQLITE_EXTENSIONS=/usr/lib/x86_64-linux-gnu/mod_spatialite.so \
    datasette -h 0.0.0.0 test23.sqlite

Running it with an invalid environment variable option shows an error:

$ SQLITE_EXTENSIONS=/usr/lib/x86_64-linux-gnu/blah.so datasette \
    -h 0.0.0.0 test23.sqlite
Usage: datasette -h [OPTIONS] [FILES]...

Error: Invalid value for "--load-extension": Path "/usr/lib/x86_64-linux-gnu/blah.so" does not exist.

I could support explicit label columns using additional arguments to datasette serve:

datasette serve mydb.py --label-column mydb:table1:name --label-column mydb:table2:title

This would mean "in mydb, set the label column for table1 to name, and the label column for table2 to title"

csvs-to-sqlite is now capable of generating databases with foreign key lookup tables: https://github.com/simonw/csvs-to-sqlite/releases/tag/0.3

We now have a Dockerfile that compiles spatialite! https://github.com/simonw/datasette/pull/114/commits/6c6b63d890529eeefcefb7ab126ea3bd7b2315c1

Fantastic! Thank you very much.

Looks like your tests are failing because of a bug which I fixed in https://github.com/simonw/datasette/commit/9199945a1bcec4852e1cb866eb3642614dd32a48 - if you rebase to master the tests should pass.