13 rows where issue = 271301468 sorted by updated_at descending

View and edit SQL

Suggested facets: reactions, created_at (date), updated_at (date)

author_association

issue

  • Dockerfile should build more recent SQLite with FTS5 and spatialite support · 13
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions issue performed_via_github_app
345259115 https://github.com/simonw/datasette/issues/46#issuecomment-345259115 https://api.github.com/repos/simonw/datasette/issues/46 MDEyOklzc3VlQ29tbWVudDM0NTI1OTExNQ== simonw 9599 2017-11-17T14:32:12Z 2017-11-17T14:32:12Z OWNER

OK, I can confirm that the version in the new docker container supports FTS5, JSON and spatialite!

Notes on how I built the container and tested the spatialite extension are here: https://github.com/simonw/datasette/issues/112#issuecomment-345255655

To confirm that JSON and FTS5 are working, I ran the following:

$ docker run -it -p 8001:8001 6c9ca7e29181 python
Python 3.6.3 (default, Nov  4 2017, 14:24:48) 
[GCC 6.3.0 20170516] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sqlite3
>>> sqlite3.connect(':memory:').execute('CREATE VIRTUAL TABLE email USING fts5(sender, title, body);')
<sqlite3.Cursor object at 0x7f2d90839960>
>>> list(sqlite3.connect(':memory:').execute('''SELECT json(' { "this" : "is", "a": [ "test" ] } ') '''))
[('{"this":"is","a":["test"]}',)]

If I do the same thing in python3 on my OS X laptop directly, I get this:

$ python3
Python 3.5.1 (default, Apr 18 2016, 11:46:32) 
[GCC 4.2.1 Compatible Apple LLVM 7.3.0 (clang-703.0.29)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sqlite3
>>> sqlite3.connect(':memory:').execute('CREATE VIRTUAL TABLE email USING fts5(sender, title, body);')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
sqlite3.OperationalError: no such module: fts5
>>> list(sqlite3.connect(':memory:').execute('''SELECT json(' { "this" : "is", "a": [ "test" ] } ') '''))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
sqlite3.OperationalError: no such function: json
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Dockerfile should build more recent SQLite with FTS5 and spatialite support 271301468  
345138347 https://github.com/simonw/datasette/issues/46#issuecomment-345138347 https://api.github.com/repos/simonw/datasette/issues/46 MDEyOklzc3VlQ29tbWVudDM0NTEzODM0Nw== simonw 9599 2017-11-17T03:52:25Z 2017-11-17T03:52:25Z OWNER
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Dockerfile should build more recent SQLite with FTS5 and spatialite support 271301468  
345002908 https://github.com/simonw/datasette/issues/46#issuecomment-345002908 https://api.github.com/repos/simonw/datasette/issues/46 MDEyOklzc3VlQ29tbWVudDM0NTAwMjkwOA== ingenieroariel 54999 2017-11-16T17:47:49Z 2017-11-16T17:47:49Z CONTRIBUTOR

I'll try to find alternatives to the Dockerfile option - I also think we should not use that old one without sources or license.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Dockerfile should build more recent SQLite with FTS5 and spatialite support 271301468  
344995571 https://github.com/simonw/datasette/issues/46#issuecomment-344995571 https://api.github.com/repos/simonw/datasette/issues/46 MDEyOklzc3VlQ29tbWVudDM0NDk5NTU3MQ== simonw 9599 2017-11-16T17:22:32Z 2017-11-16T17:22:32Z OWNER

The JSON extension would be very worthwhile too: https://www.sqlite.org/json1.html

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Dockerfile should build more recent SQLite with FTS5 and spatialite support 271301468  
344989340 https://github.com/simonw/datasette/issues/46#issuecomment-344989340 https://api.github.com/repos/simonw/datasette/issues/46 MDEyOklzc3VlQ29tbWVudDM0NDk4OTM0MA== simonw 9599 2017-11-16T17:02:07Z 2017-11-16T17:02:07Z OWNER

The fact that prolocutor/python3-sqlite-ext doesn't provide a visible Dockerfile and hasn't been updated in two years makes me hesitant to bake it into datasette itself. I'd rather put together a Dockerfile that enables the necessary extensions and can live in the datasette repository itself.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Dockerfile should build more recent SQLite with FTS5 and spatialite support 271301468  
344988591 https://github.com/simonw/datasette/issues/46#issuecomment-344988591 https://api.github.com/repos/simonw/datasette/issues/46 MDEyOklzc3VlQ29tbWVudDM0NDk4ODU5MQ== simonw 9599 2017-11-16T16:59:51Z 2017-11-16T16:59:51Z OWNER

OK, --load-extension is now a supported command line option - see #110 which includes my notes on how I manually tested it using the prolocutor/python3-sqlite-ext Docker image.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Dockerfile should build more recent SQLite with FTS5 and spatialite support 271301468  
344976882 https://github.com/simonw/datasette/issues/46#issuecomment-344976882 https://api.github.com/repos/simonw/datasette/issues/46 MDEyOklzc3VlQ29tbWVudDM0NDk3Njg4Mg== simonw 9599 2017-11-16T16:25:07Z 2017-11-16T16:25:07Z OWNER

Maybe part of the solution here is to add a --load-extension argument to datasette - so when you run the command you can specify SQLite extensions that should be loaded.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Dockerfile should build more recent SQLite with FTS5 and spatialite support 271301468  
344976104 https://github.com/simonw/datasette/issues/46#issuecomment-344976104 https://api.github.com/repos/simonw/datasette/issues/46 MDEyOklzc3VlQ29tbWVudDM0NDk3NjEwNA== simonw 9599 2017-11-16T16:22:45Z 2017-11-16T16:22:45Z OWNER
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Dockerfile should build more recent SQLite with FTS5 and spatialite support 271301468  
344975156 https://github.com/simonw/datasette/issues/46#issuecomment-344975156 https://api.github.com/repos/simonw/datasette/issues/46 MDEyOklzc3VlQ29tbWVudDM0NDk3NTE1Ng== simonw 9599 2017-11-16T16:19:44Z 2017-11-16T16:19:44Z OWNER

That's fantastic! Thank you very much for that.

Do you know if it's possible to view the Dockerfile used by https://hub.docker.com/r/prolocutor/python3-sqlite-ext/ ?

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Dockerfile should build more recent SQLite with FTS5 and spatialite support 271301468  
344810525 https://github.com/simonw/datasette/issues/46#issuecomment-344810525 https://api.github.com/repos/simonw/datasette/issues/46 MDEyOklzc3VlQ29tbWVudDM0NDgxMDUyNQ== ingenieroariel 54999 2017-11-16T04:11:25Z 2017-11-16T04:11:25Z CONTRIBUTOR

@simonw On the spatialite support, here is some info to make it work and a screenshot:

https://user-images.githubusercontent.com/54999/32873420-f8a6d5a0-ca59-11e7-8a73-7d58d467e413.png">

I used the following Dockerfile:

FROM prolocutor/python3-sqlite-ext:3.5.1-spatialite as build

RUN mkdir /code
ADD . /code/

RUN pip install /code/

EXPOSE 8001
CMD ["datasette", "serve", "/code/ne.sqlite", "--host", "0.0.0.0"]

and added this to prepare_connection:

  conn.enable_load_extension(True)
  conn.execute("SELECT load_extension('/usr/local/lib/mod_spatialite.so')")
{
    "total_count": 1,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 1,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Dockerfile should build more recent SQLite with FTS5 and spatialite support 271301468  
344161430 https://github.com/simonw/datasette/issues/46#issuecomment-344161430 https://api.github.com/repos/simonw/datasette/issues/46 MDEyOklzc3VlQ29tbWVudDM0NDE2MTQzMA== simonw 9599 2017-11-14T06:42:44Z 2017-11-14T06:42:44Z OWNER
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Dockerfile should build more recent SQLite with FTS5 and spatialite support 271301468  
344161371 https://github.com/simonw/datasette/issues/46#issuecomment-344161371 https://api.github.com/repos/simonw/datasette/issues/46 MDEyOklzc3VlQ29tbWVudDM0NDE2MTM3MQ== simonw 9599 2017-11-14T06:42:15Z 2017-11-14T06:42:15Z OWNER
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Dockerfile should build more recent SQLite with FTS5 and spatialite support 271301468  
344161226 https://github.com/simonw/datasette/issues/46#issuecomment-344161226 https://api.github.com/repos/simonw/datasette/issues/46 MDEyOklzc3VlQ29tbWVudDM0NDE2MTIyNg== simonw 9599 2017-11-14T06:41:21Z 2017-11-14T06:41:21Z OWNER

Spatial extensions would be really useful too. https://www.gaia-gis.it/spatialite-2.1/SpatiaLite-manual.html

{
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Dockerfile should build more recent SQLite with FTS5 and spatialite support 271301468  

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options:

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
, [performed_via_github_app] TEXT);
CREATE INDEX [idx_issue_comments_issue]
                ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
                ON [issue_comments] ([user]);