# milestones

title: Datasette 1.0
html_url: https://github.com/simonw/datasette/milestone/7
node_id: MDk6TWlsZXN0b25lMzI2ODMzMA==
state: open
created_at: 2018-04-15T21:11:18Z
updated_at: 2023-11-06T04:06:12Z
description:
  https://github.com/simonw/datasette/issues/519 - stable JSON API, documented template context, plugin API guarantee that plugins will not break until a 2.0 release.


title: Datasette 0.29
html_url: https://github.com/simonw/datasette/milestone/11
node_id: MDk6TWlsZXN0b25lNDQ3MTAxMA==
state: closed
created_at: 2019-07-07T19:59:16Z
updated_at: 2019-07-08T03:36:57Z
due_on: 2019-07-07T07:00:00Z
closed_at: 2019-07-08T03:36:57Z
description:
  Big new features in this release are ASGI support, new plugin hooks for ASGI and extra template variables (to support release of [datasette-auth-github](https://github.com/simonw/datasette-auth-github)), plus facet-by-date.


title: Datasette 0.44
html_url: https://github.com/simonw/datasette/milestone/14
node_id: MDk6TWlsZXN0b25lNTUxMjM5NQ==
state: closed
created_at: 2020-06-06T00:41:56Z
updated_at: 2020-06-12T22:05:31Z
closed_at: 2020-06-12T22:05:31Z
description:
  Writeable canned queries, authentication and permissions. [Changelog](https://datasette.readthedocs.io/en/latest/changelog.html#v0-44).


title: 0.51
html_url: https://github.com/simonw/datasette/milestone/19
node_id: MDk6TWlsZXN0b25lNjAyNjA3MA==
state: closed
created_at: 2020-10-23T19:42:45Z
updated_at: 2021-11-20T19:35:48Z
closed_at: 2020-10-31T22:35:05Z
description:
  A new visual design, plugin hooks for adding navigation options, better handling of binary data, URL building utility methods and better support for running Datasette behind a proxy.


# issues

node_id: MDU6SXNzdWUyNjc1MTM0MjQ=
title: Addressable pages for every row in a table
state: closed
created_at: 2017-10-23T00:44:16Z
updated_at: 2017-10-24T14:11:04Z
closed_at: 2017-10-24T14:11:03Z
author_association: OWNER
type: issue
state_reason: completed
body:
      /database-name-7sha256/table-name/compound-pk
      /database-name-7sha256/table-name/compound-pk.json

  Tricky part will be figuring out what the private key is - especially since it could be a compound primary key and it might involve different data types.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNjc1MTM1MjM=
title: Initial proof-of-concept
state: closed
created_at: 2017-10-23T00:45:37Z
updated_at: 2017-10-23T01:26:39Z
closed_at: 2017-10-23T00:45:53Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Implemented in https://github.com/simonw/stateless-datasets/commit/de04d7a854d71003ffcf98028eab976a936c2dba

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/2/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNjc1MTU2Nzg=
title: Make individual column valuables addressable, with smart content types
state: open
created_at: 2017-10-23T01:11:32Z
updated_at: 2017-12-10T03:11:58Z
author_association: OWNER
type: issue
body:
  Some SQLite databases embed images in columns. It would be cool if these had URLs.

      /database-name-7sha256/table-name/compound-pk/column
      /database-name-7sha256/table-name/compound-pk/column.json
      /database-name-7sha256/table-name/compound-pk/column.png
      /database-name-7sha256/table-name/compound-pk/column.gif
      /database-name-7sha256/table-name/compound-pk/column.txt

  The one without an explicit file extension auto-detects the correct extension.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/3/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNjc1MTU4MzY=
title: Make URLs immutable
state: closed
created_at: 2017-10-23T01:13:30Z
updated_at: 2017-10-24T02:38:24Z
closed_at: 2017-10-24T02:38:24Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Absolutely everything should have a far-future expires header

  Part of the URL will be the truncated sha1 hash of the database file itself, calculated at build time

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/4/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNjc1MTYwNjY=
title: Implement sensible query pagination
state: closed
created_at: 2017-10-23T01:16:00Z
updated_at: 2017-11-10T20:41:39Z
closed_at: 2017-11-10T20:41:39Z
author_association: OWNER
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/5/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNjc1MTYzMjk=
title: Better JSON response options
state: closed
created_at: 2017-10-23T01:18:47Z
updated_at: 2017-10-24T15:07:58Z
closed_at: 2017-10-24T15:07:58Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Default returns this:

      {
          “Columns”: [“id”, “name”, “age”],
          “Rows”: [
               [45, “Simon”, 36]
          ]
      }

  .jsono instead returns a list of objects each duplicating the headers in its keys.

  They both probably share the same pagination mechanism so it might not be a jsono flat list.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/6/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNjc1MTY2NTA=
title: Framework where by every page is JSON plus a template
state: closed
created_at: 2017-10-23T01:22:03Z
updated_at: 2017-10-24T02:27:25Z
closed_at: 2017-10-24T02:27:25Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Every single page of my interface should be implemented as a function that returns JSON.

  I can then build my jinja templates on top of the exact data that would be returned by the API version.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/7/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNjc1MTczMTQ=
title: Attempting an INSERT or UPDATE should return a sane error message
state: closed
created_at: 2017-10-23T01:28:25Z
updated_at: 2017-10-23T15:28:12Z
closed_at: 2017-10-23T15:28:08Z
author_association: OWNER
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/8/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNjc1MTczNDg=
title: Initial test suite
state: closed
created_at: 2017-10-23T01:28:46Z
updated_at: 2017-10-24T05:55:33Z
closed_at: 2017-10-24T05:55:33Z
author_association: OWNER
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/9/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNjc1MTczODE=
title: Set up Travis
state: closed
created_at: 2017-10-23T01:29:07Z
updated_at: 2017-11-04T23:48:57Z
closed_at: 2017-11-04T23:48:57Z
author_association: OWNER
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/10/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNjc1MjI1NDk=
title: Code that generates compile-time properties about the database
state: closed
created_at: 2017-10-23T02:18:24Z
updated_at: 2017-10-23T16:04:23Z
closed_at: 2017-10-23T16:04:23Z
author_association: OWNER
type: issue
state_reason: completed
body:
  At a minimum this will include:

  * sha hash of each database file
  * list of tables with row counts for each database file

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/11/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNjc1MjM1MTE=
title: Make it so you can override templates
state: closed
created_at: 2017-10-23T02:25:35Z
updated_at: 2017-11-30T16:42:46Z
closed_at: 2017-11-30T16:38:34Z
author_association: OWNER
type: issue
state_reason: completed
body:
  The app will ship with default templates but, just like with the Django admin, you will be able to override them using either explicit configuration settings or just by dropping in templates with certain file names.

  Template inheritance should work here, both allowing you to override just the base template and allowing you to customize tiny bits of others.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/12/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNjc1NDIzMzg=
title: Add a syntax highlighting SQL editor
state: closed
created_at: 2017-10-23T05:03:33Z
updated_at: 2017-11-15T02:04:51Z
closed_at: 2017-11-15T02:04:51Z
author_association: OWNER
body: https://ace.c9.io/#nav=embedding looks like a good option
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/13/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNjc3MDc5NDA=
title: Datasette Plugins
state: closed
created_at: 2017-10-23T15:15:28Z
updated_at: 2019-05-13T18:58:20Z
closed_at: 2019-05-13T18:58:19Z
author_association: OWNER
type: issue
state_reason: completed
body:
  It would be neat if additional functionality could be opted-in to the system in the form of easy-to-add plugins, hosted as separate packages. First example: a Google Analytics plugin, which adds GA tracking code with your tracking ID to the web interface for your dataset.

  This may be an opportunity to experiment with entry points: http://amir.rachum.com/blog/2017/07/28/python-entry-points/

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/14/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNjc3MTMyMjY=
title: Support multiple databases
state: closed
created_at: 2017-10-23T15:29:51Z
updated_at: 2017-10-24T02:01:38Z
closed_at: 2017-10-24T02:01:38Z
author_association: OWNER
type: issue
state_reason: completed
body:
  I'm going to loop through every database file in the app root directory and bundle all of them.

  Each one will be accessible at /databasename

  Note this is without the file extension, and we will disallow multiple files with the same name but different extensions.

  Supported extensions to start with will be `.db` and `.sqlite` and `.sqlite3`

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/15/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNjc3MjYyMTk=
title: Default HTML/CSS needs to look reasonable and be responsive
state: closed
created_at: 2017-10-23T16:05:22Z
updated_at: 2017-11-11T20:19:07Z
closed_at: 2017-11-11T20:19:07Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Version one should have the following characteristics:

  - Looks OK
  - Works great on mobile
  - Loads extremely fast
  - No JavaScript! At least not in v1.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/16/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNjc3MzIwMDU=
title: In development mode, should still pick up new .db files
state: closed
created_at: 2017-10-23T16:22:40Z
updated_at: 2017-10-24T02:26:48Z
closed_at: 2017-10-24T02:26:47Z
author_association: OWNER
body: Follow on from #11
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/17/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNjc3Mzk1OTM=
title: See if I can get a websockets interface working
state: closed
created_at: 2017-10-23T16:46:41Z
updated_at: 2021-01-04T20:05:52Z
closed_at: 2021-01-04T20:05:48Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Since I am already running on Sanic, how hard would it be to add a websocket ebdpoint that lets you talk to sqlite interactively?

  Could this be used to efficiently support streaming in answers to giant queries?

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/18/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNjc3NDEyNjI=
title: Efficient url for downloading the raw database file
state: closed
created_at: 2017-10-23T16:52:17Z
updated_at: 2017-10-25T15:21:16Z
closed_at: 2017-10-25T15:19:37Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Use Sanic support for steaming large files http://sanic.readthedocs.io/en/latest/sanic/response.html#file-streaming

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/19/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNjc3NTkxMzY=
title: Config file with support for defining canned queries
state: closed
created_at: 2017-10-23T17:53:06Z
updated_at: 2017-12-05T19:05:35Z
closed_at: 2017-12-05T17:44:09Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Probably using YAML because then we get support for multiline strings:

      bats:
        db: bats.sqlite3
        name: "Bat sightings"
        queries:
          specific_row: |
            select * from Bats
            where a = 1;

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/20/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNjc3NjkwMzQ=
title: Use Sanic configuration mechanism
state: closed
created_at: 2017-10-23T18:25:14Z
updated_at: 2017-11-10T20:45:42Z
closed_at: 2017-11-10T20:45:42Z
author_association: OWNER
body: http://sanic.readthedocs.io/en/latest/sanic/config.html
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/21/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNjc3Njk0MzE=
title: Refactor to use class based views
state: closed
created_at: 2017-10-23T18:26:22Z
updated_at: 2019-05-27T20:05:56Z
closed_at: 2017-10-24T02:25:53Z
author_association: OWNER
body: http://sanic.readthedocs.io/en/latest/sanic/class_based_views.html
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/22/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNjc3ODg4ODQ=
title: Support Django-style filters in querystring arguments
state: closed
created_at: 2017-10-23T19:29:42Z
updated_at: 2017-10-25T04:23:03Z
closed_at: 2017-10-25T04:23:02Z
author_association: OWNER
type: issue
state_reason: completed
body:
  e.g

      /database/table?name__contains=Simon&age__gte=4

  Same format as Django: double underscore as the split.

  If you need to match against a column that happens to contain a double underscore in its official name, do this:

      /database/table?weird__column__exact=Simon

  __exact is the default operation if none is supplied.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/23/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNjc4Mjg3NDY=
title: Implement full URL design
state: closed
created_at: 2017-10-23T21:49:05Z
updated_at: 2017-10-24T14:12:00Z
closed_at: 2017-10-24T14:12:00Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Full URL design:

      /database-name
      /database-name.json
      /database-name-7sha256
      /database-name-7sha256.json
      /database-name/table-name
      /database-name/table-name.json
      /database-name-7sha256/table-name
      /database-name-7sha256/table-name.json
      /database-name-7sha256/table-name/compound-pk
      /database-name-7sha256/table-name/compound-pk.json

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/24/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNjc4NTc2MjI=
title: Endpoint that returns SQL ready to be piped into DB
state: closed
created_at: 2017-10-24T00:19:26Z
updated_at: 2017-11-15T05:11:12Z
closed_at: 2017-11-15T05:11:11Z
author_association: OWNER
type: issue
state_reason: completed
body:
  It would be cool if I could figure out a way to generate both the create table statements and the inserts for an individual table or the entire database  and then stream them down to the client.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/25/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNjc4NjEyMTA=
title: Command line tool for uploading one or more DBs to Now
state: closed
created_at: 2017-10-24T00:43:10Z
updated_at: 2017-11-11T07:25:30Z
closed_at: 2017-11-11T07:25:30Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Uploading files appears to be undocumented, but I found it in their code here: https://github.com/zeit/now-cli/blob/0ca7d1fe44ebdf460b64fdc38ba543b8e295ac40/src/providers/sh/util/index.js#L291

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/26/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNjc4ODYzMzA=
title: Ability to plot a simple graph
state: closed
created_at: 2017-10-24T03:34:59Z
updated_at: 2018-07-10T17:52:41Z
closed_at: 2018-07-10T17:52:41Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Might be as simple as: pick he type of chart (bar, line) and then pick the column for the X axis and the column for the Y axis. Maybe also allow a pie chart. It’s up to the user to come up with SQL that gets the right values.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/27/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNjc4ODY4NjU=
title: /database?sql= should redirect correctly
state: closed
created_at: 2017-10-24T03:38:44Z
updated_at: 2017-10-24T23:54:30Z
closed_at: 2017-10-24T23:54:30Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Needs to redirect to the location with the hash while retaining the query string. This should also work with the .json extension.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/28/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNjgwNTA4MjE=
title: Handle bytestring records encoding to JSON
state: closed
created_at: 2017-10-24T14:18:45Z
updated_at: 2017-10-24T14:59:00Z
closed_at: 2017-10-24T14:58:47Z
author_association: OWNER
type: issue
state_reason: completed
body:
  http://localhost:8006/northwind-40d049b/Categories.json 500s right now

  The string representation of one of the values looks like this:

      b"\x15\x1c/\x00\x02\x00

  This is a bytestring from the database which cannot be naively converted to a unicode string.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/29/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNjgwNzg0NTM=
title: Do something neat with foreign keys
state: closed
created_at: 2017-10-24T15:29:29Z
updated_at: 2017-11-14T18:29:08Z
closed_at: 2017-11-14T18:29:01Z
author_association: OWNER
type: issue
state_reason: completed
body:
  https://www.sqlite.org/pragma.html#pragma_foreign_key_list

  SQLite has robust support for introspecting foreign keys. I could use that to automatically link to the corresponding record from my tables.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/30/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNjgwODc1NDI=
title: Idea: colour scheme based on sha256 of db
state: closed
created_at: 2017-10-24T15:52:38Z
updated_at: 2018-05-28T18:10:45Z
closed_at: 2017-11-09T14:14:59Z
author_association: OWNER
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/31/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNjgxMDY4MDM=
title: Try running SQLite queries in a separate thread
state: closed
created_at: 2017-10-24T16:48:42Z
updated_at: 2017-11-09T14:05:56Z
closed_at: 2017-11-09T14:05:56Z
author_association: OWNER
type: issue
state_reason: completed
body:
  https://pymotw.com/3/asyncio/executors.html

  Would be good to have some actual benchmarks so I can evaluate if this is worth it or not.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/32/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNjgxMTA3Njk=
title: Use locust for benchmarking and load tests
state: open
created_at: 2017-10-24T17:00:09Z
updated_at: 2017-12-10T03:12:16Z
author_association: OWNER
body:
  https://github.com/locustio/locust

  Needed for #32

type: issue
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/33/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNjgxNzY1MDU=
title: Support CSV export with a .csv extension
state: closed
created_at: 2017-10-24T20:34:43Z
updated_at: 2021-06-17T18:14:48Z
closed_at: 2018-05-28T20:45:34Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Maybe do this using streaming with multiple pagination SQL queries so we can support arbritrarily large exports.

  How would this work against a view which doesn’t have an obvious efficient pagination mechanism? Maybe limit views to up to 1000 exported records?

  Relates to #5 

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/34/reactions", "total_count": 2, "+1": 2, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNjgyNjI0ODA=
title: date, year, month and day querystring lookups
state: closed
created_at: 2017-10-25T04:23:45Z
updated_at: 2018-05-28T17:30:53Z
closed_at: 2018-05-28T17:30:53Z
author_association: OWNER
type: issue
state_reason: completed
body:
  - [ ] `?timestamp___date=2017-07-17` - return every item where the timestamp falls on that date
  - [ ] `?timestamp___year=2017` - return every item where the timestamp falls within 2017
  - [ ] `?timestamp___month=1` - return every item where the month component is January
  - [ ] `?timestamp___day=10` - return every item where the day-of-the-month component is 10

  Follow on from #23 

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/36/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNjg0NTM5Njg=
title: Ability to serialize massive JSON without blocking event loop
state: closed
created_at: 2017-10-25T15:58:03Z
updated_at: 2020-05-30T17:29:20Z
closed_at: 2020-05-30T17:29:20Z
author_association: OWNER
type: issue
state_reason: completed
body:
  We run the risk of someone attempting a select statement that returns thousands of rows and hence takes several seconds just to JSON encode the response, effectively blocking the event loop and pausing all other traffic.

  The Twisted community have a solution for this, can we adapt that in some way? http://as.ynchrono.us/2010/06/asynchronous-json_18.html?m=1

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/37/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNjg0NjI3Njg=
title: Experiment with patterns for concurrent long running queries
state: closed
created_at: 2017-10-25T16:23:42Z
updated_at: 2018-05-28T20:47:31Z
closed_at: 2018-05-28T20:47:31Z
author_association: OWNER
type: issue
state_reason: completed
body:
  I want to understand how the system could perform under load with many concurrent long-running queries. Can we serve these without blocking the event loop?

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/38/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNjg0Njk1Njk=
title: Protect against malicious SQL that causes damage even though our DB is immutable
state: closed
created_at: 2017-10-25T16:44:27Z
updated_at: 2021-08-17T23:52:07Z
closed_at: 2017-11-05T02:53:47Z
author_association: OWNER
type: issue
state_reason: completed
body:
  I’m currently operating under the assumption that it’s safe to allow arbitrary SQL statements because we are dealing with an immutable database. But this might not be the case - there are some pretty weird SQLite language extensions (ATTACH, PRAGMA etc) and I’m not certain they cannot be used to break things in a way that would affect future requests to the API.

  Solution: provide a “safe mode” option which disables the ?sql= mechanism. This still leaves the URL filter lookups, so I need to make sure that those are “safe”.

  In the future I may also implement a whitelist option where datasets can be configured to only allow specific filters against specific columns.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/39/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNjg0NzA1NzI=
title: Implement command-line tool interface
state: closed
created_at: 2017-10-25T16:47:15Z
updated_at: 2017-11-11T07:27:33Z
closed_at: 2017-11-11T07:27:33Z
author_association: OWNER
type: issue
state_reason: completed
body:
  The first version needs to take one or more file names or URLs, then generate and deploy an app to Now. It will assume you already have the now command installed and configured.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/40/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNjg1OTA3Nzc=
title: Homepage should show summary of databases
state: closed
created_at: 2017-10-26T00:18:11Z
updated_at: 2017-10-27T04:05:35Z
closed_at: 2017-10-27T04:05:35Z
author_association: OWNER
type: issue
state_reason: completed
body:
  I sch database should have a name, optional description, download link and a summary of the tables

  Flights.db
  Flights and suchlike blah. 
  URL? License?
  577373 rows across 14 tables
  airports, routes, airlines... 

  Title of the homepage is derived from the databases or can be manually overridden e. “Datasets of Flights, NHS, Blah...” - or if only one database just the title of that.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/41/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNjg1OTEzMzI=
title: Homepage UI for editing metadata file
state: closed
created_at: 2017-10-26T00:22:03Z
updated_at: 2017-12-10T03:02:14Z
closed_at: 2017-12-10T03:02:14Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Since we are going to have a metadata file which sets the title/description/etc for each database, why not allow you to run the app in —dev mode which makes the homepage into a WYSIWYG editor that  can save to that file format.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/42/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNjg1OTI4OTQ=
title: While running, server should spot new db files added to its directory
state: closed
created_at: 2017-10-26T00:32:37Z
updated_at: 2017-11-14T08:25:53Z
closed_at: 2017-11-14T08:25:37Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Maybe in each request it checks the time and if 5s has elapsed since t last scanned the directory it scans it again

  This would allow people with dedicated hosting to run the app there and just upload new datasets whenever they want. It would also be very convenient for development.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/43/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNjk3MzEzNzQ=
title: ?_group_count=country - return counts by specific column(s)
state: closed
created_at: 2017-10-30T19:50:32Z
updated_at: 2018-04-26T15:09:58Z
closed_at: 2018-04-26T15:09:58Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Imagine if this:

  https://stateless-datasets-jykibytogk.now.sh/flights-07d1283/airports.jsono?country__contains=gu&_group_count=country

  Turned into this:

  https://stateless-datasets-jykibytogk.now.sh/flights-07d1283?sql=select%20country,%20count(*)%20as%20group_count_country%20from%20airports%20where%20country%20like%20%27%gu%%27%20group%20by%20country%20order%20by%20group_count_country%20desc

  This would involve introducing a new precedent of query string arguments that start with an _ having special meanings. While we're at it, could try adding _fields=x,y,z

  Tasks:

  - [x] Get initial version working
  - [ ] Refactor code to not just "pretend to be a view"
  - [ ] Get foreign key relationships expanded

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/44/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzEyNDI4MjQ=
title: Run SQLite operations in a thread pool
state: closed
created_at: 2017-11-05T02:27:12Z
updated_at: 2017-11-05T02:27:34Z
closed_at: 2017-11-05T02:27:33Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Let's run SQLite operations in threads, so we don't end up blocking our core event loop.

  These articles are helpful:

  * https://pymotw.com/3/asyncio/executors.html
  * https://marlinux.wordpress.com/2017/05/19/python-3-6-asyncio-sqlalchemy/

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/45/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzEzMDE0Njg=
title: Dockerfile should build more recent SQLite with FTS5 and spatialite support
state: closed
created_at: 2017-11-05T18:16:22Z
updated_at: 2017-11-17T14:32:12Z
closed_at: 2017-11-17T14:32:12Z
author_association: OWNER
type: issue
state_reason: completed
body:
  The SQLite bundled with Python 3 doesn't support the FTS5 search extension. It would be nice if the SQLite built by our Dockerfile could support as many modern SQLite features as possible.

  https://web.archive.org/web/20170212034155/http://charlesleifer.com/blog/using-the-sqlite-json1-and-fts5-extensions-with-python/ has instructions on building a more recent SQLite and the pysqlite package. Our Dockerfile could carry out an updated version of this process.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/46/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzE4MzE0MDg=
title: Create neat example database
state: closed
created_at: 2017-11-07T13:29:38Z
updated_at: 2017-11-14T03:08:13Z
closed_at: 2017-11-14T03:08:13Z
author_association: OWNER
type: issue
state_reason: completed
body:
  How about data from open elections  eg https://github.com/openelections/openelections-data-ca?files=1

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/47/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzIzOTE2NjU=
title: Switch to ujson
state: closed
created_at: 2017-11-08T23:50:29Z
updated_at: 2019-06-24T06:57:54Z
closed_at: 2019-06-24T06:57:43Z
author_association: OWNER
body: ujson is already a dependency of Sanic, and should be quite a bit faster.
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/48/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzI2NjEzMzY=
title: Pick a name
state: closed
created_at: 2017-11-09T17:56:17Z
updated_at: 2017-11-10T18:33:22Z
closed_at: 2017-11-10T18:33:22Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Options so far:

  * immutabase
  * datasite
  * sqlstatic
  * dbserve
  * sqlserve

  Terms to play with:

  * immutable
  * sqlite
  * dataset
  * json
  * static
  * serve

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/49/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzI2OTQxMzY=
title: Unit tests against application itself
state: closed
created_at: 2017-11-09T19:31:49Z
updated_at: 2017-11-11T22:23:22Z
closed_at: 2017-11-11T22:23:22Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Use Sanic’s testing mechanism. Test should create a temporary SQLite database file on disk by executing sql that is stored in the test themselves.

  For the moment we can just test the JSON API more thoroughly and just sanity check that the HTML output doesn’t throw any errors.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/50/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzI3MzUyNTc=
title: Make a proper README
state: closed
created_at: 2017-11-09T21:46:07Z
updated_at: 2017-11-13T18:44:23Z
closed_at: 2017-11-13T18:44:23Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Include instructions on building a local Docker container - currently detailed here: https://gist.github.com/simonw/0ea5c960608c2d876e4637a5e48aa95d (those instructions don't work now that we have removed the Dockerfile in favour of a template generated by `datasette publish`)

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/51/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzMwMjY2MDI=
title: Solution for temporarily uploading DB so it can be built by docker
state: closed
created_at: 2017-11-10T18:55:25Z
updated_at: 2017-12-10T03:02:57Z
closed_at: 2017-12-10T03:02:57Z
author_association: OWNER
type: issue
state_reason: completed
body:
  For the `datasette publish` command I ideally need a way of uploading the specified DB to somewhere temporary on the internet so that when the Dockerfile is built by the final hosting location it can download that database as part of the build process.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/52/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzMwNTQ2NTI=
title: Implement a better database index page
state: closed
created_at: 2017-11-10T20:47:36Z
updated_at: 2017-11-12T21:19:33Z
closed_at: 2017-11-12T01:50:27Z
author_association: OWNER
type: issue
state_reason: completed
body:
  This view isn't great. I should do a better job of separating out tables from views and indexes, showing the count of rows in each table, and maybe move the SQL to the individual table pages.

  <img width="871" alt="flights" src="https://user-images.githubusercontent.com/9599/32423242-1b4458ce-c25a-11e7-910f-2dc1de909b8f.png">

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/53/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzMxMjE4MDM=
title: Views should not attempt to link to records / use rowids
state: closed
created_at: 2017-11-11T05:44:54Z
updated_at: 2017-11-12T21:29:42Z
closed_at: 2017-11-12T21:29:33Z
author_association: OWNER
type: issue
state_reason: completed
body:
  http://localhost:8001/parlgov-development-25f9855/view_variable

  <img width="837" alt="parlgov-development__view_variable" src="https://user-images.githubusercontent.com/9599/32686757-5b40f6a8-c660-11e7-88de-5e8dfb12ccf1.png">

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/54/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzMxMjcxMTc=
title: Ship first version to PyPI
state: closed
created_at: 2017-11-11T07:38:48Z
updated_at: 2017-11-13T21:19:43Z
closed_at: 2017-11-13T21:19:43Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Just before doing this, update the Dockerfile template to `pip install datasette` https://github.com/simonw/datasette/blob/65e350ca2a4845c25752a62c16ba58cfe2c14b9b/datasette/utils.py#L125

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/55/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzMxMjc0NDM=
title: Easy way to block search engine crawling in robots.txt
state: closed
created_at: 2017-11-11T07:46:07Z
updated_at: 2018-05-28T20:50:25Z
closed_at: 2018-05-28T20:50:24Z
author_association: OWNER
body: For people who don't want their datasets to be crawled by search engines.
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/56/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzMxMjc2OTQ=
title: Ship a Docker image of the whole thing
state: closed
created_at: 2017-11-11T07:51:28Z
updated_at: 2018-06-28T04:01:51Z
closed_at: 2018-06-28T04:01:38Z
author_association: OWNER
type: issue
state_reason: completed
body:
  The generated Docker images can then just inherit from that. This will speed up deploys as no need to `pip install` anything.

  - [x] Ship that image to Docker Hub
  - [ ] Update the generated Dockerfile to use it

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/57/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzMxMjg2MDg=
title: publish command should detect if "now" is installed
state: closed
created_at: 2017-11-11T08:10:17Z
updated_at: 2017-11-11T16:00:07Z
closed_at: 2017-11-11T16:00:07Z
author_association: OWNER
body: If now is not installed, it should tell you where to get it.
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/58/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzMxNTcwODU=
title: datasette publish hyper
state: closed
created_at: 2017-11-11T16:27:26Z
updated_at: 2019-05-13T19:01:00Z
closed_at: 2019-05-13T19:00:44Z
author_association: OWNER
type: issue
state_reason: completed
body:
  This is a bit tricky, because unlike Now there doesn't seem to be a way to tell Hyper to "build this Dockerfile and deploy the resulting image". They expect you to build a container and publish it to a registry instead.

  https://docs.hyper.sh/Reference/CLI/load.html allows you to publish an image directly from a tarball, but that still leaves the challenge of creating that image. The nice thing about the Now integration is that you don't need to have Docker installed on your local machine.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/59/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzMxNjM5MDU=
title: Rethink how metadata is generated and stored
state: closed
created_at: 2017-11-11T18:01:28Z
updated_at: 2017-11-11T20:12:17Z
closed_at: 2017-11-11T20:12:16Z
author_association: OWNER
type: issue
state_reason: completed
body:
  I broke the existing mechanism in 407795b61217205625f2d4e084afbf69f1db781b In order to get unit tests for the sanic app working.

  I think i should ditch the build-metadata.json cache file entirely and calculate the SHA hashes on startup.

  Not sure what to do about the table row counts.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/60/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzMxNzMxMTY=
title: Common header and footer
state: closed
created_at: 2017-11-11T20:20:08Z
updated_at: 2017-11-11T20:37:19Z
closed_at: 2017-11-11T20:37:19Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Split from #16 

  - [x] A link to the homepage from some kind of navigation bar in the header
  - [x] link to github.com/simonw/datasette in the footer
  - [x] Slightly better titles (maybe ditch the visited link colours for titles only? should keep those for primary key links)

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/61/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzMxNzQzOTc=
title: Link to .json and .jsono versions on various pages
state: closed
created_at: 2017-11-11T20:37:47Z
updated_at: 2017-11-11T22:41:06Z
closed_at: 2017-11-11T22:41:06Z
author_association: OWNER
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/62/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzMxNzQ0NDc=
title: Review design of JSON output
state: closed
created_at: 2017-11-11T20:38:33Z
updated_at: 2017-11-11T22:20:17Z
closed_at: 2017-11-11T22:20:17Z
author_association: OWNER
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/63/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzMxODEwMjA=
title: Support for ?field__isnull=1 or similar
state: closed
created_at: 2017-11-11T22:26:52Z
updated_at: 2017-11-17T14:38:21Z
closed_at: 2017-11-17T14:38:21Z
author_association: OWNER
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/64/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzMxOTE2MDg=
title: Re-implement ?sql= mode
state: closed
created_at: 2017-11-12T01:47:17Z
updated_at: 2017-11-12T02:36:37Z
closed_at: 2017-11-12T02:35:42Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Here's the code I removed:

      async def data(self, request, name, hash):
          sql = 'select * from sqlite_master'
          custom_sql = False
          params = {}
          if request.args.get('sql'):
              params = request.raw_args
              sql = params.pop('sql')
              validate_sql_select(sql)
              custom_sql = True
          rows = await self.execute(name, sql, params)
          columns = [r[0] for r in rows.description]
          return {
              'database': name,
              'rows': rows,
              'columns': columns,
              'query': {
                  'sql': sql,
                  'params': params,
              }
          }, {
              'database_hash': hash,
              'custom_sql': custom_sql,
          }

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/65/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzMxOTE4MDY=
title: Show table SQL on table page
state: closed
created_at: 2017-11-12T01:51:23Z
updated_at: 2017-11-12T21:17:29Z
closed_at: 2017-11-12T21:17:29Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Let's do the SQL for the table you are looking at, plus SQL for any indexes that mention that table.

  The page for  a view should show the SQL for that view.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/66/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzMxOTI3ODk=
title: Command that builds a local docker container
state: closed
created_at: 2017-11-12T02:13:29Z
updated_at: 2017-11-13T16:17:52Z
closed_at: 2017-11-13T16:17:52Z
author_association: OWNER
body: Be nice to indicate that this isn't just for Now. Shouldn't be too hard either.
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/67/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzMyNDcxODY=
title: Support for title/source/license metadata
state: closed
created_at: 2017-11-12T17:04:21Z
updated_at: 2017-12-04T04:55:43Z
closed_at: 2017-11-13T15:26:11Z
author_association: OWNER
type: issue
state_reason: completed
body:
  I've decided this is important for launch: I want to set a precedent for people citing, licensing and documenting their datasets.

  Not sure how best to go about supporting this. I'd like to allow for the following data to be optionally attached to any given database:

  - Title
  - Description, potentially in markdown?
  - Original source URL
  - License

  I'd also like the ability to attach descriptions to individual tables - and maybe even to table columns?

  The question then becomes: how should this information be stored. A few options:

  - In the SQLite database itself, in a specially named table. Problem here is that this means having to modify SQLite databases before publishing them.
  - In a separate SQLite database that can be published alongside the databases we are publishing.
  - In a JSON file. This is neat, but JSON files are not a great editing experience once you start including multiple lines (e.g. a markdown description).
  - In a YAML file. This is a better format for multi-line descriptions, but still isn't a great editing experience.

  Whatever the format, it can be made much more usable by offering a web-based editing UI for populating it (a special mode the server can be run in).

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/68/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzMyNDgzNjY=
title: Enforce pagination (or at least limits) for arbitrary custom SQL
state: closed
created_at: 2017-11-12T17:21:33Z
updated_at: 2017-11-13T20:32:47Z
closed_at: 2017-11-13T19:35:47Z
author_association: OWNER
type: issue
state_reason: completed
body:
  It's way too easy to accidentally trigger a page that returns 100,000 rows at the moment. I need to use the LIMIT clause on views and custom SQL - I can support pagination "next" links using offset as well.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/69/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzMyNjcwODE=
title: Paginate views using OFFSET/LIMIT
state: closed
created_at: 2017-11-12T21:30:29Z
updated_at: 2017-11-13T21:11:01Z
closed_at: 2017-11-13T21:11:01Z
author_association: OWNER
body: As with #69 these should obey a maximum offset setting, which can be over-ridden.
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/70/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzMyNzg4NDA=
title: Set up some example datasets on a Cloudflare-backed domain
state: closed
created_at: 2017-11-13T00:06:30Z
updated_at: 2017-11-13T02:09:34Z
closed_at: 2017-11-13T02:09:34Z
author_association: OWNER
type: issue
state_reason: completed
body:
  To better demonstrate the caching and HTTP/2 features, I'd like to go live with some demos that are hosted behind Cloudflare.

  - [x] Redirect https://datasettes.com/ and https://www.datasettes.com/ to https://github.com/simonw/datasette
  - [x] Have `now domain add -e datasettes.com` run without errors (hopefully just a matter of waiting for the DNS to update)
  - [x] Alias an example dataset hosted on Now on a datasettes.com subdomain
  - [x] Confirm that HTTP caching and HTTP/2 redirect pushing works as expected - this may require another page rule

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/71/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzMyODMxNjY=
title: publish command should take an optional --name argument
state: closed
created_at: 2017-11-13T00:59:35Z
updated_at: 2017-11-13T02:12:27Z
closed_at: 2017-11-13T02:12:27Z
author_association: OWNER
type: issue
state_reason: completed
body:
  To set the directory name so that now will inherit it as the name of the app.

  Defaults to datasette 

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/72/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzMyOTYxNzg=
title: _nocache=1 query string option for use with sort-by-random
state: closed
created_at: 2017-11-13T02:57:10Z
updated_at: 2018-05-28T17:25:15Z
closed_at: 2018-05-28T17:25:15Z
author_association: OWNER
type: issue
state_reason: completed
body:
  The one place where we wouldn’t want cdching is if we have something which uses sort by random to return random items. We can offer a _nocache=1 querystring argument to support this.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/73/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzMyOTY2ODQ=
title: Send a 302 redirect to the new hash for hits to old hashes
state: closed
created_at: 2017-11-13T03:00:59Z
updated_at: 2017-11-13T18:49:59Z
closed_at: 2017-11-13T18:49:59Z
author_association: OWNER
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/74/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzM1MDkxNTk=
title: Add --cors argument to serve
state: closed
created_at: 2017-11-13T17:16:19Z
updated_at: 2017-11-13T18:17:52Z
closed_at: 2017-11-13T18:17:52Z
author_association: OWNER
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/75/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzM1MTA3ODE=
title: publish should have required argument specifying publisher
state: closed
created_at: 2017-11-13T17:21:26Z
updated_at: 2017-11-13T18:41:01Z
closed_at: 2017-11-13T18:41:01Z
author_association: OWNER
body: Initially the only argument will be “now” - but “hyper” can be added in the future
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/76/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzM1Mzc5NDA=
title: Add Travis CI badge to README
state: closed
created_at: 2017-11-13T18:52:25Z
updated_at: 2017-11-13T21:24:15Z
closed_at: 2017-11-13T21:24:15Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Also fix this newline issue:

  <img width="647" alt="simonw_datasette__instant_json_api_for_your_sqlite_database" src="https://user-images.githubusercontent.com/9599/32743234-ae81b224-c860-11e7-98a9-980b7b448ffc.png">

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/77/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzM1NTQ5NDk=
title: Rename after to next and provide a next_url
state: closed
created_at: 2017-11-13T19:48:31Z
updated_at: 2017-11-13T20:35:03Z
closed_at: 2017-11-13T20:35:03Z
author_association: OWNER
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/78/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzM1NjkwNjg=
title: Add more detailed API documentation to the README
state: closed
created_at: 2017-11-13T20:36:21Z
updated_at: 2018-05-28T17:24:48Z
closed_at: 2018-05-28T17:24:48Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Need to document:

  - [ ] The ?column__gt=4 style filter arguments for tables
  - [ ] The ?sql= API, and how named parameters work
  - [ ] How API pagination works
  - [ ] How redirects and cache headers work

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/79/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzM1Njk0Nzc=
title: Deploy final versions of fivethirtyeight and parlgov datasets (with view pagination)
state: closed
created_at: 2017-11-13T20:37:46Z
updated_at: 2017-11-13T22:09:46Z
closed_at: 2017-11-13T22:09:46Z
author_association: OWNER
body: Final versions should be deployed using the first released version of datasette.
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/80/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MTUyMzYwNzQw
title: :fire: Removes DS_Store
state: closed
created_at: 2017-11-13T22:07:52Z
updated_at: 2017-11-14T02:24:54Z
closed_at: 2017-11-13T22:16:55Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/81
type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/81/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzM1OTYxNTk=
title: Post a blog entry announcing it to the world
state: closed
created_at: 2017-11-13T22:10:35Z
updated_at: 2017-11-14T01:46:10Z
closed_at: 2017-11-14T01:46:10Z
author_association: OWNER
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/82/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzM2MjY4MTU=
title: Individual row view is broken
state: closed
created_at: 2017-11-14T00:29:11Z
updated_at: 2017-11-14T00:45:34Z
closed_at: 2017-11-14T00:45:34Z
author_association: OWNER
type: issue
state_reason: completed
body:
  https://parlgov.datasettes.com/parlgov-25f9855/viewcalc_parliament_composition/18

  <img width="822" alt="cursor_and_localhost_8002_parlgov-25f9855_viewcalc_parliament_composition_18" src="https://user-images.githubusercontent.com/9599/32756593-c439c71c-c88f-11e7-9243-b6e1b778c8fa.png">

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/83/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzM2NjA0MjU=
title: datasette package --metadata does not work with a relative path
state: closed
created_at: 2017-11-14T04:00:50Z
updated_at: 2017-11-15T05:18:35Z
closed_at: 2017-11-15T05:18:35Z
author_association: OWNER
type: issue
state_reason: completed
body:
      $ datasette package ~/parlgov-db/parlgov.db --metadata=~/parlgov-db/parlgov.json
      Usage: datasette package [OPTIONS] FILES...

      Error: Invalid value for "-m" / "--metadata": Could not open file: ~/parlgov-db/parlgov.json: No such file or directory
      
      simonw-07542:~ simonw$ cd ~/parlgov-db/
      simonw-07542:parlgov-db simonw$ datasette package ~/parlgov-db/parlgov.db --metadata=parlgov.json
      Sending build context to Docker daemon   4.46MB
      Step 1/7 : FROM python:3

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/84/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzM2Nzg2NzM=
title: Detect foreign keys and use them to link HTML pages together
state: closed
created_at: 2017-11-14T06:12:05Z
updated_at: 2017-11-19T06:08:19Z
closed_at: 2017-11-19T06:08:19Z
author_association: OWNER
type: issue
state_reason: completed
body:
  https://stackoverflow.com/a/44430157/6083 documents the PRAGMA needed to extract foreign key references for a table.

  At a minimum we can link column values known to be foreign keys to the corresponding row page.
  We could try to summarize the linked row in some way too - somehow extracting a sensible link title, maybe based on additional configuration in the metadata.json file.

  Still todo:

  - [x] Fix it to csvs-to-sqlite refactoring command correctly creates primary key on generated tables
  - [x] Ship new csvs-to-sqlite with refactoring command
  - [x] Refactor column logic to be more predictable in our templates (the rowid special case)
  - [x] Mechanism by which table metadata can specify the "label" column for a table
  - [x] Automatically set the label column as the first column that isn't a primary key (falling back on primary key)
  - [x] Code which runs a "select id, label from table where id in (...)" query as part of the tableview and populates a lookup dictionary
  - [x] Modify templates to use values from that lookup dictionary

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/85/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzM3MDM4Mjk=
title: Filter UI on table page
state: closed
created_at: 2017-11-14T08:22:43Z
updated_at: 2017-11-23T20:34:32Z
closed_at: 2017-11-23T20:34:32Z
author_association: OWNER
type: issue
state_reason: completed
body:
  A UI for building up simple table queries by adding additional filter rules that get executed as query parameters in the URL.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/86/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzM3MDkxOTQ=
title: Configure Travis to release new tags to PyPI
state: closed
created_at: 2017-11-14T08:44:08Z
updated_at: 2018-07-10T17:49:13Z
closed_at: 2018-07-10T17:49:12Z
author_association: OWNER
body: https://docs.travis-ci.com/user/deployment/pypi/
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/87/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzM3NzUyMTI=
title: Add NHS England Hospitals example to wiki
state: closed
created_at: 2017-11-14T12:29:10Z
updated_at: 2021-03-22T23:46:36Z
closed_at: 2017-11-14T22:54:06Z
author_association: CONTRIBUTOR
type: issue
state_reason: completed
body:
  https://nhs-england-hospitals.now.sh

  and an associated map visualisation:

  http://run.plnkr.co/preview/cj9zlf1qc0003414y90ajkwpk/

  Datasette is wonderful!

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/88/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MTUyNTIyNzYy
title: SQL syntax highlighting with CodeMirror
state: closed
created_at: 2017-11-14T14:43:33Z
updated_at: 2017-11-15T02:03:01Z
closed_at: 2017-11-15T02:03:01Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/89
type: pull
body:
  Addresses #13 

  Future enhancements could include autocompletion of table and column names, e.g. with

  ```javascript
  extraKeys: {"Ctrl-Space": "autocomplete"},
  hintOptions: {tables: {
        users: ["name", "score", "birthDate"],
        countries: ["name", "population", "size"]
      }}
  ```

  (see https://codemirror.net/doc/manual.html#addon_sql-hint and source at http://codemirror.net/mode/sql/)

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/89/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzM4NDYxMjM=
title: datasette publish heroku
state: closed
created_at: 2017-11-14T16:01:39Z
updated_at: 2017-12-10T03:06:34Z
closed_at: 2017-12-10T03:05:48Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Heroku has Docker container support so this should not be too hard:

  https://devcenter.heroku.com/articles/container-registry-and-runtime

  See also #59 

  This should work exactly like the existing “datasette publish now....” command except it would be “datasette publish heroku...”

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/90/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzM4Nzg4NzM=
title: Option to serve databases from a different prefix, serve regular content elsewhere
state: closed
created_at: 2017-11-14T17:32:46Z
updated_at: 2017-12-10T03:07:58Z
closed_at: 2017-12-10T03:07:53Z
author_association: OWNER
type: issue
state_reason: completed
body:
  It would be useful if the databases themselves could be served from a prefix e.g.

      datasette serve mydb.db --path-prefix=db

  Now my database is at `http://localhost:8001/db/mydb-23423`

  This would free up the rest of the URL namespace for other things. Maybe we could have an option to serve static content from a known folder e.g.

      datasette serve mydb.db --path-prefix=db --root-content=~/my-project/static

  Now a hit to `http://localhost:8001/news/` serves content from `~/my-project/static/news/index.html`

  This would make it trivial to package up entire HTML/CSS/JS apps with one or more underlying SQLite databases. Running without `--cors` would be fine here because any JS apps would be hosted on the same origin.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/91/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzM4OTUzNDQ=
title: Add --license --license_url --source --source_url --title arguments to datasette publish
state: closed
created_at: 2017-11-14T18:27:07Z
updated_at: 2017-11-15T05:04:41Z
closed_at: 2017-11-15T05:04:41Z
author_association: OWNER
type: issue
state_reason: completed
body:
  I keep on using the `echo '{"source": "..."}' | datasette publish now --metadata=-` pattern, which suggests it makes sense for us to support these as optional arguments.

  https://gist.github.com/simonw/9f8bf23b37a42d7628c4dcc4bba10253

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/92/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzM5NDQ5NTI=
title: Package as standalone binary
state: closed
created_at: 2017-11-14T21:14:07Z
updated_at: 2021-11-21T07:00:23Z
closed_at: 2021-11-21T07:00:23Z
author_association: NONE
type: issue
state_reason: completed
body:
  hint: more than the docker image a standalone and multiplatform binary (containing the app and the database) could be simpler to distribute.

  i would like to investigate the possibility to package everything with [pyinstaller](http://www.pyinstaller.org/) adding the database as a [data file](https://pythonhosted.org/PyInstaller/spec-files.html#adding-data-files)

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/93/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MTUyNjMxNTcw
title: Initial add simple prod ready Dockerfile refs #57
state: closed
created_at: 2017-11-14T22:09:09Z
updated_at: 2017-11-15T03:08:04Z
closed_at: 2017-11-15T03:08:04Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/94
type: pull
body:
  Multi-stage build based off official python:3.6-slim

  Example usage:
  ```
  docker run --rm -t -i -p 9000:8001 -v $(pwd)/db:/db datasette datasette serve /db/chinook.db
  ```

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/94/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzM5OTg1MTM=
title: Allow shorter time limits to be set using a ?_sql_time_limit_ms =20 query string limit
state: closed
created_at: 2017-11-15T01:02:16Z
updated_at: 2017-11-15T02:56:13Z
closed_at: 2017-11-15T02:56:13Z
author_association: OWNER
body: This cannot be greater than the configured time limit.
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/95/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzQwMDE0NTM=
title: UI for editing named parameters
state: closed
created_at: 2017-11-15T01:19:21Z
updated_at: 2017-11-16T01:45:51Z
closed_at: 2017-11-16T01:33:38Z
author_association: OWNER
type: issue
state_reason: completed
body:
  On any page displaying a custom query that includes named parameters, we should show HTML form fields for editing those parameters.

  Eg the breed parameter on https://australian-dogs.now.sh/australian-dogs-3ba9628?sql=select+name%2C+count%28*%29+as+n+from+%28%0D%0A%0D%0Aselect+upper%28%22Animal+name%22%29+as+name+from+%5BAdelaide-City-Council-dog-registrations-2013%5D+where+Breed+like+%3Abreed%0D%0A%0D%0Aunion+all%0D%0A%0D%0Aselect+upper%28Animal_Name%29+as+name+from+%5BAdelaide-City-Council-dog-registrations-2014%5D+where+Breed_Description+like+%3Abreed%0D%0A%0D%0Aunion+all+%0D%0A%0D%0Aselect+upper%28Animal_Name%29+as+name+from+%5BAdelaide-City-Council-dog-registrations-2015%5D+where+Breed_Description+like+%3Abreed%0D%0A%0D%0Aunion+all%0D%0A%0D%0Aselect+upper%28%22AnimalName%22%29+as+name+from+%5BCity-of-Port-Adelaide-Enfield-Dog_Registrations_2016%5D+where+AnimalBreed+like+%3Abreed%0D%0A%0D%0Aunion+all%0D%0A%0D%0Aselect+upper%28%22Animal+Name%22%29+as+name+from+%5BMitcham-dog-registrations-2015%5D+where+Breed+like+%3Abreed%0D%0A%0D%0Aunion+all%0D%0A%0D%0Aselect+upper%28%22DOG_NAME%22%29+as+name+from+%5Bburnside-dog-registrations-2015%5D+where+DOG_BREED+like+%3Abreed%0D%0A%0D%0Aunion+all+%0D%0A%0D%0Aselect+upper%28%22Animal_Name%22%29+as+name+from+%5Bcity-of-playford-2015-dog-registration%5D+where+Breed_Description+like+%3Abreed%0D%0A%0D%0Aunion+all%0D%0A%0D%0Aselect+upper%28%22Animal+Name%22%29+as+name+from+%5Bcity-of-prospect-dog-registration-details-2016%5D+where%22Breed+Description%22+like+%3Abreed%0D%0A%0D%0A%29+group+by+name+order+by+n+desc%3B&breed=pug

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/96/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzQwMjI5NTA=
title: Link to JSON for the list of tables
state: closed
created_at: 2017-11-15T03:29:05Z
updated_at: 2018-05-29T18:51:35Z
closed_at: 2018-05-28T20:57:21Z
author_association: OWNER
body: https://twitter.com/yschimke/status/930606210855854080
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/97/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzQwMjM0MTc=
title: Default to 127.0.0.1 not 0.0.0.0
state: closed
created_at: 2017-11-15T03:31:55Z
updated_at: 2017-11-15T05:08:54Z
closed_at: 2017-11-15T05:08:54Z
author_association: OWNER
body: https://twitter.com/yschimke/status/930606210855854080
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/98/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzQwMjM2MjU=
title: Start a change log
state: closed
created_at: 2017-11-15T03:33:21Z
updated_at: 2017-11-16T15:12:46Z
closed_at: 2017-11-16T15:12:45Z
author_association: OWNER
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/99/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzQxNjA3MjM=
title: TemplateAssertionError: no filter named 'tojson'
state: closed
created_at: 2017-11-15T13:43:41Z
updated_at: 2017-11-16T09:25:10Z
closed_at: 2017-11-16T00:14:13Z
author_association: NONE
type: issue
state_reason: completed
body:
  A 500 error is raised upon clicking on the name of a table on the homepage, say _http://0.0.0.0:8001/_ to _http://0.0.0.0:8001/test_check-c1f4771/users_ The API part seems to function as intended, though...

  ```
  2017-11-15 14:33:57 - (sanic)[ERROR]: Traceback (most recent call last):
    File "/usr/local/lib/python3.5/dist-packages/sanic/app.py", line 503, in handle_request
      response = await response
    File "/usr/local/lib/python3.5/dist-packages/datasette/app.py", line 155, in get
      return await self.view_get(request, name, hash, **kwargs)
    File "/usr/local/lib/python3.5/dist-packages/datasette/app.py", line 219, in view_get
      **context,
    File "/usr/local/lib/python3.5/dist-packages/sanic_jinja2/__init__.py", line 84, in render
      return html(self.render_string(template, request, **context))
    File "/usr/local/lib/python3.5/dist-packages/sanic_jinja2/__init__.py", line 81, in render_string
      return self.env.get_template(template).render(**context)
    File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 812, in get_template
      return self._load_template(name, self.make_globals(globals))
    File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 786, in _load_template
      template = self.loader.load(self, name, globals)
    File "/usr/lib/python3/dist-packages/jinja2/loaders.py", line 125, in load
      code = environment.compile(source, name, filename)
    File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 565, in compile
      self.handle_exception(exc_info, source_hint=source_hint)
    File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 754, in handle_exception
      reraise(exc_type, exc_value, tb)
    File "/usr/lib/python3/dist-packages/jinja2/_compat.py", line 37, in reraise
      raise value.with_traceback(tb)
    File "/usr/local/lib/python3.5/dist-packages/datasette/templates/table.html", line 29, in template
      <pre>params = {{ query.params|tojson(4) }}</pre>
    File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 515, in _generate
      return generate(source, self, name, filename, defer_init=defer_init)
    File "/usr/lib/python3/dist-packages/jinja2/compiler.py", line 62, in generate
      generator.visit(node)
    File "/usr/lib/python3/dist-packages/jinja2/visitor.py", line 38, in visit
      return f(node, *args, **kwargs)
    File "/usr/lib/python3/dist-packages/jinja2/compiler.py", line 849, in visit_Template
      self.blockvisit(block.body, block_frame)
    File "/usr/lib/python3/dist-packages/jinja2/compiler.py", line 492, in blockvisit
      self.visit(node, frame)
    File "/usr/lib/python3/dist-packages/jinja2/visitor.py", line 38, in visit
      return f(node, *args, **kwargs)
    File "/usr/lib/python3/dist-packages/jinja2/compiler.py", line 1172, in visit_If
      self.blockvisit(node.body, if_frame)
    File "/usr/lib/python3/dist-packages/jinja2/compiler.py", line 492, in blockvisit
      self.visit(node, frame)
    File "/usr/lib/python3/dist-packages/jinja2/visitor.py", line 38, in visit
      return f(node, *args, **kwargs)
    File "/usr/lib/python3/dist-packages/jinja2/compiler.py", line 1353, in visit_Output
      self.visit(argument, frame)
    File "/usr/lib/python3/dist-packages/jinja2/visitor.py", line 38, in visit
      return f(node, *args, **kwargs)
    File "/usr/lib/python3/dist-packages/jinja2/compiler.py", line 1565, in visit_Filter
      self.fail('no filter named %r' % node.name, node.lineno)
    File "/usr/lib/python3/dist-packages/jinja2/compiler.py", line 427, in fail
      raise TemplateAssertionError(msg, lineno, self.name, self.filename)
  jinja2.exceptions.TemplateAssertionError: no filter named 'tojson'

  2017-11-15 14:33:57 - (network)[INFO][127.0.0.1:41316]: GET http://0.0.0.0:8001/test_check-c1f4771/users  500 144
  2017-11-15 14:33:57 - (network)[INFO][127.0.0.1:41316]: GET http://0.0.0.0:8001/favicon.ico  200 0
  ```

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/100/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzQxNjE5NjQ=
title: TemplateAssertionError: no filter named 'tojson'
state: closed
created_at: 2017-11-15T13:47:32Z
updated_at: 2017-11-15T13:48:55Z
closed_at: 2017-11-15T13:48:55Z
author_association: NONE
type: issue
state_reason: completed
body:
  I get an exception clicking on the table link: 

  ```
  2017-11-15 08:40:10 - (sanic)[ERROR]: Traceback (most recent call last):
    File "/Users/e/anaconda3-4.2.0/lib/python3.5/site-packages/sanic/app.py", line 503, in handle_request
      response = await response
    File "/Users/e/anaconda3-4.2.0/lib/python3.5/site-packages/datasette/app.py", line 155, in get
      return await self.view_get(request, name, hash, **kwargs)
    File "/Users/e/anaconda3-4.2.0/lib/python3.5/site-packages/datasette/app.py", line 219, in view_get
      **context,
    File "/Users/e/anaconda3-4.2.0/lib/python3.5/site-packages/sanic_jinja2/__init__.py", line 84, in render
      return html(self.render_string(template, request, **context))
    File "/Users/e/anaconda3-4.2.0/lib/python3.5/site-packages/sanic_jinja2/__init__.py", line 81, in render_string
      return self.env.get_template(template).render(**context)
    File "/Users/e/anaconda3-4.2.0/lib/python3.5/site-packages/jinja2/environment.py", line 812, in get_template
      return self._load_template(name, self.make_globals(globals))
    File "/Users/e/anaconda3-4.2.0/lib/python3.5/site-packages/jinja2/environment.py", line 786, in _load_template
      template = self.loader.load(self, name, globals)
    File "/Users/e/anaconda3-4.2.0/lib/python3.5/site-packages/jinja2/loaders.py", line 125, in load
      code = environment.compile(source, name, filename)
    File "/Users/e/anaconda3-4.2.0/lib/python3.5/site-packages/jinja2/environment.py", line 565, in compile
      self.handle_exception(exc_info, source_hint=source_hint)
    File "/Users/e/anaconda3-4.2.0/lib/python3.5/site-packages/jinja2/environment.py", line 754, in handle_exception
      reraise(exc_type, exc_value, tb)
    File "/Users/e/anaconda3-4.2.0/lib/python3.5/site-packages/jinja2/_compat.py", line 37, in reraise
      raise value.with_traceback(tb)
    File "/Users/e/anaconda3-4.2.0/lib/python3.5/site-packages/datasette/templates/table.html", line 29, in template
      <pre>params = {{ query.params|tojson(4) }}</pre>
    File "/Users/e/anaconda3-4.2.0/lib/python3.5/site-packages/jinja2/environment.py", line 515, in _generate
      return generate(source, self, name, filename, defer_init=defer_init)
    File "/Users/e/anaconda3-4.2.0/lib/python3.5/site-packages/jinja2/compiler.py", line 62, in generate
      generator.visit(node)
    File "/Users/e/anaconda3-4.2.0/lib/python3.5/site-packages/jinja2/visitor.py", line 38, in visit
      return f(node, *args, **kwargs)
    File "/Users/e/anaconda3-4.2.0/lib/python3.5/site-packages/jinja2/compiler.py", line 849, in visit_Template
      self.blockvisit(block.body, block_frame)
    File "/Users/e/anaconda3-4.2.0/lib/python3.5/site-packages/jinja2/compiler.py", line 492, in blockvisit
      self.visit(node, frame)
    File "/Users/e/anaconda3-4.2.0/lib/python3.5/site-packages/jinja2/visitor.py", line 38, in visit
      return f(node, *args, **kwargs)
    File "/Users/e/anaconda3-4.2.0/lib/python3.5/site-packages/jinja2/compiler.py", line 1172, in visit_If
      self.blockvisit(node.body, if_frame)
    File "/Users/e/anaconda3-4.2.0/lib/python3.5/site-packages/jinja2/compiler.py", line 492, in blockvisit
      self.visit(node, frame)
    File "/Users/e/anaconda3-4.2.0/lib/python3.5/site-packages/jinja2/visitor.py", line 38, in visit
      return f(node, *args, **kwargs)
    File "/Users/e/anaconda3-4.2.0/lib/python3.5/site-packages/jinja2/compiler.py", line 1353, in visit_Output
      self.visit(argument, frame)
    File "/Users/e/anaconda3-4.2.0/lib/python3.5/site-packages/jinja2/visitor.py", line 38, in visit
      return f(node, *args, **kwargs)
    File "/Users/e/anaconda3-4.2.0/lib/python3.5/site-packages/jinja2/compiler.py", line 1565, in visit_Filter
      self.fail('no filter named %r' % node.name, node.lineno)
    File "/Users/e/anaconda3-4.2.0/lib/python3.5/site-packages/jinja2/compiler.py", line 427, in fail
      raise TemplateAssertionError(msg, lineno, self.name, self.filename)
  jinja2.exceptions.TemplateAssertionError: no filter named 'tojson'
  ```

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/101/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzQyNjQxNzU=
title: datasette publish elasticbeanstalk
state: closed
created_at: 2017-11-15T18:48:31Z
updated_at: 2021-01-04T20:13:20Z
closed_at: 2021-01-04T20:13:19Z
author_association: OWNER
type: issue
state_reason: completed
body:
  It looks like Elastic Beanstalk is the most convenient way to deploy a docker container to AWS without first deploying a cluster.

  https://aws.amazon.com/blogs/devops/dockerizing-a-python-web-app/ looks helpful. We would need to automate the deployment with Boto: http://boto3.readthedocs.io/en/latest/reference/services/elasticbeanstalk.html

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/102/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzQyNjU4Nzg=
title: datasette publish appengine
state: closed
created_at: 2017-11-15T18:54:18Z
updated_at: 2021-01-04T20:05:14Z
closed_at: 2021-01-04T20:05:14Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Similar approach to Heroku, discussed in #90 

  Looks like this could be pretty easy:  https://cloud.google.com/appengine/docs/flexible/python/quickstart

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/103/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MTUyODcwMDMw
title: [WIP] Add publish to heroku support
state: closed
created_at: 2017-11-15T19:56:22Z
updated_at: 2017-11-21T20:55:05Z
closed_at: 2017-11-21T20:55:05Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/104
body: Refs #90
type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/104/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzQzMTQ5NDA=
title: Consider data-package as a format for metadata
state: closed
created_at: 2017-11-15T21:43:34Z
updated_at: 2017-11-20T19:50:53Z
closed_at: 2017-11-20T19:50:53Z
author_association: OWNER
body: http://frictionlessdata.io/specs/data-package/
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/105/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzQzMTUxOTM=
title: Document how pagination works
state: closed
created_at: 2017-11-15T21:44:32Z
updated_at: 2019-06-24T06:42:33Z
closed_at: 2019-06-24T06:42:33Z
author_association: OWNER
body: I made a start at that in this comment: https://news.ycombinator.com/item?id=15691926
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/106/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MTUyOTE0NDgw
title: add support for ?field__isnull=1
state: closed
created_at: 2017-11-15T23:36:36Z
updated_at: 2017-11-17T15:12:29Z
closed_at: 2017-11-17T13:29:22Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/107
body: Is this what you had in mind for [this issue](https://github.com/simonw/datasette/issues/64)?
type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/107/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzQzNzQzMTc=
title: Include version in python code, output in template
state: closed
created_at: 2017-11-16T02:32:40Z
updated_at: 2017-11-16T15:30:04Z
closed_at: 2017-11-16T15:30:04Z
author_association: OWNER
type: issue
state_reason: completed
body:
  It would be useful if I could tell which version of datasette was running on a site. Embed version number and include it in maybe a tooltip on the “powered by datasette” link

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/108/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzQzNzgzMDE=
title: Set up readthedocs
state: closed
created_at: 2017-11-16T02:58:01Z
updated_at: 2017-11-16T16:53:26Z
closed_at: 2017-11-16T16:13:56Z
author_association: OWNER
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/109/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzQ1NzgxNDI=
title: Add --load-extension option to datasette for loading extra SQLite extensions
state: closed
created_at: 2017-11-16T16:26:19Z
updated_at: 2017-11-16T18:38:30Z
closed_at: 2017-11-16T16:58:50Z
author_association: OWNER
type: issue
state_reason: completed
body:
  This would allow users with extra SQLite extensions installed (like spatialite) to load them at runtime.

  Inspired by this comment: https://github.com/simonw/datasette/issues/46#issuecomment-344810525

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/110/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzQ2MTU0NTI=
title: Add “updated” to metadata
state: open
created_at: 2017-11-16T18:22:20Z
updated_at: 2021-09-21T22:48:27Z
author_association: OWNER
type: issue
body:
  To give an indication as to when the data was last updated.

  This should be a field in the metadata that is then shown on the index page and in the footer, if it is set.

  Also support setting it using an option to “datasette publish” and “datasette package” - which can either be a string or can be the magic string “today” to set it to today’s date:

      datasette publish file.db --updated=today

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/111/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzQ2MTcyNDA=
title: Allow --load-extension to be set via environment variables
state: closed
created_at: 2017-11-16T18:28:31Z
updated_at: 2017-11-17T14:19:23Z
closed_at: 2017-11-17T14:17:27Z
author_association: OWNER
type: issue
state_reason: completed
body:
  This will make it easier to package up datasette in a Docker container with a bunch of pre-compiled extensions without the user having to remember to include all of the options every time.

  Click has a mechanism for this: http://click.pocoo.org/5/options/#multiple-values-from-environment-values

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/112/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzQ2NjIzNzg=
title: Fix the &nbsp; bug on the database custom SQL query view
state: closed
created_at: 2017-11-16T21:01:26Z
updated_at: 2017-11-17T15:40:52Z
closed_at: 2017-11-17T15:40:52Z
author_association: OWNER
type: issue
state_reason: completed
body:
  https://sf-film-locations.now.sh/sf-film-locations-57704b7?sql=select+*+from+Film_Locations_in_San_Francisco

  <img width="852" alt="sf-film-locations" src="https://user-images.githubusercontent.com/9599/32915540-19bad984-cace-11e7-9ed3-bd7294891dc0.png">

  This is the bug I fixed in 01e0c3fa18cd0dd7970e208790ffd683a420c924 - but I only fixed it in one place.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/113/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MTUzMjAxOTQ1
title: Add spatialite, switch to debian and local build
state: closed
created_at: 2017-11-17T02:37:09Z
updated_at: 2017-11-17T03:50:52Z
closed_at: 2017-11-17T03:50:52Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/114
type: pull
body:
  Improves the Dockerfile to support spatial datasets, work with the local datasette code (Friendly with git tags and Dockerhub) and moves to slim debian, a small image easy to extend via apt packages for sqlite.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/114/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MTUzMzA2ODgy
title: Add keyboard shortcut to execute SQL query
state: closed
created_at: 2017-11-17T14:13:33Z
updated_at: 2017-11-17T15:16:34Z
closed_at: 2017-11-17T14:22:56Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/115
type: pull
body:
  Very cool tool, thanks a lot!

  This PR adds a `Shift-Enter` short cut to execute the SQL query. I used CodeMirrors keyboard handling.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/115/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzQ4ODQyMDk=
title: Add documentation section about SQLite extensions
state: closed
created_at: 2017-11-17T14:36:30Z
updated_at: 2018-05-28T17:23:42Z
closed_at: 2018-05-28T17:23:41Z
author_association: OWNER
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/116/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MTUzMzI0MzAx
title: Don't prevent tabbing to `Run SQL` button
state: closed
created_at: 2017-11-17T15:27:50Z
updated_at: 2017-11-19T20:30:24Z
closed_at: 2017-11-18T00:53:43Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/117
body:
  Mentioned in #115

  Here you go!

type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/117/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MTUzNDMyMDQ1
title: Foreign key information on row and table pages
state: closed
created_at: 2017-11-18T03:13:27Z
updated_at: 2017-11-18T03:15:57Z
closed_at: 2017-11-18T03:15:50Z
author_association: OWNER
pull_request: simonw/datasette/pulls/118
type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/118/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzUwODIxNTg=
title: Build an "export this data to google sheets" plugin
state: closed
created_at: 2017-11-18T14:14:51Z
updated_at: 2020-06-04T18:46:40Z
closed_at: 2020-06-04T18:46:39Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Inspired by https://github.com/kren1/tosheets

  It should be a plug-in because I'd like to keep all interactions with proprietary / non-open-source software encapsulated in plugins rather than shipped as part of core.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/119/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzUwODczOTc=
title: Plugin that adds an authentication layer of some sort
state: closed
created_at: 2017-11-18T15:39:13Z
updated_at: 2020-03-16T18:48:06Z
closed_at: 2020-03-16T18:48:06Z
author_association: OWNER
body:
  Would allow people who want to host private data to do so.
  .sh

type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/120/reactions", "total_count": 7, "+1": 5, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 2, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzUwODk1MzU=
title: ?_json=foo&_json=bar query string argument
state: closed
created_at: 2017-11-18T16:09:55Z
updated_at: 2018-05-31T13:48:12Z
closed_at: 2018-05-28T18:11:51Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Causes the specified columns in the output to be treated as JSON, and returned deserialized in the .json or .jsono response.

  This will be particularly powerful when combined with https://sqlite.org/json1.html

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/121/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzUwOTI0NTM=
title: Redesign JSON output, ditch jsono, offer variants controlled by parameter instead
state: closed
created_at: 2017-11-18T16:52:28Z
updated_at: 2018-04-08T14:54:09Z
closed_at: 2018-04-08T14:54:09Z
author_association: OWNER
type: issue
state_reason: completed
body:
  I want to support three variants for the rows output:

  * a list of lists, with a columns key saying what they are
  * a list of dictionaries
  * a single dictionary where the keys are the primary keys of the rows and the values are the row dictionaries themselves

  I also want to make the various bits of metadata opt-in - so you don't get the SQL statement unless you ask for it.

  These output options should be controlled by query string arguments.

  I will set the .jsono URL to redirect to .json with the corresponding options. 

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/122/reactions", "total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 1, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzUxMjU1NjE=
title: Datasette serve should accept paths/URLs to CSVs and other file formats
state: open
created_at: 2017-11-19T02:05:48Z
updated_at: 2021-07-19T00:04:32Z
author_association: OWNER
type: issue
body:
  This would remove the csvs-to-sqlite step which I end up using for almost everything.

  I'm hesitant to introduce pandas as a required dependency though since it require compiling numpy. Could build it so this option is only available if you have pandas installed.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/123/reactions", "total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 1, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzUxMjU4MDU=
title: Option to open readonly but not immutable
state: closed
created_at: 2017-11-19T02:11:03Z
updated_at: 2019-06-24T06:43:46Z
closed_at: 2019-06-24T06:43:46Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Immutable assumes no other process can modify the file. An option to open reqdonly instead would enable other processes to update the file in place.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/124/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzUxMzUzOTM=
title: Plot rows on a map with Leaflet and Leaflet.markercluster
state: closed
created_at: 2017-11-19T06:05:05Z
updated_at: 2018-04-26T15:14:31Z
closed_at: 2018-04-26T15:14:31Z
author_association: OWNER
type: issue
state_reason: completed
body:
  https://github.com/Leaflet/Leaflet.markercluster would allow us to paginate-load in an enormous set of rows with latitude/longitude points, e.g. https://australian-dunnies.now.sh/

  Here's a demo of it loading 50,000 markers: https://leaflet.github.io/Leaflet.markercluster/example/marker-clustering-realworld.50000.html - and it looks like it's easy to support progress bars for if we were iteratively loading 1,000 markers at a time using datasette pagination.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/125/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzUxMzU1MzU=
title: Blog entry announcing foreign key support
state: closed
created_at: 2017-11-19T06:09:06Z
updated_at: 2017-11-30T16:49:24Z
closed_at: 2017-11-30T16:49:24Z
author_association: OWNER
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/126/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzUxMzU3MTk=
title: Filtered tables should show count of all matching rows, if fast enough
state: closed
created_at: 2017-11-19T06:13:29Z
updated_at: 2017-11-24T22:02:01Z
closed_at: 2017-11-24T22:02:01Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Relates to #86. If you are viewing a filtered page e.g. https://fivethirtyeight.datasettes.com/fivethirtyeight-2628db9/bob-ross%2Felements-by-episode?CLOUDS=1 we should show the count of matching rows.

  Since this could be an expensive operation, we will run it with a strict time limit (maybe 50ms). If the time limit is exceeded we will display "many" instead, perhaps? Maybe even link to a count(*) query that would get the full 1000ms time limit which the user can click on if they like (that could even Ajax-in the result).

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/127/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzUxNTk3MTA=
title: Every visualization should have an "embed" button
state: open
created_at: 2017-11-19T13:38:13Z
updated_at: 2019-05-13T18:33:51Z
author_association: OWNER
type: issue
body:
  At least for the first round of visualizations, any time you construct one using the UI the result should include an "embed this" button that returns source code to copy and paste

  These examples should use unpkg.com (or similarl) urls with SRI hashes, eg https://www.srihash.org - and should load data from the datasette JSON API.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/128/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzUxNjQ1NTg=
title: Hide FTS-created tables by default on the database index page
state: closed
created_at: 2017-11-19T14:50:42Z
updated_at: 2017-11-22T20:22:02Z
closed_at: 2017-11-22T20:19:04Z
author_association: OWNER
type: issue
state_reason: completed
body:
  SQLite databases that use FTS include a number of automatically generated tables, e.g.:

  https://sf-trees-search.now.sh/sf-trees-search-a899b92

  <img width="730" alt="sf-trees-search_and_sf-trees-search" src="https://user-images.githubusercontent.com/9599/32991960-bf05abee-ccf9-11e7-8bd1-54dcde4ca491.png">

  Of these, only the `Street_Tree_List` table is actually relevant to the user.

  We can detect which tables are FTS tables by first finding the virtual tables:

      sqlite> .headers on
      sqlite> select * from sqlite_master where rootpage = 0;
      type|name|tbl_name|rootpage|sql
      table|Search|Search|0|CREATE VIRTUAL TABLE "Street_Tree_List_fts" USING FTS4 ("qAddress", "qCaretaker", "qSpecies")

  Then parsing the above to figure out which ones are USING FTS? - then assume that any table which starts with that `Street_Tree_List_fts` prefix was created to support search:

      sqlite> select * from sqlite_master where type='table' and tbl_name like 'Street_Tree_List_fts%';
      type|name|tbl_name|rootpage|sql
      table|Search_content|Search_content|10355|CREATE TABLE 'Street_Tree_List_fts_content'(docid INTEGER PRIMARY KEY, 'c0qAddress', 'c1qCaretaker', 'c2qSpecies')
      table|Search_segments|Search_segments|10356|CREATE TABLE 'Street_Tree_List_fts_segments'(blockid INTEGER PRIMARY KEY, block BLOB)
      table|Search_segdir|Search_segdir|10357|CREATE TABLE 'Street_Tree_List_fts_segdir'(level INTEGER,idx INTEGER,start_block INTEGER,leaves_end_block INTEGER,end_block INTEGER,root BLOB,PRIMARY KEY(level, idx))
      table|Search_docsize|Search_docsize|10359|CREATE TABLE 'Street_Tree_List_fts_docsize'(docid INTEGER PRIMARY KEY, size BLOB)
      table|Search_stat|Search_stat|10360|CREATE TABLE 'Street_Tree_List_fts_stat'(id INTEGER PRIMARY KEY, value BLOB)

  We won't hide these completely - instead, we'll default the database index view to not showing them with a message that says "5 hidden tables" and support ?_hidden=1 to display them.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/129/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzUxNjYwNzg=
title: Rename "datasette build" to "datasette inspect"
state: closed
created_at: 2017-11-19T15:08:02Z
updated_at: 2017-12-07T16:57:58Z
closed_at: 2017-12-07T16:57:58Z
author_association: OWNER
type: issue
state_reason: completed
body:
  This command introspects the databases and writes out a JSON summary.

  I think I'd like to use `datasette build` for something more interesting, potentially duplicating functionality from https://github.com/simonw/csvs-to-sqlite

  Since the internal method that does this is called `ds.inspect()` that seems like a reasonable replacement name for the command.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/130/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzUxNjY2Njk=
title: UI support for running FTS searches
state: closed
created_at: 2017-11-19T15:16:20Z
updated_at: 2017-11-19T17:18:05Z
closed_at: 2017-11-19T17:00:12Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Here's an example query that searches all FTS indexed columns in a table: https://sf-trees-search.now.sh/sf-trees-search-a899b92?sql=select+*+from+Street_Tree_List+where+rowid+in+%28select+rowid+from+Street_Tree_List_fts+where+Street_Tree_List_fts+match+%27grove+london+dpw%27%29%0D%0A

  And here's a query that searches a specific column: https://sf-trees-search.now.sh/sf-trees-search-a899b92?sql=select+*+from+Street_Tree_List+where+rowid+in+%28select+rowid+from+Street_Tree_List_fts+where+qSpecies+match+%27london%27%29%0D%0A

  If we detect that a table has FTS enabled (which we can do by looking for it as a content table reference in another FTS table's create definition) we should add a search box to the table page which constructs this query - maybe using `?_search=XXX` in the query string?

  <s>To support search against specified columns, we can do `?_search__ qSpecies=London`.</s> - not necessary, see comment below.

  - [x] Detect if a table has a FTS index defined against it as a content= parameter
  - [x] Decide what to do if there is more than one FTS index (maybe just pick the first one?)
  - [x] Add the `?_search=` query string argument
  - [x] Add the UI

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/131/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzUxNzU5Mjk=
title: Row view is not currently expanding foreign keys
state: closed
created_at: 2017-11-19T17:24:25Z
updated_at: 2017-11-23T21:51:51Z
closed_at: 2017-11-23T21:51:30Z
author_association: OWNER
body: Eg https://sf-trees.now.sh/sf-trees-ebc2ad9/Street_Tree_List/1
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/132/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzUxNzYwMDY=
title: If view is filtered, search should apply within those filtered rows
state: closed
created_at: 2017-11-19T17:25:36Z
updated_at: 2017-11-24T22:30:32Z
closed_at: 2017-11-24T22:30:15Z
author_association: OWNER
body: Eg on https://sf-trees.now.sh/sf-trees-ebc2ad9/Street_Tree_List?qSpecies=1
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/133/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzUxNzYwOTQ=
title: Filtered table view should show a count
state: closed
created_at: 2017-11-19T17:26:53Z
updated_at: 2017-11-19T18:10:49Z
closed_at: 2017-11-19T18:10:49Z
author_association: OWNER
body: Let's do the thing where we attempt to show an accurate count if it can be done in less than 50ms
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/134/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzUxNzk3MjQ=
title: ?_search=x should work if used directly against a FTS virtual table
state: closed
created_at: 2017-11-19T18:17:53Z
updated_at: 2017-12-07T04:54:41Z
closed_at: 2017-12-07T04:54:41Z
author_association: OWNER
body: e.g. https://sf-trees.now.sh/sf-trees-ebc2ad9/Street_Tree_List_fts?_search=grove should work
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/135/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzUyMjg4MzQ=
title: "Reformat SQL" button next to SQL editor textarea
state: closed
created_at: 2017-11-20T03:42:19Z
updated_at: 2019-10-14T03:46:13Z
closed_at: 2019-10-14T03:46:13Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Can use this:

  https://github.com/zeroturnaround/sql-formatter
  https://zeroturnaround.github.io/sql-formatter/

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/136/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzU0MTU3OTk=
title: Ability to combine multiple SQL queries on a single graph
state: open
created_at: 2017-11-20T16:26:57Z
updated_at: 2019-05-13T18:33:51Z
author_association: OWNER
type: issue
body:
  This would make visualizations significantly more powerful. The interesting challenge will be around the URL design. It would be useful to be able to combine either multiple explicit SQL queries or multiple queries based on the filter string parameters passed to one or more table views.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/137/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzU0NzY4Mzk=
title: Per-database and per-table metadata, probably using data-package
state: closed
created_at: 2017-11-20T19:50:10Z
updated_at: 2017-12-10T03:08:36Z
closed_at: 2017-12-10T03:08:26Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Ability to annotate databases and tables with extra metadata describing their purpose, providing source and licensing information and describing individual columns.

  http://frictionlessdata.io/specs/data-package/ looks like a great format for this, see #105 

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/138/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzU0OTM4NTE=
title: Build a visualization plugin for Vega
state: closed
created_at: 2017-11-20T20:47:41Z
updated_at: 2018-07-10T17:48:18Z
closed_at: 2018-07-10T17:48:18Z
author_association: OWNER
type: issue
state_reason: completed
body:
  https://vega.github.io/vega/examples/population-pyramid/ for example looks pretty easy to hook up to Datasette.

  Depends on #14 

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/139/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzU3NTU0NzU=
title: Heatmap visualization plugin
state: open
created_at: 2017-11-21T15:34:23Z
updated_at: 2019-05-13T18:33:51Z
author_association: OWNER
body: Could use https://github.com/scottbedard/svelte-heatmap
type: issue
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/140/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzU4MTQ5NDE=
title: datasette publish can fail if /tmp is on a different device
state: closed
created_at: 2017-11-21T18:28:05Z
updated_at: 2020-04-29T03:27:54Z
closed_at: 2017-12-08T16:06:36Z
author_association: CONTRIBUTOR
type: issue
state_reason: completed
body:
  `datasette publish` uses hard links to avoid copying the db into a tmp directory. This can fail if `/tmp` is on another device, because hardlinks can't cross devices. You'll see something like this:

  ```
  $ datasette publish heroku whatever.db
  ...
  OSError: [Errno 18] Invalid cross-device link: '/mnt/c/Users/jacob/c/datasette/whatever.db' -> '/tmp/tmpvxq2yof6/whatever.db'
  ```
  [In my case this is failing because I'm on a Windows machine, using WSL, so my code's on a different virtual filesystem from the Linux subsystem, Because Reasons.]

  I'm not sure if it's possible to detect this (can you figure out which device `/tmp` is on?), or what the fallback should be (soft link? copy?).

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/141/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzU5MTc3NjA=
title: Show extra instructions with the interrupted
state: closed
created_at: 2017-11-22T01:44:29Z
updated_at: 2018-05-28T21:25:06Z
closed_at: 2018-05-28T21:24:35Z
author_association: OWNER
type: issue
state_reason: completed
body:
  When you are using Datasette locally for ad-hoc analysis it can be frustrating to hit the time limit.

   If you start it with the correct command line arguments you can disable that time limit.  So how about we tell you how to do that anytime you hit the interrupted error provided you are accessing it from localhost.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/142/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzU5MzkxODg=
title: Mechanism for "suggested visualizations"
state: closed
created_at: 2017-11-22T04:10:25Z
updated_at: 2018-07-10T17:48:34Z
closed_at: 2018-07-10T17:48:34Z
author_association: OWNER
type: issue
state_reason: completed
body:
   Each visualization should have a way of deciding if it might be appropriate for the current view of data. 

   We can then offer a "suggested visualizations" prompt which shows previews.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/143/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzYwOTEyNzk=
title: apsw as alternative sqlite3 binding (for full text search)
state: closed
created_at: 2017-11-22T14:40:39Z
updated_at: 2018-05-28T21:29:42Z
closed_at: 2018-05-28T21:29:42Z
author_association: NONE
type: issue
state_reason: completed
body:
  Hey there,

  Have you considered providing apsw support as an alternative to stock python sqlite3?  I use apsw because it keeps up with sqlite3 and is straightforward to bring in extensions like FTS5.  FTS really accelerates the kind of searching often done by web clients.

  I may be able to help (it shouldn't be much code), but there are a couple of stylistic questions that come up when supporting an optional package.  Also, apsw is tricky in that it doesn't have a pypi package (author says limitations in providing options to setup.py). 

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/144/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MTU0MjQ2ODE2
title: Fix pytest version conflict
state: closed
created_at: 2017-11-22T20:15:34Z
updated_at: 2017-11-22T20:17:54Z
closed_at: 2017-11-22T20:17:52Z
author_association: OWNER
pull_request: simonw/datasette/pulls/145
type: pull
body:
  https://travis-ci.org/simonw/datasette/jobs/305929426

      pkg_resources.VersionConflict: (pytest 3.2.1 (/home/travis/virtualenv/python3.5.3/lib/python3.5/site-packages), 
      Requirement.parse('pytest==3.2.3'))

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/145/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzY0NTU3NDg=
title: datasette publish gcloud
state: closed
created_at: 2017-11-23T18:55:03Z
updated_at: 2019-06-24T06:48:20Z
closed_at: 2019-06-24T06:48:20Z
author_association: OWNER
type: issue
state_reason: completed
body:
  See also #103 

  It looks like you can start a Google Cloud VM with a "docker container" option - and the Google Cloud Registry is easy to push containers to. So it would be feasible to have `datasette publish gcloud ...` automatically build a container, push it to GCR, then start a new VM instance with it:

  https://cloud.google.com/container-registry/docs/pushing-and-pulling

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/146/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzY0NzY2NzA=
title: Tidy up design of the header of the table page
state: closed
created_at: 2017-11-23T21:52:58Z
updated_at: 2017-11-24T22:02:46Z
closed_at: 2017-11-24T22:02:46Z
author_association: OWNER
type: issue
state_reason: completed
body:
  This is a bit messy:

  <img width="883" alt="fatal-police-shootings-data__fatal-police-shootings-data" src="https://user-images.githubusercontent.com/9599/33189214-9379807c-d055-11e7-96c4-58a70529cef0.png">

  Depends on #127 

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/147/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzY0Nzc4ODg=
title: Need a != filter
state: closed
created_at: 2017-11-23T22:05:22Z
updated_at: 2017-11-23T22:10:02Z
closed_at: 2017-11-23T22:10:01Z
author_association: OWNER
type: issue
state_reason: completed
body:
  https://datasette-demos.now.sh/sf-trees-ebc2ad9/Street_Tree_List?qCareAssistant=1 shows trees managed by FUF - but how about trees that are NOT managed by FUF?

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/148/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzY3MDQxMjc=
title: Update custom SQL results to match new table view header
state: closed
created_at: 2017-11-24T22:03:59Z
updated_at: 2017-11-24T22:42:10Z
closed_at: 2017-11-24T22:42:09Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Follow-on from #147 - the custom SQL results page should more closely match the design of the table view, which now looks like this:

  <img width="798" alt="conventional_power_plants_eu__conventional_power_plants_eu__14_rows_where_company____nuon_" src="https://user-images.githubusercontent.com/9599/33224621-4d547f18-d120-11e7-9b33-44ee738a5ea2.png">

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/149/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzY3MDQzMjc=
title: _group_count= feature improvements
state: closed
created_at: 2017-11-24T22:06:18Z
updated_at: 2018-05-28T16:41:28Z
closed_at: 2018-05-28T16:41:28Z
author_association: OWNER
type: issue
state_reason: completed
body:
  - [ ] The "apply filters" form should keep you on the _group_count= page
  - [ ] Foreign key references should be expand
  - [ ] Page title should reflect the view you are on

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/150/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzY3MTg2MDU=
title: Set up a pattern portfolio
state: closed
created_at: 2017-11-25T02:09:49Z
updated_at: 2020-07-02T00:13:24Z
closed_at: 2020-05-03T03:13:16Z
author_association: OWNER
type: issue
state_reason: completed
body:
  https://www.slideshare.net/nataliedowne/practical-maintainable-css/75

  This will be a single page that demonstrates all of the different CSS styles and classes available to Datasette.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/151/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzY3NjUwNzA=
title: Incorrect display of rows page for tables with a primary key
state: closed
created_at: 2017-11-25T17:29:54Z
updated_at: 2017-12-07T05:23:20Z
closed_at: 2017-12-07T05:23:19Z
author_association: OWNER
type: issue
state_reason: completed
body:
  This is a regression. Here's the old version:

  <img width="420" alt="parlgov__party_family" src="https://user-images.githubusercontent.com/9599/33232903-031ef1a0-d1c3-11e7-94b4-10f9f0076b4f.png">

  And here's the new, broken one:

  https://parlgov-xtxlddmtiz.now.sh/parlgov-25f9855/party_family/1

  <img width="420" alt="parlgov__party_family" src="https://user-images.githubusercontent.com/9599/33232908-12f885be-d1c3-11e7-97f1-cb420f2a5de4.png">

  The JSON output is the same for both - it's only the HTML representation that exhibits the bug.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/152/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzY4NDI1MzY=
title: Ability to customize presentation of specific columns in HTML view
state: closed
created_at: 2017-11-26T17:46:11Z
updated_at: 2017-12-10T02:08:45Z
closed_at: 2017-12-07T06:17:33Z
author_association: NONE
type: issue
state_reason: completed
body:
  This ties into https://github.com/simonw/datasette/issues/3 in some ways. It would be great to have some adaptability in the HTML views and to specific some columns as displaying in certain ways.

  - [x] 1. **Auto-parsing URIs into in-browser links.** Why? Lots of public data around cultural commons stuff links to a specific URL. This would be a great utility to turn on at the command line, just parse everything for URLs. Maybe they need to be underlined or represented in a different way than internal URLs.
  - [x] 2. **Ability to identify a column as plain/preformatted text.** Why? Was trying to import the Enron emails, the body collapses. Hard to read. These fields also tend to screw up the ability to scan a table view. If you knew it was text the system could set an `overflow` property on the relevant CSS, so you could still scan.
  - [x] 3. **Ability to identify a column as HTML.** Why? I want to spider some stuff and drop sections into SQLite, and just keep them as HTML.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/153/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzY4NzM4OTE=
title: Datasette CSS should include content hash in the URL
state: closed
created_at: 2017-11-27T00:57:36Z
updated_at: 2017-12-09T03:10:23Z
closed_at: 2017-12-09T03:10:22Z
author_association: OWNER
type: issue
state_reason: completed
body:
  When I deployed the latest version of datasette to https://fivethirtyeight.datasettes.com/ I noticed I was getting served stale CSS since it had been cached. Including the sha of he contents in its URL should fix that.

  I can calculate this on server start.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/154/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzc1ODk1Njk=
title: A primary key column that has foreign key restriction associated won't rendering label column
state: closed
created_at: 2017-11-29T00:40:02Z
updated_at: 2017-12-07T05:39:53Z
closed_at: 2017-12-07T05:39:53Z
author_association: NONE
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/155/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzgxODk3MDg=
title: Document CSS hooks and custom templates
state: closed
created_at: 2017-11-30T16:43:15Z
updated_at: 2017-11-30T17:11:34Z
closed_at: 2017-11-30T17:10:58Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Documentation currently lives in commit messages on https://github.com/simonw/datasette/commit/8ab3a169d42d096f2c7979c6d3d7746618d30f0b and 3cd06729f457d690603b6060dc552b535517ab09

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/156/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzgxOTAzMjE=
title: Teach "datasette publish" about custom template directories
state: closed
created_at: 2017-11-30T16:44:57Z
updated_at: 2020-01-15T16:05:13Z
closed_at: 2017-12-09T18:28:54Z
author_association: OWNER
type: issue
state_reason: completed
body:
  The following command should copy the custom templates into the deployment and ensure `datasette serve` correctly serves them:

      datasette publish now mydb.db --template-dir=custom-templates/

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/157/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzgxOTA5ODE=
title: Ensure default templates are designed to be extended
state: closed
created_at: 2017-11-30T16:46:41Z
updated_at: 2017-12-07T05:41:09Z
closed_at: 2017-12-07T05:41:08Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Since custom templates can do `{% extends "default:table.html" %}` the default templates should include sensible named `{% block %}` components designed to support common extension patterns. Since we already support `{{ super() }}` we may not have much if anything to add here.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/158/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzgxOTEyMjM=
title: Come up with an elegant mechanism for per-row template customization
state: closed
created_at: 2017-11-30T16:47:26Z
updated_at: 2017-12-07T06:12:27Z
closed_at: 2017-12-07T06:12:26Z
author_association: OWNER
type: issue
state_reason: completed
body:
  It would be nice if customizing the display of an individual row in a custom table template was as simple as possible - refs #153 

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/159/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzgyMDgwMTE=
title: Ability to bundle and serve additional static files
state: closed
created_at: 2017-11-30T17:37:51Z
updated_at: 2019-02-02T00:58:20Z
closed_at: 2017-12-09T18:29:11Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Since we now have custom templates, we should support including custom static files with them as well. Maybe something like this:

      datasette mydb.db --template-dir=templates/ --static-dir=static/

  This should also be supported by datasette publish - see also #157 

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/160/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzg4MTQyMjA=
title: Support WITH query
state: closed
created_at: 2017-12-03T20:00:40Z
updated_at: 2017-12-08T06:18:12Z
closed_at: 2017-12-04T04:52:41Z
author_association: NONE
type: issue
state_reason: completed
body:
  Currently datasettle failed with error message: Statement must begin with SELECT

  Example query
  ```sql
  WITH RECURSIVE
    cnt(x) AS (
       SELECT 1
       UNION ALL
       SELECT x+1 FROM cnt
        LIMIT 1000000
    )
  SELECT x FROM cnt;
  ```

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/161/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzkxOTk5MTY=
title: Link should not show up in the column selection dropdowns
state: closed
created_at: 2017-12-05T00:19:04Z
updated_at: 2017-12-07T05:05:58Z
closed_at: 2017-12-07T05:05:58Z
author_association: OWNER
type: issue
state_reason: completed
body:
  e.g. on https://san-francisco.datasettes.com/food-trucks-921342f/Applicant

  <img width="729" alt="food-trucks__applicant__131_rows" src="https://user-images.githubusercontent.com/9599/33583381-ece18de6-d90e-11e7-8287-d5e4813dcb78.png">

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/162/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyNzk1NDc4ODY=
title: Document the querystring argument for setting a different time limit
state: closed
created_at: 2017-12-05T22:05:08Z
updated_at: 2021-03-23T02:44:33Z
closed_at: 2017-12-06T15:06:57Z
author_association: OWNER
type: issue
state_reason: completed
body:
  http://datasette.readthedocs.io/en/latest/sql_queries.html#query-limits

  Need to explain why this is useful too.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/163/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyODAwMTM5MDc=
title: datasette skeleton command for kick-starting database and table metadata
state: closed
created_at: 2017-12-07T06:13:28Z
updated_at: 2021-03-23T02:45:12Z
closed_at: 2017-12-07T06:20:45Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Generates an example `metadata.json` file populated with all of the databases and tables inspected from the specified databases.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/164/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyODAwMTQyODc=
title: metadata.json support for per-database and per-table information
state: closed
created_at: 2017-12-07T06:15:34Z
updated_at: 2017-12-07T16:48:34Z
closed_at: 2017-12-07T16:47:29Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Every database and every table should be able to support the following optional metadata:

  	title
  	description
  	description_html
  	license
  	license_url
  	source
  	source_url

  If `description_html` is provided it over-rides `description` and will be displayed unescaped.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/165/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyODAwMjMyMjU=
title: Documentation for metadata.json and datasette skeleton
state: closed
created_at: 2017-12-07T07:02:52Z
updated_at: 2017-12-07T17:20:35Z
closed_at: 2017-12-07T17:20:25Z
author_association: OWNER
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/166/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyODAzMTUzNTI=
title: Nasty bug: last column not being correctly displayed
state: closed
created_at: 2017-12-07T23:23:46Z
updated_at: 2017-12-10T01:00:21Z
closed_at: 2017-12-10T01:00:20Z
author_association: OWNER
type: issue
state_reason: completed
body:
  e.g. https://datasette-bwnojrhmmg.now.sh/dk3-bde9a9a/dk?source__contains=http

  ![2017-12-07 at 3 22 pm](https://user-images.githubusercontent.com/9599/33743613-7ee97be0-db62-11e7-8e81-9b9ec69d93f0.png)

  The JSON output shows that the column is there, but is being displayed incorrectly:

  https://datasette-bwnojrhmmg.now.sh/dk3-bde9a9a/dk.jsono?source__contains=http

  ![2017-12-07 at 3 23 pm](https://user-images.githubusercontent.com/9599/33743645-9489b302-db62-11e7-898b-72e812e8855d.png)

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/167/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MTU3MzY1ODEx
title: Upgrade to Sanic 0.7.0
state: closed
created_at: 2017-12-09T01:25:08Z
updated_at: 2017-12-09T03:00:34Z
closed_at: 2017-12-09T03:00:34Z
author_association: OWNER
pull_request: simonw/datasette/pulls/168
type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/168/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyODA3NDQzMDk=
title: Release v0.14 with templates and static files features
state: closed
created_at: 2017-12-09T18:52:48Z
updated_at: 2017-12-10T02:04:56Z
closed_at: 2017-12-10T02:04:56Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Everything in this milestone https://github.com/simonw/datasette/milestone/6 - plus various other fixes: https://github.com/simonw/datasette/compare/0.13...6bdfcf60760c27e29ff34692d06e62b36aeecc56

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/169/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyODA3NDU0NzA=
title: Custom template for named canned query
state: closed
created_at: 2017-12-09T19:07:51Z
updated_at: 2017-12-09T21:35:30Z
closed_at: 2017-12-09T21:34:52Z
author_association: OWNER
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/170/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyODA3NDU3NDY=
title: HTML comments specifying custom templates for page
state: closed
created_at: 2017-12-09T19:11:13Z
updated_at: 2017-12-09T21:50:50Z
closed_at: 2017-12-09T21:48:03Z
author_association: OWNER
type: issue
state_reason: completed
body:
   This would make the custom templating system self-documenting, and save people from having to figure out the right template names for customizing specific pages.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/171/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyODA4OTYyOTA=
title: Show size of .db file next to download link
state: closed
created_at: 2017-12-11T05:12:46Z
updated_at: 2019-02-06T05:09:06Z
closed_at: 2019-02-06T05:00:36Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Size in bytes should be calculated by datasette inspect.

  Template should display it in KB or MB or GB

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/172/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyODExMTAyOTU=
title: I18n and L10n support
state: open
created_at: 2017-12-11T17:49:58Z
updated_at: 2021-04-26T12:10:01Z
author_association: NONE
type: issue
body:
  It would be less geeky and more user friendly if the display strings in the filter menu and possibly other parts could be localized.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/173/reactions", "total_count": 2, "+1": 2, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyODExOTc4NjM=
title: License/Source in footer should inherit from top level
state: closed
created_at: 2017-12-11T23:01:35Z
updated_at: 2018-08-11T17:46:51Z
closed_at: 2018-08-11T17:46:51Z
author_association: OWNER
type: issue
state_reason: completed
body:
  The footer on https://vice-police-shootings.now.sh/vice-bc7c892/ViceNews_FullOISData does not show license and source information... but that Datasette has that information, it's just defined at the top level: https://vice-police-shootings.now.sh/

  The footer for a row/table page should fall back on information for the database, and if there is none for the database it should fall back on the top-level metadata instead.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/174/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyODI5NzE5NjE=
title: Add project topic "automatic-api"
state: closed
created_at: 2017-12-18T18:09:17Z
updated_at: 2017-12-21T18:33:55Z
closed_at: 2017-12-21T18:33:55Z
author_association: NONE
type: issue
state_reason: completed
body:
  Hi there! Could you add the ~~tag~~ topic `automatic-api` to your repository? I am [making a list](https://github.com/dbohdan/automatic-api) of all projects that automatically expose APIs to databases. (Your Show&nbsp;HN made me do it.&nbsp;:-) I knew about PostgREST and PostGraphQL, but it took adding Datasette to sell me on the concept.) They will be easier to discover if there is a standard GitHub tag, and `automatic-api` seems as good a candidate as any. Two projects [already use it](https://github.com/topics/automatic-api).

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/175/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyODUxNjg1MDM=
title: Add GraphQL endpoint
state: open
created_at: 2017-12-29T23:21:01Z
updated_at: 2020-04-21T14:16:24Z
author_association: NONE
type: issue
body:
  Would make it much easier to build React & similar frontends. Maybe with https://github.com/graphql-python/sanic-graphql ?

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/176/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyODY5Mzg1ODk=
title: Publishing to Heroku - metadata file not uploaded?
state: closed
created_at: 2018-01-09T01:04:31Z
updated_at: 2018-01-25T16:45:32Z
closed_at: 2018-01-25T16:45:32Z
author_association: CONTRIBUTOR
type: issue
state_reason: completed
body:
  Trying to run *datasette* (version 0.14)  on Heroku with a `metadata.json` doesn't seem to be picking up the `metadata.json` file? 

  On a Mac with dodgy `tar` support:

  ```
   ▸    Couldn't detect GNU tar. Builds could fail due to decompression errors
   ▸    See
   ▸    https://devcenter.heroku.com/articles/platform-api-deploying-slugs#create-slug-archive
   ▸    Please install it, or specify the '--tar' option
   ▸    Falling back to node's built-in compressor
  ```

  Could that be causing the issue?

  Also, I'm not seeing custom query links anywhere obvious when I run the metadata file with a local *datasette* server?

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/177/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MTYxOTgyNzEx
title: If metadata exists, add it to heroku launch command
state: closed
created_at: 2018-01-09T21:42:21Z
updated_at: 2018-01-15T09:42:46Z
closed_at: 2018-01-14T21:05:16Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/178
type: pull
body:
  The heroku build does seem to make use of any provided `metadata.json` file.

  Add the `--metadata` switch to the Heroku web launch command if a `metadata.json` file is available.

  Addresses: https://github.com/simonw/datasette/issues/177

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/178/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyODg0Mzg1NzA=
title: More metadata options for template authors
state: open
created_at: 2018-01-14T20:51:04Z
updated_at: 2019-05-13T18:33:33Z
author_association: OWNER
body: See this thread on Twitter: https://twitter.com/simonw/status/952637152797458432
type: issue
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/179/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MTYzNTIzOTc2
title: make html title more readable in query template
state: closed
created_at: 2018-01-17T18:56:03Z
updated_at: 2018-04-03T16:03:38Z
closed_at: 2018-04-03T15:24:05Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/180
body: tiny tweak to make this easier to visually parse—I think it matches your style in other templates
type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/180/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MTYzNTYxODMw
title: add "format sql" button to query page, uses sql-formatter
state: closed
created_at: 2018-01-17T21:50:04Z
updated_at: 2019-11-11T03:08:25Z
closed_at: 2019-11-11T03:08:25Z
author_association: NONE
pull_request: simonw/datasette/pulls/181
type: pull
body:
  Cool project!

  This fixes #136 using the suggested [sql formatter](https://github.com/zeroturnaround/sql-formatter) library. I included the minified version in the bundle and added the relevant scripts to the codemirror includes instead of adding new files, though I could also add new files. I wanted to keep it all together, since the result of the format needs access to the editor in order to properly update the codemirror instance.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/181/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MTY1MDI5ODA3
title: Add db filesize next to download link
state: closed
created_at: 2018-01-25T04:58:56Z
updated_at: 2019-03-22T13:50:57Z
closed_at: 2019-02-06T04:59:38Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/182
body: Took a stab at #172, will this do the trick?
type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/182/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyOTE2MzkxMTg=
title: Custom Queries - escaping strings
state: closed
created_at: 2018-01-25T16:49:13Z
updated_at: 2019-06-24T06:45:07Z
closed_at: 2019-06-24T06:45:07Z
author_association: CONTRIBUTOR
type: issue
state_reason: completed
body:
  If a SQLite table column name contains spaces, they are usually referred to in double quotes:

  `SELECT * FROM mytable WHERE "gappy column name"="my value";`

  In the JSON metadata file, this is passed by escaping the double quotes:

  `"queries": {"my query": "SELECT * FROM mytable WHERE \"gappy column name\"=\"my value\";"}`

  When specifying a custom query in `metadata.json` using double quotes, these are then rendered in the *datasette* query box using single quotes:

  `SELECT * FROM mytable WHERE 'gappy column name'='my value';`

  which does not work.

  Alternatively, a valid custom query can be passed using backticks (\`) to quote the column name and single (unescaped) quotes for the matched value:

  ``"queries": {"my query": "SELECT * FROM mytable WHERE `gappy column name`='my value';"}``

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/183/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyOTIwMTEzNzk=
title: 500 from missing table name
state: closed
created_at: 2018-01-26T19:46:45Z
updated_at: 2019-05-21T16:17:29Z
closed_at: 2018-04-13T18:18:59Z
author_association: NONE
type: issue
state_reason: completed
body:
  https://github.com/simonw/datasette/blob/56623e48da5412b25fb39cc26b9c743b684dd968/datasette/app.py#L517-L519 throws an error if it gets an empty list back. Simplest solution is to write a helper func that just says 

  ```python
  result = list(await self.execute(name, sql, params)
  if result:
      return result[0][0]
  ```

  and use it anywhere `[0][0]` is now.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/184/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUyOTk3NjA2ODQ=
title: Metadata should be a nested arbitrary KV store
state: open
created_at: 2018-02-23T16:02:07Z
updated_at: 2019-05-13T18:33:33Z
author_association: NONE
type: issue
body:
  I started using the metadata feature and was surprised to find that values are not inherited from the root object down to specific databases and tables. This makes metadata much less useful and requires a lot of pointless duplication.

  Ideally, metadata should allow arbitrary key-value pairs, and there should be a way of accessing metadata either in an inherited or non-inherited manner. Something like `metadata.page.key` vs. `metadata.this.key` might work as an interface.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/185/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMDY4MTE1MTM=
title: proposal new option to disable user agents cache
state: closed
created_at: 2018-03-20T10:42:20Z
updated_at: 2018-03-21T09:07:22Z
closed_at: 2018-03-21T01:28:31Z
author_association: NONE
type: issue
state_reason: completed
body:
  I think it would be very useful for debugging an option of adding headers to http replies

  ```
  Cache-Control: no-cache 
  ```

  especially in the html output

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/186/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMDkwMzM5OTg=
title: Windows installation error
state: closed
created_at: 2018-03-27T16:04:37Z
updated_at: 2019-06-15T21:44:23Z
closed_at: 2019-06-15T21:44:23Z
author_association: NONE
type: issue
state_reason: completed
body:
  On attempting install on a Win 7 PC with py 3.6.2 (Anaconda dist) I get the error:

  ```
  Collecting uvloop>=0.5.3 (from Sanic==0.7.0->datasette)
    Downloading uvloop-0.9.1.tar.gz (1.8MB)
      100% |¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦| 1.8MB 12.8MB/s
      Complete output from command python setup.py egg_info:
      Traceback (most recent call last):
        File "<string>", line 1, in <module>
        File "C:\Users\RCole\AppData\Local\Temp\pip-build-juakfqt8\uvloop\setup.py
  ", line 10, in <module>
          raise RuntimeError('uvloop does not support Windows at the moment')
      RuntimeError: uvloop does not support Windows at the moment
  ```

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/187/reactions", "total_count": 4, "+1": 4, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMDkwNDc0NjA=
title: Ability to bundle metadata and templates inside the SQLite file
state: open
created_at: 2018-03-27T16:42:07Z
updated_at: 2020-12-04T17:18:34Z
author_association: OWNER
type: issue
body:
  One of the nicest qualities of SQLite as a data format is that you get a single file which you can then backup or share with other people. 

  Datasette breaks this a little once you start including custom metadata.json or template files and CSS.

  It would be cool if there was an optional mechanism for baking that extra configuration into the SQLite file itself. That way entire datasette mini-applications (including canned queries and custom HTML and CSS) could be constructed as single .db files.

  Since datasette configuration is all file-based, one way to achieve that would be to support a "datasette_files" table which, if present is used to search for file contents by path.

  This is inline with the philosophy described by https://www.sqlite.org/appfileformat.html

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/188/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMDk0NzE4MTQ=
title: Ability to sort (and paginate) by column
state: closed
created_at: 2018-03-28T18:04:51Z
updated_at: 2018-04-15T18:54:22Z
closed_at: 2018-04-09T05:16:02Z
author_association: OWNER
type: issue
state_reason: completed
body:
  As requested in https://github.com/simonw/datasette/issues/185#issuecomment-376614973

  I've previously avoided this for performance reasons: sort-by-column on a column without an index is likely to perform badly for hundreds of thousands of rows.

  That's not a good enough reason to avoid the feature entirely though. A few options:

  * Allow sort-by-column by default, give users the option to disable it for specific tables/columns
  * Disallow sort-by-column by default, give users option (probably in `metadata.json`) to enable it for specific tables/columns
  * Automatically detect if a column either has an index on it OR a table has less than X rows in it

  We already have the mechanism in place to cut off SQL queries that take more than X seconds, so if someone DOES try to sort by a column that's too expensive it won't actually hurt anything - but it would be nice to not show people a "sort" option which is guaranteed to throw a timeout error.

  The vast majority of datasette usage that I've seen so far is on smaller datasets where the performance penalties of sort-by-column are extremely unlikely to show up.

  ----

  Still left to do:

  - [x] UI that shows which sort order is currently being applied (in HTML and in JSON)
  - [x] UI for applying a sort order (with rel=nofollow to avoid Google crawling it)
  - [x] Sort column names should be escaped correctly in generated SQL
  - [x] Validation that the selected sort order is a valid column
  - [x] Throw error if user attempts to apply _sort AND _sort_desc at the same time
  - [x] Ability to disable sorting (or sort only for specific columns) in metadata.json
  - [x] Fix "201 rows where sorted by sortable_with_nulls " bug

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/189/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMDk1NTg4MjY=
title: Keyset pagination doesn't work correctly for compound primary keys
state: closed
created_at: 2018-03-28T22:45:06Z
updated_at: 2018-03-30T06:31:15Z
closed_at: 2018-03-30T06:26:28Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Consider https://datasette-issue-190-compound-pks.now.sh/compound-pks-9aafe8f/compound_primary_key

  ![2018-03-28 at 3 47 pm](https://user-images.githubusercontent.com/9599/38060388-56da86dc-329f-11e8-9f20-5576153ad55c.png)

  The next= link is to `d,v`:

  https://datasette-issue-190-compound-pks.now.sh/compound-pks-9aafe8f/compound_primary_key?_next=d%2Cv

  But that page starts with:

  ![2018-03-28 at 3 48 pm](https://user-images.githubusercontent.com/9599/38060402-6b0f5984-329f-11e8-85b8-44a666c4ee71.png)

  The next key in the sequence should be `d,w`. Also we should return the full a-z of the ones that start with the letter e - in this example we only return `e-w`, `e-x`, `e-y` and `e-z`

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/190/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMTA1MzMyNTg=
title: Figure out how to bundle a more up-to-date SQLite
state: closed
created_at: 2018-04-02T16:33:25Z
updated_at: 2018-07-10T17:46:13Z
closed_at: 2018-07-10T17:46:13Z
author_association: OWNER
type: issue
state_reason: completed
body:
  The version of SQLite that ships with Python 3 is a bit limited - it doesn't support row values for example https://www.sqlite.org/rowvalue.html

  Figure out how to bundle a more recent SQLite engine with datasette. We need to figure out two cases:

  * Bundling a recent version in a Dockerfile build. I expect this to be quite easy.
  * Making a more recent version available to people hacking around in Mac OS X. I have no idea how to start on this.

  I want it working on Mac OS X too because I don't want to force Docker as a dependency for anyone who just want to hack around with Datasette a little and run the test suite.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/191/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MTc5MTA4OTYx
title: New ?_shape=objects/object/lists param for JSON API
state: closed
created_at: 2018-04-03T14:02:58Z
updated_at: 2018-04-03T14:53:00Z
closed_at: 2018-04-03T14:52:55Z
author_association: OWNER
pull_request: simonw/datasette/pulls/192
body: Refs #122
type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/192/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMTA4ODIxMDA=
title: Cleaner mechanism for handling custom errors
state: closed
created_at: 2018-04-03T15:19:13Z
updated_at: 2018-04-13T18:18:59Z
closed_at: 2018-04-13T18:18:59Z
author_association: OWNER
type: issue
state_reason: completed
body:
  This code is pretty messy:

  https://github.com/simonw/datasette/blob/0abd3abacb309a2bd5913a7a2df4e9256585b1bb/datasette/app.py#L245-L265

  Instead, it would be nice if I could raise an exception that would be converted into the appropriate JSON or HTML error message, with a corresponding HTTP code.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/193/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMTIzMTIxMjU=
title: Rename table_rows and filtered_table_rows to have _count suffix
state: closed
created_at: 2018-04-08T14:53:37Z
updated_at: 2018-04-09T05:25:22Z
closed_at: 2018-04-09T05:25:22Z
author_association: OWNER
type: issue
state_reason: completed
body:
  These fields represent counts of items:

      "table_rows": 131,
      "filtered_table_rows": 8,

  But the names make it sound like they might be arrays full of rows. Adding a `_count` suffix would make this more clear:

      "table_rows_count": 131,
      "filtered_table_rows_count": 8,

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/194/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMTIzMTM0OTY=
title: Run pks_for_table in inspect, executing once at build time rather than constantly
state: closed
created_at: 2018-04-08T15:12:40Z
updated_at: 2018-04-10T00:54:43Z
closed_at: 2018-04-10T00:54:43Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Right now several Datasette views call the `await self.pks_for_table(...)` method to figure out what primary keys are set for a specific table. This executes a `PRAGMA table_info` SQL query.

  It would be faster and more efficient to execute this query for each table as part of the `inspect()` method.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/195/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MTgwMTg4Mzk3
title: _sort= and _sort_desc= parameters to table view
state: closed
created_at: 2018-04-09T00:07:21Z
updated_at: 2018-04-09T05:10:29Z
closed_at: 2018-04-09T05:10:23Z
author_association: OWNER
pull_request: simonw/datasette/pulls/196
body: See #189
type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/196/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMTIzOTU3OTA=
title: Ability to sort by more than one column
state: open
created_at: 2018-04-09T05:13:30Z
updated_at: 2018-07-10T17:45:37Z
author_association: OWNER
type: issue
body:
  Split off from #189.

  I'd like to support "sort by X descending, then by Y ascending if there are dupes for X" as well. Suggested syntax for that:

      ?_sort_desc=X&_sort=Y

  we currently only allow one argument to be sent. We should allow as many arguments as there are columns, for example:

      ?_sort=department&_sort_desc=precinct&_sort=age&_sort_desc=size

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/197/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMTIzOTYwOTU=
title: Ability to sort with nulls last
state: open
created_at: 2018-04-09T05:15:40Z
updated_at: 2018-07-10T17:45:37Z
author_association: OWNER
type: issue
body:
  Split off from #189

  Here's how to do that in SQL: https://fivethirtyeight.datasettes.com/fivethirtyeight-2628db9?sql=select+rowid%2C+*+from+%5Bnfl-wide-receivers%2Fadvanced-historical%5D%0D%0Aorder+by+case+when+career_ranypa+is+null+then+1+else+0+end%2C+career_ranypa%2C+rowid

      order by case when career_ranypa is null then 1 else 0 end, career_ranypa

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/198/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMTI2MjA1NjY=
title: Ability to apply sort on mobile in portrait mode
state: closed
created_at: 2018-04-09T17:35:04Z
updated_at: 2018-04-10T00:37:53Z
closed_at: 2018-04-10T00:34:38Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Missed this in #189... on mobile in portrait mode we hide the column headers, which means you can't click them to sort! You can sort in landscape mode at least.

  Need to come up with an alternative sort UI for portrait on mobile.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/199/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MTgxMDMzMDI0
title: Hide Spatialite system tables
state: closed
created_at: 2018-04-11T21:26:58Z
updated_at: 2018-04-12T21:34:48Z
closed_at: 2018-04-12T21:34:48Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/200
body: They were getting on my nerves.
type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/200/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMTM1MTI3NDg=
title: Support explain select / explain query plan select
state: closed
created_at: 2018-04-11T22:41:26Z
updated_at: 2018-04-13T21:17:14Z
closed_at: 2018-04-12T21:32:52Z
author_association: OWNER
body: See https://www.sqlite.org/eqp.html and https://www.sqlite.org/lang_explain.html
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/201/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MTgxMjQ3NTY4
title: Raise 404 on nonexistent table URLs
state: closed
created_at: 2018-04-12T15:47:06Z
updated_at: 2018-04-13T19:22:56Z
closed_at: 2018-04-13T18:19:15Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/202
type: pull
body:
  Currently they just 500. Also cleaned the logic up a bit, I hope I didn't miss anything.

  This is issue #184.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/202/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMTM4MzczMDM=
title: Support for units
state: closed
created_at: 2018-04-12T18:24:28Z
updated_at: 2018-04-16T21:59:17Z
closed_at: 2018-04-16T21:59:17Z
author_association: CONTRIBUTOR
type: issue
state_reason: completed
body:
  It would be nice to be able to attach a unit to a column in the metadata, and have it rendered with that unit (and SI prefix) when it's displayed.

  It would also be nice to support entering the prefixes in variables when querying.

  With my radio licensing app I've put all frequencies in Hz. It's easy enough to special-case the row rendering to add the SI prefixes, but it's pretty unusable when querying by that field.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/203/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MTgxNjAwOTI2
title: Initial units support
state: closed
created_at: 2018-04-13T21:32:49Z
updated_at: 2018-04-14T09:44:33Z
closed_at: 2018-04-14T03:32:54Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/204
type: pull
body:
  Add support for specifying units for a column in metadata.json and rendering them on display using [pint](https://pint.readthedocs.io/en/latest/).

  Example table metadata:
  ```json
                  "license_frequency": {
                      "units": {
                          "frequency": "Hz",
                          "channel_width": "Hz",
                          "height": "m",
                          "antenna_height": "m",
                          "azimuth": "degrees"
                      }
                  }
  ```

  [Example result](https://wtr-api.herokuapp.com/wtr-663ea99/license_frequency/1)

  This works surprisingly well! I'd like to add support for using units when querying but this is PR is pretty usable as-is.

  (Pint doesn't seem to support decibels though - it thinks they're decibytes - which is an annoying omission.)

  (ref ticket #203)

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/204/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MTgxNjQyMTE0
title: Support filtering with units and more
state: closed
created_at: 2018-04-14T10:47:51Z
updated_at: 2018-04-14T15:24:04Z
closed_at: 2018-04-14T15:24:04Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/205
type: pull
body:
  The first commit:
  * Adds units to exported JSON
  * Adds units key to metadata skeleton
  * Adds some docs for units

  The second commit adds filtering by units by the first method I mentioned in #203:
  ![image](https://user-images.githubusercontent.com/45057/38767463-7193be16-3fd9-11e8-8a5f-ac4159415c6d.png)

  [Try it here](https://wtr-api.herokuapp.com/wtr-663ea99/license_frequency?frequency__gt=50GHz&height__lt=50ft). I think it integrates pretty neatly.

  The third commit adds support for registering custom units with Pint from metadata.json. Probably pretty niche, but I need decibels!

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/205/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MTgxNjQ0ODA1
title: Fix sqlite error when loading rows with no incoming FKs
state: closed
created_at: 2018-04-14T12:08:17Z
updated_at: 2018-04-14T14:32:42Z
closed_at: 2018-04-14T14:24:25Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/206
type: pull
body:
  This fixes `ERROR: conn=<sqlite3.Connection object at 0x10bbb9f10>, sql
  = 'select ', params = {'id': '1'}` caused by an invalid query loading incoming FKs when none exist.

  The error was ignored due to async but it still got printed to the console.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/206/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MTgxNjQ3NzE3
title: Link foreign keys which don't have labels
state: closed
created_at: 2018-04-14T13:27:14Z
updated_at: 2018-04-14T15:00:00Z
closed_at: 2018-04-14T15:00:00Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/207
type: pull
body:
  This renders unlabeled FKs as simple links. I can't see why this would cause any major problems.

  ![image](https://user-images.githubusercontent.com/45057/38768722-ea15a000-3fef-11e8-8664-ffd7aa4894ea.png)

  Also includes bonus fixes for two minor issues:

  * In foreign key link hrefs the primary key was escaped using HTML escaping rather than URL escaping. This broke some non-integer PKs.
  * Print tracebacks to console when handling 500 errors.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/207/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MTgxNjU0ODM5
title: Return HTTP 405 on InvalidUsage rather than 500
state: closed
created_at: 2018-04-14T16:12:50Z
updated_at: 2018-04-14T18:00:39Z
closed_at: 2018-04-14T18:00:39Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/208
type: pull
body:
  This also stops it filling up the logs. This happens for HEAD requests at the moment - which perhaps should be handled better, but that's a different issue.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/208/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MTgxNzIzMzAz
title: Don't duplicate simple primary keys in the link column
state: closed
created_at: 2018-04-15T21:56:15Z
updated_at: 2018-04-18T08:40:37Z
closed_at: 2018-04-18T01:13:04Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/209
type: pull
body:
  When there's a simple (single-column) primary key, it looks weird to duplicate it in the link column.

  This change removes the second PK column and treats the link column as if it were the PK column from a header/sorting perspective. 

  This might make it a bit more difficult to tell what the link for the row is, I'm not sure yet. I feel like the alternative is to change the link column to just have the text "view" or something, instead of repeating the PK. (I doubt it makes much more sense with compound PKs.)

  Bonus change in this PR: fix urlencoding of links in the displayed HTML.

  Before:
  ![image](https://user-images.githubusercontent.com/45057/38783830-e2ababb4-40ff-11e8-97fb-25e286a8c920.png)

  After:
  ![image](https://user-images.githubusercontent.com/45057/38783835-ebf6b48e-40ff-11e8-8c47-6a864cf21ccc.png)

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/209/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MTgxNzMxOTU2
title: Start of the plugin system, based on pluggy
state: closed
created_at: 2018-04-16T00:51:30Z
updated_at: 2018-04-16T00:56:16Z
closed_at: 2018-04-16T00:56:16Z
author_association: OWNER
pull_request: simonw/datasette/pulls/210
body: Refs #14
type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/210/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMTQ0NzE3NDM=
title: Load plugins from a `--plugins-dir=plugins/` directory
state: closed
created_at: 2018-04-16T01:17:43Z
updated_at: 2018-04-16T05:22:02Z
closed_at: 2018-04-16T05:22:02Z
author_association: OWNER
type: issue
state_reason: completed
body:
  In #14 and 33c7c53ff87c2 I've added working support for setuptools entry_points plugins. These can be installed from PyPI using `pip install ...`.

  I imagine some projects will benefit from being able to add plugins without first publishing them to PyPI. Datasette already supports [loading custom templates](http://datasette.readthedocs.io/en/latest/custom_templates.html#custom-templates) like so:

      datasette serve --template-dir=mytemplates/ mydb.db

  I propose an additional option, `--plugins-dir=` which specifies a directory full of `blah.py` files which will be loaded into Datasette when the application server starts.

      datasette serve --plugins-dir=myplugins/ mydb.db

  This will also need to be supported by `datasette publish` as those Python files should be copied up as part of the deployment.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/211/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MTgxNzU1MjIw
title: New --plugins-dir=plugins/ option
state: closed
created_at: 2018-04-16T05:19:28Z
updated_at: 2018-04-16T05:22:18Z
closed_at: 2018-04-16T05:22:01Z
author_association: OWNER
pull_request: simonw/datasette/pulls/212
body: Refs #211
type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/212/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMTQ1MDYwMzM=
title: Documentation for plugins system
state: closed
created_at: 2018-04-16T05:27:07Z
updated_at: 2018-04-16T15:12:48Z
closed_at: 2018-04-16T15:12:48Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Documentation for #14 - how to write plugins, how to ship plugins to PyPI and how to use the `--plugins-dir` option added in #211 

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/213/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMTQ1MDY0NDY=
title: Ability for plugins to define extra JavaScript and CSS
state: closed
created_at: 2018-04-16T05:29:34Z
updated_at: 2020-09-30T20:36:11Z
closed_at: 2018-04-18T03:13:03Z
author_association: OWNER
type: issue
state_reason: completed
body:
  This can hook in to the existing `extra_css_urls` and `extra_js_urls` mechanism:

  https://github.com/simonw/datasette/blob/b2955d9065ea019500c7d072bcd9d49d1967f051/datasette/app.py#L304-L305

  The plugins should be able to bundle their own assets though, so it will also have to integrate with the `/static/` static mounts mechanism somehow:

  https://github.com/simonw/datasette/blob/b2955d9065ea019500c7d072bcd9d49d1967f051/datasette/app.py#L1255-L1257

  Refs #14

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/214/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMTQ1MDY2Njk=
title: Allow plugins to define additional URL routes and views
state: closed
created_at: 2018-04-16T05:31:09Z
updated_at: 2020-06-09T03:14:32Z
closed_at: 2020-06-09T03:12:08Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Might be as simple as having plugins get passed the `app` after the other routes have been defined:

  https://github.com/simonw/datasette/blob/b2955d9065ea019500c7d072bcd9d49d1967f051/datasette/app.py#L1270-L1274

  Refs #14

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/215/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMTQ2NjUxNDc=
title: Bug: Sort by column with NULL in next_page URL
state: closed
created_at: 2018-04-16T14:03:18Z
updated_at: 2018-04-17T01:45:24Z
closed_at: 2018-04-17T01:45:24Z
author_association: NONE
type: issue
state_reason: completed
body:
  Copy-pasting from https://github.com/simonw/datasette/issues/189#issuecomment-381429213, since that issue is closed:

  I think I found a bug. I tried to sort by middle initial in my salaries set, and many middle initials are null. The `next_url` gets set by Datasette to:

  http://localhost:8001/salaries-d3a5631/2017+Maryland+state+salaries?_next=None%2C391&_sort=middle_initial

  But then None is interpreted literally and it tries to find a name with the middle initial "None" and ends up skipping ahead to O on page 2.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/216/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMTQ3MjUzNDI=
title: Plugin support for datasette publish
state: closed
created_at: 2018-04-16T16:17:14Z
updated_at: 2018-07-26T05:33:39Z
closed_at: 2018-07-26T05:16:00Z
author_association: OWNER
type: issue
state_reason: completed
body:
  It should be possible to support additional deployment options by writing a plugin (see #59).

  As part of this, rewrite the Heroku and Now publishers to be implemented as plugins (they will still ship with datasette by default).

  Maybe `datasette package` should be changed to being part of publish instead, `datasette publish docker` perhaps? 

  Refs #14

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/217/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMTQ3NzE2MTU=
title: Support custom unit display in order to handle "$10,000"
state: open
created_at: 2018-04-16T18:39:31Z
updated_at: 2018-07-10T17:45:38Z
author_association: OWNER
type: issue
body:
  I tried to get Datasette to display `$10,000` using the new units support but we currently only display units as a suffix:

  https://github.com/simonw/datasette/blob/10a34f995c70daa37a8a2aa02c3135a4b023a24c/datasette/app.py#L563-L572

  It would be neat if there was a mechanism for specifying a custom unit display - maybe something like this:

  ```
  {
      "custom_units": {
          "us_dollar": {
              "unit": "us_dollar = [] = $",
              "format": "${:,}"
          }
      }
  }
  ```

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/218/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMTQ4MzQ3ODM=
title: Expose units in the JSON API?
state: open
created_at: 2018-04-16T22:04:25Z
updated_at: 2018-04-16T22:04:25Z
author_association: CONTRIBUTOR
type: issue
body:
  From #203: it would be nice for the JSON API to (optionally) return columns rendered with units in them - if, for example, you're consuming the JSON to render the rows on a map.

  I'm not entirely sure how useful this will be though - at the moment my map queries are custom SQL queries (a few have joins in, the rest might be fetching large amounts of data so it makes sense to limit columns fetched). Perhaps the SQL function is a better approach in general.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/219/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMTQ4NDc1NzE=
title: Investigate syntactic sugar for plugins
state: closed
created_at: 2018-04-16T23:01:39Z
updated_at: 2020-06-11T21:50:06Z
closed_at: 2020-06-11T21:49:55Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Suggested by @andrewhayward on Twitter: https://twitter.com/arhayward/status/986015118965268480?s=21

  > Have you considered a basic abstraction on top of that, for standard hook features?

  ```
  @sql_function
  random_integer(a,b):
      return random.randint(a,b)

  @template_filter
  uppercase(str):
      return str.upper()
  ```

  Maybe `from datasette.plugins import template_filter`?

  Would have to work out how to get this to play well with pluggy

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/220/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMTUxNDI0MTQ=
title: Allow plugins to add new cli sub commands
state: closed
created_at: 2018-04-17T16:40:13Z
updated_at: 2021-01-04T20:12:14Z
closed_at: 2021-01-04T20:12:14Z
author_association: OWNER
type: issue
state_reason: completed
body:
  I could then test this out by having https://github.com/simonw/csvs-to-sqlite register itself as a plugin

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/221/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MTgyMzU3NjEz
title: Fix for plugins in Python 3.5
state: closed
created_at: 2018-04-18T03:21:01Z
updated_at: 2018-04-18T04:26:50Z
closed_at: 2018-04-18T03:24:21Z
author_association: OWNER
pull_request: simonw/datasette/pulls/222
type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/222/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMTUzMjc4NjA=
title: datasette publish --install=name-of-plugin
state: closed
created_at: 2018-04-18T04:33:59Z
updated_at: 2018-04-18T14:56:17Z
closed_at: 2018-04-18T14:56:17Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Mechanism for causing datasette publish and datasette package to install one or more additional plugins using `pip install` - refs #14 

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/223/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMTU1MTc1Nzg=
title: Ability for plugins to bundle templates
state: closed
created_at: 2018-04-18T14:57:53Z
updated_at: 2018-04-19T05:50:36Z
closed_at: 2018-04-19T05:50:36Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Plugins should be able to bundle templates.

  The Datasette template loader should then consult those plugins first when loading a template.

  Jinja2 has a `PackageLoader` class that can help with this: http://jinja.pocoo.org/docs/2.10/api/#jinja2.PackageLoader

  Refs #14

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/224/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMTU1NDg0OTU=
title: /-/(inspect|metadata|plugins)(.json)? introspection
state: closed
created_at: 2018-04-18T16:14:58Z
updated_at: 2018-04-19T05:25:33Z
closed_at: 2018-04-19T05:25:33Z
author_association: OWNER
type: issue
state_reason: completed
body:
  3 pages (and accompanying .json endpoints) for viewing:

  * the metadata.json that datasette was loaded with
  * the output of ds.inspect()
  * a list of installed plugins, detected by pluggy

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/225/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMTU3Mzg2OTY=
title: Unit tests for installable plugins
state: closed
created_at: 2018-04-19T06:05:32Z
updated_at: 2020-11-24T19:52:51Z
closed_at: 2020-11-24T19:52:46Z
author_association: OWNER
type: issue
state_reason: completed
body:
  I'd like more thorough unit test coverage of the plugins mechanism - in particular for installable plugins.

  I think I can do this while still having the code live in the same repo, by creating a subdirectory in tests/example_plugin with its own setup.py and then running `python setup.py install` as part of the test runner.

  I imagine I will need to bump the version number every time I change the plugin in case someone runs the test again in the same virtual environment.

  If that doesn't work I can instead ship a datasette-plugins-tests two to PyPI and add that as a tests_require dependency.

  Refs #14

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/226/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMTU5NjAyNzI=
title: prepare_context() plugin hook
state: closed
created_at: 2018-04-19T16:55:26Z
updated_at: 2020-03-24T22:19:54Z
closed_at: 2020-03-24T22:19:54Z
author_association: OWNER
type: issue
state_reason: completed
body:
  This would be called with the context dictionary before each template is rendered. It would have the opportunity to modify that context.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/227/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMTYwMzE1NjY=
title: If spatialite detected, mark idx_XXX_Geometry tables as hidden
state: closed
created_at: 2018-04-19T20:37:24Z
updated_at: 2018-04-26T03:25:39Z
closed_at: 2018-04-26T03:25:39Z
author_association: OWNER
type: issue
state_reason: completed
body:
  https://timezones-api.now.sh/timezones-faf26d0

  ![2018-04-19 at 1 36 pm](https://user-images.githubusercontent.com/9599/39016906-a5acbb3e-43d6-11e8-9a31-814ff1d0022e.png)

  Need to update this logic:

  https://github.com/simonw/datasette/blob/e2750c7cc0585adaa8c866be611089e62961ee35/datasette/app.py#L1276-L1288

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/228/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMTYxMjMyNTY=
title: Table view should support ?_size=400 parameter
state: closed
created_at: 2018-04-20T04:23:18Z
updated_at: 2018-04-26T04:49:46Z
closed_at: 2018-04-26T04:48:32Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Allows callers to request more rows at once. The limit will still be `max_returned_rows` (defaults to 1000).

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/229/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMTYxMjg5NTU=
title: Setting page size AND max returned rows to 1000 doesn't seem to work
state: closed
created_at: 2018-04-20T05:05:11Z
updated_at: 2018-04-26T04:04:25Z
closed_at: 2018-04-26T04:04:25Z
author_association: OWNER
type: issue
state_reason: completed
body:
  It appears that if the two settings are the same Datasette fails to return any results, probably because of the trick where we try to fetch 1001 rows so we know if there's a next page.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/230/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMTYzMjMzMzY=
title: metadata.json support for plugin configuration options
state: closed
created_at: 2018-04-20T15:58:47Z
updated_at: 2019-05-13T18:56:21Z
closed_at: 2019-05-13T18:56:21Z
author_association: OWNER
type: issue
state_reason: completed
body:
  My [datasette-cluster-map](https://github.com/simonw/datasette-cluster-map) plugin currently works by detecting `latitude` and `longitude` columns. I'd like to be able to configure it to look for different column names.

  One way to do this could be to support optional plugin configuration as part of `metadata.json`. Something like this:

      {
          "title": "Polar Bear Ear Tags, 2009-2011",
          "source": "USGS Alaska Science Center, Polar Bear Research Program",
          "source_url": "https://alaska.usgs.gov/products/data.php?dataid=130",
          "plugins": {
              "datasette_cluster_map": {
                  "latitude_columns": [
                      "latitude",
                      "Capture Latitude"
                  ],
                  "longitude_columns": [
                      "longitude",
                      "Capture Longitude"
                  ]
              }
          }
      }

  These settings should be supported at the root level or at the individual database or table level.

  They could also be exposed in the https://datasette-cluster-map-demo.now.sh/-/plugins debug tool.

  Refs #14

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/231/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MTgzMTM1NjA0
title: Fix a typo
state: closed
created_at: 2018-04-20T18:20:04Z
updated_at: 2018-04-21T00:19:08Z
closed_at: 2018-04-21T00:19:08Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/232
type: pull
body:
  It looks like this was the only instance of it: https://github.com/simonw/datasette/search?utf8=%E2%9C%93&q=SOLite&type=

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/232/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMTY0NDQ3MjA=
title: Option to expose expanded foreign keys in JSON/CSV
state: closed
created_at: 2018-04-21T00:18:25Z
updated_at: 2018-06-16T22:26:21Z
closed_at: 2018-06-16T22:20:14Z
author_association: OWNER
type: issue
state_reason: completed
body:
  https://datasette-cluster-map-demo.datasettes.com/sf-trees-02c8ef1/Street_Tree_List?qCareAssistant=1

  ![f36b87c0-478e-4d55-9a5f-ad37df0b47cb](https://user-images.githubusercontent.com/9599/39078411-bb3e4f88-44be-11e8-9d0c-d22324793c77.png)

  It would be nice if the info bubbles there could expose more than just the IDs, and if the title showed the expanded name of the selected qCareAssistant.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/233/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMTY1MjY0MzM=
title: label_column option in metadata.json
state: closed
created_at: 2018-04-21T21:19:08Z
updated_at: 2018-04-22T20:47:12Z
closed_at: 2018-04-22T20:47:12Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Currently the column used for displaying a foreign key relationship is automatically detected by `inspect()` by looking for tables that have a primary key column and one other column.

  This doesn't work for tables with more than two columns.

  Let's allow the table section in `metadata.json` to optionally define a `label_column` which, if present, will be used for those displays.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/234/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMTY2MjExMDI=
title: Add limit on the size in KB of data returned from a single query
state: open
created_at: 2018-04-22T23:01:15Z
updated_at: 2018-04-24T00:30:02Z
author_association: OWNER
type: issue
body:
  Datasette limits the number of rows returned to 1,000 and limits the time spent executing a SQL query to 1000ms - and both of these limits can be customized.

  It does not have a limit on the size of the response returned. It's possible to compose maliciously large SQL responses in a small number of rows using mechanisms like the `group_concat()` aggregate function. It would be good to avoid malicious SQL creating 100MB+ responses and potentially crashing the server.

  I think the easiest place to implement that is here:

  https://github.com/simonw/datasette/blob/f3f42957128c1e7ece584d45d9167f2ac003a3b8/datasette/app.py#L175-L190

  Currently we use `cursor.fetchmany()` to fetch up to 1,001 rows at once. Instead, we could switch to iterating through `cursor.fetchone()` (or just using `for row in cursor`) and keeping a running tally of the size of the response as we go - maybe just using `rough_response_size += len(str(row))`. If that goes above a certain threshold we can terminate the response with an error, like we do with timelimits.

  The bigger challenge here is understanding how well this approach works and what impact it will have on overall Datasette performance. I think I need #33 for this.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/235/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMTcwMDE1MDA=
title: datasette publish lambda plugin
state: open
created_at: 2018-04-23T22:10:30Z
updated_at: 2023-03-12T14:04:15Z
author_association: OWNER
type: issue
body:
  Refs #217 - create a publish plugin that can deploy to AWS Lambda.

  https://docs.aws.amazon.com/lambda/latest/dg/limits.html says lambda packages can be up to 50 MB, so this would only work with smaller databases (the command can check the filesize before attempting to package and deploy it).

  Lambdas do get a 512 MB `/tmp` directory too, so for larger databases the function could start and then download up to 512MB from an S3 bucket - so the plugin could take an optional S3 bucket to write to and know how to upload the `.db` file there and then have the lambda download it on startup.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/236/reactions", "total_count": 2, "+1": 2, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMTc0NzUxNTY=
title: Support for ?_search_colname=blah searches
state: closed
created_at: 2018-04-25T04:29:53Z
updated_at: 2018-05-05T22:56:42Z
closed_at: 2018-05-05T22:33:23Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Right now the `_search=` argument searches across all fields in a full-text index, for example:

  https://san-francisco.datasettes.com/sf-film-locations-84594a7/Film_Locations_in_San_Francisco?_search=justin

  SQLite FTS also supports searches within a specified field, for example:

  https://san-francisco.datasettes.com/sf-film-locations-84594a7?sql=select+rowid%2C+*+from+Film_Locations_in_San_Francisco+where+rowid+in+%28select+rowid+from+%5BFilm_Locations_in_San_Francisco_fts%5D+where+%5BLocations%5D+match+%3Asearch%29+order+by+rowid+limit+101&search=justin

  ```
  select rowid, * from Film_Locations_in_San_Francisco
  where rowid in (
    select rowid from [Film_Locations_in_San_Francisco_fts]
    where [Locations] match :search
  ) order by rowid limit 101
  ```

  The `_search=` parameter could be extended to support this using `_search_colname=`.

  This should also be able to support columns with spaces and special characters in their names, something like this:

  `_search_Column%20With%20Spaces=foo`

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/237/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMTc3MTQyNjg=
title: External metadata.json
state: closed
created_at: 2018-04-25T17:02:30Z
updated_at: 2019-06-24T06:52:55Z
closed_at: 2019-06-24T06:52:45Z
author_association: OWNER
type: issue
state_reason: completed
body:
  A frustration I'm having with https://register-of-members-interests.datasettes.com/ is that I keep coming up with new canned queries but I don't want to redeploy the whole thing just to add them to `metadata.json`

  Maybe Datasette could optionally take a `--metadata-url` option which causes it to load from a URL instead and occasionally check for updates.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/238/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMTc3NjAzNjE=
title: Support for hidden tables in metadata.json
state: closed
created_at: 2018-04-25T19:21:17Z
updated_at: 2018-04-26T03:45:12Z
closed_at: 2018-04-26T03:43:10Z
author_association: OWNER
body: Since we already have a hidden feature, let's expose it more to our users
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/239/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMTc5MDA1ODc=
title: FTS table detection should be part of .inspect()
state: closed
created_at: 2018-04-26T06:58:10Z
updated_at: 2018-04-29T00:04:44Z
closed_at: 2018-04-29T00:04:44Z
author_association: OWNER
type: issue
state_reason: completed
body:
  The code that detects if specific tables have a corresponding FTS column is currently called from TableView - it should instead be handled as part of `.inspect()`. This will make it easier to build other features that need to behave differently depending on whether a table can be searched, e.g. an autocomplete widget for selecting filters from foreign key tables.

  Current code:

  https://github.com/simonw/datasette/blob/f188ceaa2a3a5b2eab83425ad0f00cb0d364e24a/datasette/app.py#L728-L733

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/240/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMTg0OTAxMzM=
title: Default datasette logging format should be JSON
state: open
created_at: 2018-04-27T17:32:48Z
updated_at: 2018-07-10T17:45:40Z
author_association: OWNER
type: issue
body:
  Structured logs are better. Datasette should default to outputting it's HTTP access log lines as newline delimited JSON instead of the Sanic default format it uses at the moment.

  For improved greppability these logs should have keys ordered in a consistent way. Python's JSON module can do this with ordered dictionaries.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/241/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMTg2OTI5NTM=
title: Rename ?_sql_time_limit_ms= to ?_timelimit=
state: closed
created_at: 2018-04-29T06:11:35Z
updated_at: 2018-05-02T00:20:42Z
closed_at: 2018-05-02T00:20:42Z
author_association: OWNER
body: It's a bit of a mouthful at the moment.
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/242/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMTg3Mzc4MDg=
title: --spatialite option for datasette publish commands
state: closed
created_at: 2018-04-29T18:19:32Z
updated_at: 2018-05-31T14:17:53Z
closed_at: 2018-05-31T14:17:53Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Performs the necessary incantations to install Spatialite on Zeit Now or Heroku and sets the corresponding environment variable to ensure the module is correctly loaded by datasette serve.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/243/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMTg3MzgwMDA=
title: /-/versions page
state: closed
created_at: 2018-04-29T18:22:15Z
updated_at: 2018-05-03T14:13:49Z
closed_at: 2018-05-03T14:09:53Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Displays the current version of:

  * datasette
  * Python
  * SQLite
  * Spatialite (if available)

  Installed plugin versions should be shown on /-/plugins

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/244/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMTkzNTgyMDA=
title: ?_shape=array option
state: closed
created_at: 2018-05-01T23:11:07Z
updated_at: 2018-05-03T14:14:33Z
closed_at: 2018-05-02T00:12:20Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Some tools (`pandas.DataFrame(...)` for example) are happiest when you give them a raw array of JSON objects.

  `?_shape=array` should do just that

  While I'm at it, rename the default `?_shape=lists` to instead be called `?shape=arrays`

  And validate that `_shape` is a valid option

  And have `?_shape=object` return the object at the root level rather than nested in `.rows` to better match the behavior of `?_shape=array`

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/245/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MTg1MzA3NDA3
title: ?_shape=array and _timelimit=
state: closed
created_at: 2018-05-02T00:18:54Z
updated_at: 2018-05-02T00:20:41Z
closed_at: 2018-05-02T00:20:40Z
author_association: OWNER
pull_request: simonw/datasette/pulls/246
type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/246/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMTk0NDk4NTI=
title: SQLite code decoupled from Datasette
state: open
created_at: 2018-05-02T08:03:28Z
updated_at: 2018-05-21T15:29:31Z
author_association: NONE
type: issue
body:
  I'm working on the possibility of use Datasette with other file formats that aren't SQLite, like files with [PyTables](https://github.com/PyTables/PyTables) format.

  In order to accomplish that, I've started [a fork for decoupling the code related with SQLite](https://github.com/jsancho-gpl/datasette/tree/feature/db-type-plugin) and putting it in an external connector to allow future connectors for a lot of file formats.

  It'd be nice if you could look at it and suggest improvements for a possible PR.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/247/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMTk5NTQ1NDU=
title: /-/plugins should show version of each installed plugin
state: closed
created_at: 2018-05-03T14:50:45Z
updated_at: 2018-05-04T18:25:40Z
closed_at: 2018-05-04T18:05:04Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Refs #244 

  https://stackoverflow.com/questions/20180543/how-to-check-version-of-python-modules

  ```
  >>> import pkg_resources
  >>> pkg_resources.get_distribution('datasette_cluster_map').version
  '0.4'
  ```

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/248/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMjAwOTAzMjk=
title: ?_size=max argument
state: closed
created_at: 2018-05-03T21:42:04Z
updated_at: 2018-05-04T18:26:30Z
closed_at: 2018-05-04T18:05:04Z
author_association: OWNER
body: For plugins that want to load the most data allowable, having `?_size=max` would be useful.
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/249/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMjAxMzI2ODI=
title: Setup some issue templates
state: open
created_at: 2018-05-04T01:49:07Z
updated_at: 2018-05-04T01:49:07Z
author_association: OWNER
type: issue
body:
  https://twitter.com/left_pad/status/99216385740464537

  I like the idea of using these to help people understand some of the ways I want to use issues.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/250/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMjA1OTI2NDM=
title: Explore "distinct values for column" in inspect()
state: closed
created_at: 2018-05-06T13:27:24Z
updated_at: 2018-05-14T22:47:55Z
closed_at: 2018-05-14T22:47:55Z
author_association: OWNER
type: issue
state_reason: completed
body:
  A lot of datasets have columns which have a small number of possible values in them - this one for example: https://fivethirtyeight.datasettes.com/fivethirtyeight-2628db9?sql=select+distinct+category+from+%5Binconvenient-sequel%2Fratings%5D%3B

  Detecting these could be interesting as part of `.inspect()`, since it would allow for various UI enhancements like autocomplete / select box filters for those columns.

  The problem is detecting them efficiently. `.inspect()` shouldn't spend 5 minutes churning through columns on giant tables trying to determine if they have a small collection of unique values.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/251/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMjE2MjQwMTY=
title: /-/versions should report the FTS version supported by SQLite
state: closed
created_at: 2018-05-09T15:43:47Z
updated_at: 2018-05-11T13:19:52Z
closed_at: 2018-05-11T13:19:52Z
author_association: OWNER
type: issue
state_reason: completed
body:
  I can copy this function from `csvs-to-sqlite`: https://github.com/simonw/csvs-to-sqlite/blob/dccbf65b37bc9eed50e9edb80a42f257e93edb1f/csvs_to_sqlite/utils.py#L283-L293

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/252/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMjE2MzEwMjA=
title: Documentation explaining how to use SQLite FTS with Datasette
state: closed
created_at: 2018-05-09T16:02:08Z
updated_at: 2018-05-12T12:09:02Z
closed_at: 2018-05-12T12:06:51Z
author_association: OWNER
type: issue
state_reason: completed
body:
  In particular how to work with https://www.sqlite.org/fts3.html#_external_content_fts4_tables_ - which Datasette can automatically detect and use to add a search UI to your page.

  Examples of basic search setup like this:

  ```
  CREATE VIRTUAL TABLE "interests_fts"
      USING FTS4 (name, content="interests");
  INSERT INTO "interests_fts" (rowid, name)
      SELECT rowid, name FROM interests;
  ```

  And complex join-based search setup like this:

  ```
  CREATE VIRTUAL TABLE "interests_fts"
      USING FTS4 (name, category, member, content="interests");
  INSERT INTO "interests_fts" (rowid, name, category, member)
      SELECT interests.rowid, interests.name, interest_categories.name, members.name
      FROM interests
          JOIN interest_categories ON interests.category_id = interest_categories.id
          JOIN members ON interests.member_id = members.id;
  ```

  Also mention how `csvs-to-sqlite` can be used to do this easily.

  This will benefit from #252 

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/253/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMjIyODMwNjc=
title: Escaping named parameters in canned queries
state: closed
created_at: 2018-05-11T12:43:30Z
updated_at: 2020-05-10T14:54:14Z
closed_at: 2020-05-10T14:54:13Z
author_association: NONE
type: issue
state_reason: completed
body:
  Thank you very much for this project.

  I have created some canned queries but some of the filters include a colon eg. "com.ubuntu.cloud:server:18.04:amd64". When saved these colons are parsed as named parameters. 

  Is there a way to escape colons in a canned query?

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/254/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMjI0NzcxODc=
title: Facets
state: closed
created_at: 2018-05-12T03:00:07Z
updated_at: 2019-05-29T21:39:12Z
closed_at: 2018-05-16T15:32:12Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Ability to display facets and facet counts on the table view.

  Facets can be specified in the URL with `?_facet=column&_facet=othercolumn` or the default facets for a table can be set using a new `"facets": [...]` property in `metadata.json`

  - [x] Implement `?_facet=`
  - [x] Implement `metadata.json` `facets` key
  - [x] Design for how facets should be presented
  - [x] Facets should be able to toggle off as well as on
  - [x] Expand labels for facets that are foreign keys
  - [x] Suggest potential facets (if we can do so within a tight time limit)
  - [x] Documentation

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/255/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMjI1NTE3MjM=
title: Break up app.py into separate view modules
state: closed
created_at: 2018-05-12T23:56:33Z
updated_at: 2018-05-14T03:05:37Z
closed_at: 2018-05-14T03:05:37Z
author_association: OWNER
type: issue
state_reason: completed
body:
  `views/table.py` and `views/database.py` and `views/utils.py` as a starting point.

  Likewise, create `tests/test_views_table.py` and `tests/test_views_database.py` - these will contain both HTML and API test for those views.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/256/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MTg3NjY4ODkw
title: Refactor views
state: closed
created_at: 2018-05-13T13:00:50Z
updated_at: 2018-05-14T03:04:25Z
closed_at: 2018-05-14T03:04:24Z
author_association: OWNER
pull_request: simonw/datasette/pulls/257
type: pull
body:
  * Split out view classes from main `app.py`
  * Run [black](https://github.com/ambv/black) against resulting code to apply opinionated source code formatting
  * Run [isort](https://github.com/timothycrosley/isort) to re-order my imports

  Refs #256 

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/257/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MTg3NzcwMzQ1
title: Add new metadata key persistent_urls which removes the hash from all database urls
state: closed
created_at: 2018-05-14T09:39:18Z
updated_at: 2018-05-21T07:38:15Z
closed_at: 2018-05-21T07:38:15Z
author_association: NONE
pull_request: simonw/datasette/pulls/258
type: pull
body:
  Add new metadata key "persistent_urls" which removes the hash from all database urls when set to "true"

  This PR is just to gauge if this, or something like it, is something you would consider merging?

  I understand the reason why the substring of the hash is included in the url but
  there are some use cases where the urls should persist across deployments. For bookmarks
  for example or for scripts that use the JSON API.

  This is the initial commit for this feature. Tests and documentation updates to follow.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/258/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMjI3ODc0NzA=
title: inspect() should detect many-to-many relationships
state: closed
created_at: 2018-05-14T12:03:58Z
updated_at: 2019-05-23T03:55:37Z
closed_at: 2019-05-23T03:55:37Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Relates to #255 - in particular supporting facets across M2M relationships.

  It should be possible for `.inspect()` to notice when a table has two foreign keys to two different tables, and assume that this means there is a M2M relationship between those tables.

  When rendering a table with a m2m relationship we could display the first X associated records as a comma separated list of hyperlinks in a new column on the table view, with a column name derived from the table on the other side.

  Since SQLite doesn't have RANK or an equivalent of https://www.xaprb.com/blog/2006/12/02/how-to-number-rows-in-mysql/ this would be implemented as N+1 queries (one query per cell that we want to display an m2m summary). This should be OK in SQLite: https://sqlite.org/np1queryprob.html

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/259/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMjMyMjM4NzI=
title: Validate metadata.json on startup
state: open
created_at: 2018-05-15T13:42:56Z
updated_at: 2023-06-21T12:51:22Z
author_association: OWNER
type: issue
body:
  It's easy to misspell the name of a database or table and then be puzzled when the metadata settings silently fail.

  To avoid this, let's sanity check the provided metadata.json on startup and quit with a useful error message if we find any obvious mistakes.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/260/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MTg4MzEyNDEx
title: Facets improvements plus suggested facets
state: closed
created_at: 2018-05-16T03:52:39Z
updated_at: 2018-05-16T15:27:26Z
closed_at: 2018-05-16T15:27:25Z
author_association: OWNER
pull_request: simonw/datasette/pulls/261
body: Refs #255
type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/261/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMjM2NTg2NDE=
title: Add ?_extra= mechanism for requesting extra properties in JSON
state: open
created_at: 2018-05-16T14:55:42Z
updated_at: 2023-03-29T06:22:22Z
author_association: OWNER
type: issue
body:
  Datasette views currently work by creating a set of data that should be returned as JSON, then defining an additional, optional `template_data()` function which is called if the view is being rendered as HTML.

  This `template_data()` function calculates extra template context variables which are necessary for the HTML view but should not be included in the JSON.

  Example of how that is used today: https://github.com/simonw/datasette/blob/2b79f2bdeb1efa86e0756e741292d625f91cb93d/datasette/views/table.py#L672-L704

  With features like Facets in #255 I'm beginning to want to move more items into the `template_data()` - in the case of facets it's the `suggested_facets` array. This saves that feature from being calculated (involving several SQL queries) for the JSON case where it is unlikely to be used.

  But... as an API user, I want to still optionally be able to access that information.

  Solution: Add a `?_extra=suggested_facets&_extra=table_metadata` argument which can be used to optionally request additional blocks to be added to the JSON API.

  Then redefine as many of the current `template_data()` features as extra arguments instead, and teach Datasette to return certain extras by default when rendering templates.

  This could allow the JSON representation to be slimmed down further (removing e.g. the `table_definition` and `view_definition` keys) while still making that information available to API users who need it.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/262/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMjM2NzE1Nzc=
title: Facets should not execute for ?shape=array|object
state: closed
created_at: 2018-05-16T15:26:13Z
updated_at: 2021-06-02T02:54:34Z
closed_at: 2021-06-02T02:54:34Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Split off from #255 - there's no point executing the facet SQL for the `?_shape=array` and `?_shape=object` API responses.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/263/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMjM2NzM4OTk=
title: Make it possible to customize various facet settings
state: closed
created_at: 2018-05-16T15:31:34Z
updated_at: 2018-05-18T06:18:00Z
closed_at: 2018-05-18T05:11:52Z
author_association: OWNER
type: issue
state_reason: completed
body:
  The new Facets implementation from #255 includes several hard-coded settings which should be made configurable somehow:

  Number of rows to return in a facet (maybe this should also be an option that can be set via quersytring argument, e.g. `?_facet=qSpecies:40`):

  https://github.com/simonw/datasette/blob/9959a9e4deec8e3e178f919e8b494214d5faa7fd/datasette/views/table.py#L539

  Time limit for executing a facet:

  https://github.com/simonw/datasette/blob/9959a9e4deec8e3e178f919e8b494214d5faa7fd/datasette/views/table.py#L559-L562

  Maximum unique values returned in order for a column to be suggested as a facet:

  https://github.com/simonw/datasette/blob/9959a9e4deec8e3e178f919e8b494214d5faa7fd/datasette/views/table.py#L646-L647

  Time limit for calculating if a column should be a suggested facet:

  https://github.com/simonw/datasette/blob/9959a9e4deec8e3e178f919e8b494214d5faa7fd/datasette/views/table.py#L664-L667

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/264/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMjM2Nzc0OTk=
title: Add links to example Datasette instances to appropiate places in docs
state: closed
created_at: 2018-05-16T15:40:20Z
updated_at: 2018-06-18T15:52:15Z
closed_at: 2018-06-18T15:52:15Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Links to working examples would really help, especially on these pages:

  * http://datasette.readthedocs.io/en/latest/json_api.html
  * http://datasette.readthedocs.io/en/latest/sql_queries.html
  * http://datasette.readthedocs.io/en/latest/facets.html
  * http://datasette.readthedocs.io/en/latest/full_text_search.html

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/265/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMjM2ODE1ODk=
title: Export to CSV
state: closed
created_at: 2018-05-16T15:50:24Z
updated_at: 2021-06-17T18:14:24Z
closed_at: 2018-06-18T06:05:25Z
author_association: OWNER
body: Datasette needs to be able to export data to CSV.
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/266/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMjM3MTY0MTE=
title: Documentation for URL hashing, redirects and cache policy
state: closed
created_at: 2018-05-16T17:29:01Z
updated_at: 2019-06-24T06:41:02Z
closed_at: 2019-06-24T06:41:02Z
author_association: OWNER
body: See my comments on #258 for a starting point
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/267/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMjM3MTg4NDI=
title: Mechanism for ranking results from SQLite full-text search
state: open
created_at: 2018-05-16T17:36:40Z
updated_at: 2022-01-13T22:21:28Z
author_association: OWNER
type: issue
body:
  This isn't particularly straight-forward - all the more reason for Datasette to implement it for you. This article is helpful: http://charlesleifer.com/blog/using-sqlite-full-text-search-with-python/

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/268/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMjM3MjY4ODg=
title: If a facet fails due to timing out, let the user know somehow
state: closed
created_at: 2018-05-16T18:01:47Z
updated_at: 2018-05-18T06:11:46Z
closed_at: 2018-05-18T06:11:46Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Refs #255 - right now facets fail silently if the user requested them but they take longer than 200ms to calculate - see also #264

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/269/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMjM4MzAwNTE=
title: --limit= CLI option for setting limits
state: closed
created_at: 2018-05-17T00:14:24Z
updated_at: 2018-05-18T06:19:31Z
closed_at: 2018-05-18T06:16:39Z
author_association: OWNER
type: issue
state_reason: completed
body:
  #264 calls for four new datasette limit options, on top of the two existing ones:

  * `--max_returned_rows`
  * `--sql_time_limit_ms`

  These are already clogging up `datasette serve --help` a bit.

  How about this syntax instead?

      datasette --limit max_returned_rows:100 \
          --limit facet_timeout_ms:500 demo.db

  Then we can add as many new user over-rideable limits as we like without clogging up `--help` too much - though it would be good to have a way of optionally listings their documentation as well.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/270/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMjQxNjI0NzY=
title: Mechanism for automatically picking up changes when on-disk .db file changes
state: closed
created_at: 2018-05-17T19:53:15Z
updated_at: 2019-01-10T21:35:18Z
closed_at: 2019-01-10T21:35:18Z
author_association: OWNER
type: issue
state_reason: completed
body:
  It would be useful if Datasette could spot when a SQLite database file changes on disk and restart itself (hence re-running .inspect() and picking up the new content hash). Ideally this could happen in an atomic way so no requests get dropped during the switch-over.

  This may not play well with SQLite opening databases in immutable mode. Research required.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/271/reactions", "total_count": 2, "+1": 2, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMjQxODg5NTM=
title: Port Datasette to ASGI
state: closed
created_at: 2018-05-17T21:16:32Z
updated_at: 2019-06-24T04:54:15Z
closed_at: 2019-06-24T03:33:06Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Datasette doesn't take much advantage of Sanic, and I'm increasingly having to work around parts of it because of idiosyncrasies that are specific to Datasette - caring about the exact order of querystring arguments for example.

  Since Datasette is GET-only our needs from a web framework are actually pretty slim.

  This becomes more important as I expand the plugins #14 framework. Am I sure I want the plugin ecosystem to depend on a Sanic if I might move away from it in the future?

  If Datasette wasn't all about async/await I would use WSGI, but today it makes more sense to use ASGI. I'd like to be confident that switching to ASGI would still give me the excellent performance that Sanic provides.

  https://github.com/django/asgiref/blob/master/specs/asgi.rst

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/272/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMjQ0NTEzMjI=
title: Figure out a way to have /-/version return current git commit hash
state: closed
created_at: 2018-05-18T15:16:56Z
updated_at: 2018-05-22T19:35:22Z
closed_at: 2018-05-22T19:35:22Z
author_association: OWNER
type: issue
state_reason: completed
body:
  https://fivethirtyeight.datasettes.com/-/versions reports Datasette version `0.21`

  This isn't actually correct. The deploy script for that site actually deploys current master using `https://github.com/simonw/datasette/archive/master.zip`: https://github.com/simonw/fivethirtyeight-datasette/blob/66b4b0dfedd7237bc8c02d3e26d905bca7b84069/Dockerfile#L9

  Ideally this would show the current commit hash, but I'm not at all sure if it's possible to derive that from `pip install https://github.com/simonw/datasette/archive/master.zip`. Is there another mechanism that could be used to reliably `pip install` current master but still provide access to the most recent commit hash?

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/273/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMjQ2NTIxNDI=
title: Rename --limit to --config, add --help-config
state: closed
created_at: 2018-05-19T18:57:42Z
updated_at: 2018-05-20T17:04:55Z
closed_at: 2018-05-20T17:04:11Z
author_association: OWNER
type: issue
state_reason: completed
body:
  #270 introduced `--limit` but on further thought it should be called `--config` instead.

  `--page_size` should becomes `--config default_page_size:1000`

  Add `--help-config` to show full help showing all config settings.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/274/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMjQ3MjAwOTU=
title: "config" section in metadata.json (root, database and table level)
state: closed
created_at: 2018-05-20T16:02:28Z
updated_at: 2023-08-23T01:28:37Z
closed_at: 2023-08-23T01:28:37Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Split off from #274 

  Metadata should an optional `"config"` section at root, table or database level.

  The TableView and RowView and DatabaseView and BaseView classes could all have a `.config("key")` method which knows how to resolve the hierarchy of configs.

  This will allow individual tables (or databases) to set their own config settings for things like `sql_time_limit_ms`

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/275/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMjQ4MzU4Mzg=
title: Handle spatialite geometry columns better
state: closed
created_at: 2018-05-21T08:46:55Z
updated_at: 2022-03-21T22:22:20Z
closed_at: 2022-03-21T22:22:20Z
author_association: CONTRIBUTOR
type: issue
state_reason: completed
body:
  I'd like to see spatialite geometry columns rendered more sensibly - at the moment they come through as well-known-binary unless you use custom SQL, and WKB isn't of much use to anyone on the web.

  In HTML: they should be shown either as simple lat/long (if it's just a point, for example), or as a sensible placeholder if they're more complex geometries.

  In JSON: they should be GeoJSON geometries, (which means they can be automatically fed into a leaflet map with no further messing around).

  In CSV: they should be WKT.

  I briefly wondered if this should go into a plugin, but I suspect it needs hooking in at a deeper level than the plugin architecture will support any time soon.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/276/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MTg5MzE4NDUz
title: Refactor inspect logic
state: closed
created_at: 2018-05-21T08:49:31Z
updated_at: 2018-05-22T16:07:24Z
closed_at: 2018-05-22T14:03:07Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/277
type: pull
body:
  This pulls the logic for inspect out into a new file which makes it a bit easier to understand.

  This was going to be the first part of an implementation for #276, but it seems like that might take a while so I'm going to PR a few bits of refactoring individually.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/277/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMjUyOTQxMDI=
state: closed
created_at: 2018-05-22T13:28:40Z
updated_at: 2018-05-23T17:43:36Z
closed_at: 2018-05-23T17:43:36Z
author_association: OWNER
type: issue
state_reason: completed
title:
  Build smallest possible Docker image with Datasette plus recent SQLite (with json1) plus Spatialite 4.4.0

body:
  A Dockerfile that does the following:

  * Bundles Datasette master
  * Python 3.6 most recent version (or 3.7 if it has been released)
  * SQLite 3.23.1 (or most recent release) such that "import sqite3" in Python gets that version. Ideally with the json1 module baked in by default, but having it loadable as an optional module is fine too
  * SpatiaLite 4.4.0-RC0 (or most recent version) such that it can be loaded as an optional module
  * Uses multi-stage builds to stay as small as possible

  Note that the current "release" of SpatiaLite is 4.3.0 which is missing key features like https://www.gaia-gis.it/fossil/libspatialite/wiki?name=KNN - 4.4.0 probably needs to be compiled from source.

  I don't know the best way to get a current SQLite version bundled for Python 3. Maybe https://github.com/coleifer/pysqlite3 ?

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/278/reactions", "total_count": 2, "+1": 2, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MTg5NzA3Mzc0
title: Add version number support with Versioneer
state: closed
created_at: 2018-05-22T15:39:45Z
updated_at: 2018-05-22T19:35:23Z
closed_at: 2018-05-22T19:35:22Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/279
type: pull
body:
  I think that's all for getting Versioneer support, I've been happily using it in a couple of projects ... 

  ```
  In [2]: datasette.__version__
  Out[2]: '0.22+3.g6e12445'
  ```
  Repo:
  https://github.com/warner/python-versioneer

  Versioneer Licence:
  Public Domain (CC0-1.0)

  Closes #273

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/279/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MTg5NzIzNzE2
title: Build Dockerfile with recent Sqlite + Spatialite
state: closed
created_at: 2018-05-22T16:33:50Z
updated_at: 2018-06-28T11:26:23Z
closed_at: 2018-05-23T17:43:35Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/280
type: pull
body:
  This solves #278 without bloating the Dockerfile too much, the image size is now
  495MB (original was ~240MB) but it could be reduced significantly if we only
  copied the output of the compilation of spatialite and friends to
  /usr/local/lib, instead of the entirety of it however that will take more time.

  In the python code change references to `import sqlite3` to `import pysqlite3`
  and it should use the compiled version of sqlite3.23.1. You don't need to
  try/except because pysqlite3 falls back to builtin sqlite3 if there is no
  compiled version.

  ```bash
    $ docker run --rm -it datasette spatialite
    SpatiaLite version ..: 4.4.0-RC0	Supported Extensions:
      - 'VirtualShape'	[direct Shapefile access]
      - 'VirtualDbf'		[direct DBF access]
      - 'VirtualXL'		[direct XLS access]
      - 'VirtualText'		[direct CSV/TXT access]
      - 'VirtualNetwork'	[Dijkstra shortest path]
      - 'RTree'		[Spatial Index - R*Tree]
      - 'MbrCache'		[Spatial Index - MBR cache]
      - 'VirtualSpatialIndex'	[R*Tree metahandler]
      - 'VirtualElementary'	[ElemGeoms metahandler]
      - 'VirtualKNN'	[K-Nearest Neighbors metahandler]
      - 'VirtualXPath'	[XML Path Language - XPath]
      - 'VirtualFDO'		[FDO-OGR interoperability]
      - 'VirtualGPKG'	[OGC GeoPackage interoperability]
      - 'VirtualBBox'		[BoundingBox tables]
      - 'SpatiaLite'		[Spatial SQL - OGC]
    PROJ.4 version ......: Rel. 4.9.3, 15 August 2016
    GEOS version ........: 3.5.1-CAPI-1.9.1 r4246
    TARGET CPU ..........: x86_64-linux-gnu
    the SPATIAL_REF_SYS table already contains some row(s)
    SQLite version ......: 3.23.1
    Enter ".help" for instructions
    SQLite version 3.23.1 2018-04-10 17:39:29
    Enter ".help" for instructions
    Enter SQL statements terminated with a ";"
    spatialite>
  ```

  ```bash
  $ docker run --rm -it datasette python -c "import pysqlite3; print(pysqlite3.sqlite_version)"
  3.23.1
  ```

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/280/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MTg5ODYwMDUy
title: Reduces image size using Alpine + Multistage (re: #278)
state: closed
created_at: 2018-05-23T05:27:05Z
updated_at: 2018-05-26T02:10:38Z
closed_at: 2018-05-26T02:10:38Z
author_association: NONE
pull_request: simonw/datasette/pulls/281
type: pull
body:
  Hey Simon! 

  I got the image size down from 256MB to 110MB. 

  Seems to be working okay, but you might want to test it a bit more.

  Example output of `docker run --rm -it <my-tag> datasette`
  ```
  Serve! files=() on port 8001
  [2018-05-23 05:23:08 +0000] [1] [INFO] Goin' Fast @ http://127.0.0.1:8001
  [2018-05-23 05:23:08 +0000] [1] [INFO] Starting worker [1]
  ```

  Related: https://github.com/simonw/datasette/issues/278

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/281/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMjU3MDU5ODE=
title: Faceting breaks pagination
state: closed
created_at: 2018-05-23T13:29:47Z
updated_at: 2018-05-23T13:53:39Z
closed_at: 2018-05-23T13:42:07Z
author_association: OWNER
type: issue
state_reason: completed
body:
  e.g. on https://fivethirtyeight.datasettes.com/fivethirtyeight-5de27e3/nba-elo%2Fnbaallelo?_facet=lg_id#facet-lg_id - click the "next page" link: https://fivethirtyeight.datasettes.com/fivethirtyeight-5de27e3/nba-elo%2Fnbaallelo?_facet=lg_id&_next=100

      Invalid SQL: near "and": syntax error

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/282/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMjU5NTg1MDY=
title: Support cross-database joins
state: closed
created_at: 2018-05-24T04:18:39Z
updated_at: 2021-06-06T09:40:18Z
closed_at: 2021-02-18T22:16:46Z
author_association: OWNER
type: issue
state_reason: completed
body:
  SQLite has the ability to attach multiple databases to a single connection and then run joins across multiple databases.

  Since Datasette supports more than one database, this would make a pretty neat feature.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/283/reactions", "total_count": 2, "+1": 2, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMjYxODI4MTQ=
title: Ability to enable/disable specific features via --config
state: closed
created_at: 2018-05-24T15:47:56Z
updated_at: 2018-05-25T06:05:02Z
closed_at: 2018-05-25T05:51:09Z
author_association: OWNER
type: issue
state_reason: completed
body:
  `--config` settings from #274 can currently only be integers.

  I'd like them to be available as boooeans too. Then we can use them to have that are turned on by default but can be turned off.

  First features to get this treatment:

  - [x] `allow_sql` - whether or not the `?sql=` parameter is allowed and form is displayed
  - [X] `allow_facet` - is `?_facet=` allowed or do we only run facets defined in `metadata.json`
  - [X] `allow_download` - do we let users download the full SQLite database file?
  - [X] `suggest_facets` - do we attempt to calculate suggested facets?

  Refs #275 

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/284/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMjYxODk3NDQ=
title: num_threads and cache_max_age should be --config options
state: closed
created_at: 2018-05-24T16:04:51Z
updated_at: 2018-05-27T00:53:35Z
closed_at: 2018-05-27T00:43:33Z
author_association: OWNER
type: issue
state_reason: completed
body:
  https://github.com/simonw/datasette/blob/58b5a37dbbf13868a46bcbb284509434e66eca25/datasette/app.py#L106

  And

  https://github.com/simonw/datasette/blob/58b5a37dbbf13868a46bcbb284509434e66eca25/datasette/views/base.py#L325

  Refs #275 

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/285/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMjY1OTk1MjU=
title: Database hash should include current datasette version
state: open
created_at: 2018-05-25T17:03:42Z
updated_at: 2018-05-25T17:07:36Z
author_association: OWNER
type: issue
body:
  Right now deploying a new version of datasette doesn't invalidate existing URLs, so users may still see a cached copy of the old templates.

  We can fix this by including the current datasette version in the input to the hash function (which currently just the database file contents).

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/286/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMjY2MTc3NDQ=
title: ?_shape=arrayfirst
state: closed
created_at: 2018-05-25T18:11:03Z
updated_at: 2018-05-27T00:32:53Z
closed_at: 2018-05-27T00:32:29Z
author_association: OWNER
body: Return an array of single items (the first item in each row returned from the SQL query).
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/287/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMjY3Njc2MjY=
title: Support multiple filters of the same type
state: closed
created_at: 2018-05-26T21:13:12Z
updated_at: 2019-04-15T23:45:04Z
closed_at: 2019-04-15T23:44:26Z
author_association: OWNER
type: issue
state_reason: completed
body:
  This should work for example:

  https://fivethirtyeight.datasettes.com/fivethirtyeight-5de27e3/biopics%2Fbiopics?year_release__not=2014&year_release__not=2015

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/288/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMjY3NjgxODg=
title: ?_ttl= parameter to control caching
state: closed
created_at: 2018-05-26T21:22:55Z
updated_at: 2018-05-26T22:22:47Z
closed_at: 2018-05-26T22:17:48Z
author_association: OWNER
type: issue
state_reason: completed
body:
  This would allow clients to specify the max-age caching header that should be returned with the query.

  Most important this will allow caching to be completely urned off for specific queries using `?_ttl=0`. Sending 0 should cause a `Cache-Control: no-cache` header to be returned.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/289/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMjY3NzgxNjE=
title: Consider increasing the default for num_sql_threads (currently 3)
state: open
created_at: 2018-05-27T00:52:41Z
updated_at: 2018-05-27T00:52:41Z
author_association: OWNER
type: issue
body:
  I ran a very rough micro-benchmark on the new `num_sql_threads` config option (added in #285)

      datasette --config num_sql_threads:1 fivethirtyeight.db

  Then

      ab -n 100 -c 10 'http://127.0.0.1:8011/fivethirtyeight-2628db9/twitter-ratio%2Fsenators'

  | Number of threads | Requests/second |
  |---|---|
  | 1 | 4.57 |
  | 3 | 9.77 |
  | 10 | 13.53 |
  | 20 | 15.24 
  | 50 |  8.21 | 

  This was on my early 2018 OS X laptop. Need to benchmark in other common environments before making a decision on changing the default. That said, the default of 3 was a number I plucked out of thin air.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/290/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMjY3ODM2NzA=
title: Avoid plugins accidentally loading dependencies twice
state: closed
created_at: 2018-05-27T03:15:21Z
updated_at: 2020-09-30T20:36:12Z
closed_at: 2018-05-28T20:42:02Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Plugins that include JavaScript files risk loading the same code twice. In particular: I want to build a second plugin that uses the Leaflet mapping library (the first was [datasette-cluster-map](https://pypi.org/project/datasette-cluster-map/)). But I don't want the two plugins to load duplicate copies of Leaflet.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/291/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMjY4MDAyMTk=
title: Mechanism for customizing the SQL used to select specific columns in the table view
state: closed
created_at: 2018-05-27T09:05:52Z
updated_at: 2021-05-27T04:25:01Z
closed_at: 2021-05-27T04:25:01Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Some columns don't make a lot of sense in their default representation - binary blobs such as SpatiaLite geometries for example, or lengthy columns that really should be truncated somehow.

  We may also find that there are tables where we don't want to show all of the columns - so a mechanism to select a subset of columns would be nice.

  I think there are two features here:

  * the ability to request a subset of columns on the table view
  * the ability to override the SQL for a specific column and/or add extra columns - `AsGeoJSON(Geometry)` for example

  Both features should be available via both querystring arguments and in `metadata.json`

  The querystring argument for custom SQL should only work if `allow_sql` config is turned on.

  Refs #276

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/292/reactions", "total_count": 2, "+1": 2, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MTkwOTAxNDI5
title: Support for external database connectors
state: closed
created_at: 2018-05-28T11:02:45Z
updated_at: 2018-09-11T14:32:45Z
closed_at: 2018-09-11T14:32:45Z
author_association: FIRST_TIME_CONTRIBUTOR
pull_request: simonw/datasette/pulls/293
type: pull
body:
  I think it would be nice that Datasette could work with other file formats that aren't SQLite, like files with PyTables format. I've tried to accomplish that using external connectors published with entry points.

  These external connectors must have a structure similar to the structure [PyTables Datasette connector](https://github.com/PyTables/datasette-pytables) has.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/293/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMjczNjUxMTA=
title: inspect should record column types
state: open
created_at: 2018-05-29T15:10:41Z
updated_at: 2019-06-28T16:45:28Z
author_association: OWNER
type: issue
body:
  For each table we want to know the columns, their order and what type they are.

  I'm going to break with SQLite defaults a little on this one and allow datasette to define additional types - to start with just a `geometry` type for columns that are detected as SpatiaLite geometries.

  Possible JSON design:

      "columns": [{
          "name": "title",
          "type": "text"
      }, ...]

  Refs #276

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/294/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMjczODM3NTk=
title: Extract unit tests for inspect out to test_inspect.py
state: closed
created_at: 2018-05-29T15:55:04Z
updated_at: 2019-05-11T21:40:32Z
closed_at: 2019-05-11T21:40:32Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Right now they are bundled up as API unit tests for a relatively unimportant endpoint. They should be their own thing.

  Blocks #294

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/295/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMjczOTUyNzA=
title: Per-database and per-table /-/ URL namespace
state: open
created_at: 2018-05-29T16:23:13Z
updated_at: 2019-06-28T16:46:34Z
author_association: OWNER
type: issue
body:
  Initially this will be for subsets of `/-/inspect` and `/-/metadata` but it will also give us a URL namespace for future features like `/-/facet` (expanded list of a specific facet, linked to from `...`) and `/-/graph`

  To start:

  * `/dbname/-/inspect`
  * `/dbname/-/metadata`
  * `/dbname/tablename/-/inspect`
  * `/dbname/tablename/-/metadata`

  This means we will no longer allow databases or tables to have the name `"-"` - I think that's OK

  We will continue to support rows with a primary key of `"-"` at the following URL:

  * `/dbname/tablename/-`

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/296/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMjc0MjA5NDU=
title: datasette publish Dockerfile should use python:3.6-slim-stretch
state: closed
created_at: 2018-05-29T17:40:08Z
updated_at: 2018-05-31T14:44:37Z
closed_at: 2018-05-31T14:44:37Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Right now the Dockerfile generated by `datasette package` and `datasette publish` uses this:

  https://github.com/simonw/datasette/blob/b0a95da96386ddf99816911e08df86178ffa9a89/datasette/utils.py#L269

  This appears to result in a SQLite version of `3.8.7.1` - https://parlgov.datasettes.com/-/versions

  ```
      "sqlite": {
          "extensions": {},
          "fts_versions": [
              "FTS4",
              "FTS3"
          ],
          "version": "3.8.7.1"
      }
  ```

  Meanwhile, https://fivethirtyeight.datasettes.com/-/versions is deployed with this Dockerfile https://github.com/simonw/fivethirtyeight-datasette/blob/0849901cae06e957fe04892cd4033bdcd1fcf966/Dockerfile which uses `FROM python:3.6-slim-stretch` and results in the following version report:

  ```
      "sqlite": {
          "extensions": {
              "json1": null
          },
          "fts_versions": [
              "FTS5",
              "FTS4",
              "FTS3"
          ],
          "version": "3.16.2"
      }
  ```

  So not only do we get a more recent SQLite (including https://www.sqlite.org/rowvalue.html added in 3.15) but we also get `FTS5` and `json1` as well.

  Refs #191 

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/297/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMjc0NTk4Mjk=
title: URLify URLs in results from custom SQL statements / views
state: closed
created_at: 2018-05-29T19:41:07Z
updated_at: 2018-07-24T04:53:20Z
closed_at: 2018-07-24T03:56:50Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Consider this custom query:

  https://fivethirtyeight.datasettes.com/fivethirtyeight-5de27e3?sql=select+user%2C+%28%27https%3A%2F%2Ftwitter.com%2F%27+%7C%7C+user%29+as+user_url%2C+created_at%2C+text%2C+url+from+%5Btwitter-ratio%2Fsenators%5D+limit+10%3B

  ```select user, ('https://twitter.com/' || user) as user_url, created_at, text, url from [twitter-ratio/senators] limit 10;```

  ![2018-05-29 at 12 38 pm](https://user-images.githubusercontent.com/9599/40681177-44a36d5c-633d-11e8-935b-c49dad7ac682.png)

  It would be nice if these URLs were turned into links, as happens on the table view page: https://fivethirtyeight.datasettes.com/fivethirtyeight-5de27e3/twitter-ratio%2Fsenators

  ![2018-05-29 at 12 39 pm](https://user-images.githubusercontent.com/9599/40681206-5c69c47c-633d-11e8-9f3a-08899f8659b8.png)

  This currently does not happen because the table view render logic takes a different path through `display_columns_and_rows()` which includes this bit:

  https://github.com/simonw/datasette/blob/b0a95da96386ddf99816911e08df86178ffa9a89/datasette/views/table.py#L195-L202

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/298/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMjc0NjEzODE=
title: Documentation covering ALL datasette URLs
state: closed
created_at: 2018-05-29T19:46:15Z
updated_at: 2018-07-28T04:24:05Z
closed_at: 2018-07-28T04:22:30Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Relates to #296. We need a single page of the docs listing all of the URL patterns Datasette responds to, also detailing which templates are used to render them and linking to examples of the JSON they output when called with `.json`.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/299/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMjc1NDE5NzU=
title: Hide sort select box on larger screens
state: closed
created_at: 2018-05-30T01:34:59Z
updated_at: 2018-05-31T14:43:13Z
closed_at: 2018-05-31T14:43:13Z
author_association: OWNER
type: issue
state_reason: completed
body:
  I'm larger screens you can sort by clicking column headers, so no need to show the select box (which was added for the small screen layout that doesn't show headers)

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/300/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMjgxNTU5NDY=
title: --spatialite option for "datasette publish heroku"
state: open
created_at: 2018-05-31T14:13:09Z
updated_at: 2022-01-20T21:28:50Z
author_association: OWNER
body: Split off from #243. Need to figure out how to install and configure SpatiaLite on Heroku.
type: issue
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/301/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMjgxNzE1MTM=
title: test-2.3.sqlite database filename throws a 404
state: closed
created_at: 2018-05-31T14:50:58Z
updated_at: 2018-06-21T15:21:17Z
closed_at: 2018-06-21T15:21:16Z
author_association: OWNER
type: issue
state_reason: completed
body:
  The following almost works:

      datasette test-2.3.sqlite

  http://127.0.0.1:8001test-2.3-c88bc35/HighWays loads OK, but http://127.0.0.1:8001test-2.3-c88bc35 throws a 404:

  ![2018-05-31 at 7 50 am](https://user-images.githubusercontent.com/9599/40789434-447ae934-64a7-11e8-9a07-4eeba87147d5.png)

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/302/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMjgxNzI1MjE=
title: Support table names ending with .json or .csv
state: closed
created_at: 2018-05-31T14:53:23Z
updated_at: 2018-06-15T06:55:50Z
closed_at: 2018-06-15T06:55:50Z
author_association: OWNER
type: issue
state_reason: completed
body:
  This is needed for #266 - if a table name ends with `.json` or `.csv` right now our URL pattern matching will do the wrong thing.

  We should be smarter about this. This does mean we will have some URLs that look like this:

      http://localhost:8001/dbname/weird.json - returning HTML, not JSON
      http://localhost:8001/dbname/weird.json.json - returning JSON
      http://localhost:8001/dbname/weird.json.csv - returning CSV

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/303/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMjgyMjkyMjQ=
title: Ability to configure SQLite cache_size
state: closed
created_at: 2018-05-31T17:28:07Z
updated_at: 2018-06-04T16:13:32Z
closed_at: 2018-06-04T16:03:19Z
author_association: OWNER
type: issue
state_reason: completed
body:
  See https://www.sqlite.org/pragma.html#pragma_cache_size

  Let's call the config setting `cache_size_kb` to emphasize that we're using the negative option.

  Note this warning: perhaps we should raise an error if you try to use this setting against a SQLite version prior to 3.7.10
  > If the argument N is positive then the suggested cache size is set to N. If the argument N is negative, then the number of cache pages is adjusted to use approximately abs(N*1024) bytes of memory. Backwards compatibility note: The behavior of cache_size with a negative N was different in prior to version 3.7.10 (2012-01-16). In version 3.7.9 and earlier, the number of pages in the cache was set to the absolute value of N.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/304/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMjkxNDcyODQ=
title: Add contributor guidelines to docs
state: closed
created_at: 2018-06-04T17:25:30Z
updated_at: 2019-06-24T06:40:19Z
closed_at: 2019-06-24T06:40:19Z
author_association: OWNER
body: https://channels.readthedocs.io/en/latest/contributing.html is a nice example of this done well.
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/305/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMjk2NjE5MDU=
title: Custom URL routing with independent tests
state: closed
created_at: 2018-06-05T23:40:08Z
updated_at: 2018-06-07T15:29:28Z
closed_at: 2018-06-07T15:29:28Z
author_association: OWNER
type: issue
state_reason: completed
body:
  The more I think about #303 the more I feel that Datasette's URL routing needs go beyond Django-style regex matching.

  If we go custom, tests should live in `test_routing.py`

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/306/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MTkzMzYxMzQx
title: Initial sketch of custom URL routing, refs #306
state: closed
created_at: 2018-06-07T15:26:48Z
updated_at: 2018-06-07T15:29:54Z
closed_at: 2018-06-07T15:29:41Z
author_association: OWNER
pull_request: simonw/datasette/pulls/307
body: See #306 for background on this.
type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/307/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMzA4MjY5NzI=
title: Support extra Heroku apps:create options - region, space, team
state: open
created_at: 2018-06-08T23:08:33Z
updated_at: 2018-09-21T14:09:28Z
author_association: NONE
type: issue
body:
  It would be useful to document how to pass Heroku CLI options on `datasette publish`, e.g. `--region eu`.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/308/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMzEzNDM4MjQ=
title: On 404s with a trailing slash redirect to that page without a trailing slash
state: closed
created_at: 2018-06-11T20:46:49Z
updated_at: 2018-06-21T15:22:02Z
closed_at: 2018-06-21T15:13:15Z
author_association: OWNER
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/309/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMzI4MzAzMDk=
title: datasette publish now is broken in master
state: closed
created_at: 2018-06-15T16:01:14Z
updated_at: 2018-06-16T16:29:50Z
closed_at: 2018-06-16T16:29:50Z
author_association: OWNER
type: issue
state_reason: completed
body:
  ```
  >     gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/usr/local/include/python3.6m -c httptools/parser/parser.c -o build/temp.linux-x86_64-3.6/httptools/parser/parser.o -O2
  >     unable to execute 'gcc': No such file or directory
  >     error: command 'gcc' failed with exit status 1
  >     
  >     ----------------------------------------
  > Command "/usr/local/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-s73273rj/httptools/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-yha7dxqq/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-install-s73273rj/httptools/
  ```

  Turns out the `python-slim` base image I introduced in b18e4515855c3f1eeca3dfcccdbb6df05869084a doesn't include gcc: https://github.com/docker-library/python/issues/60#issuecomment-134322383

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/310/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MTk1MzM5MTEx
title: ?_labels=1 to expand foreign keys (in csv and json), refs #233
state: closed
created_at: 2018-06-16T16:31:12Z
updated_at: 2018-06-16T22:20:31Z
closed_at: 2018-06-16T22:20:31Z
author_association: OWNER
pull_request: simonw/datasette/pulls/311
type: pull
body:
  Output looks something like this:

      {
          "rowid": 233,
          "TreeID": 121240,
          "qLegalStatus": {
              "value" 2,
              "label": "Private"
          }
          "qSpecies": {
              "value": 16,
              "label": "Sycamore"
          }
          "qAddress": "91 Commonwealth Ave",
          ...
      }

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/311/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMzMwMDAxNjM=
title: HTML, CSV and JSON views should support ?_col=&_col=
state: closed
created_at: 2018-06-16T16:53:35Z
updated_at: 2021-06-17T18:14:24Z
closed_at: 2018-06-16T17:00:12Z
author_association: OWNER
body: To support whitelisting columns to display.
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/312/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMzMwODYwMDU=
title: Deploy demo of Datasette on every commit that passes tests
state: closed
created_at: 2018-06-17T19:19:12Z
updated_at: 2018-06-17T21:52:58Z
closed_at: 2018-06-17T21:52:58Z
author_association: OWNER
type: issue
state_reason: completed
body:
  We can use Travis CI and Zeit Now to ensure there is always a live demo of current master. We can ship archived demos for releases as well.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/313/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMzMwOTYxNzY=
title: HTML table does not correctly display entirely blank rows
state: closed
created_at: 2018-06-17T21:58:06Z
updated_at: 2018-06-21T16:04:59Z
closed_at: 2018-06-21T15:26:26Z
author_association: OWNER
type: issue
state_reason: completed
body:
  https://958b75c.datasette.io/fixtures-35b6eb6/simple_view

  ![2018-06-17 at 2 56 pm](https://user-images.githubusercontent.com/9599/41512541-b52e90be-723e-11e8-95c9-7d091738d5cc.png)

  https://958b75c.datasette.io/fixtures-35b6eb6/simple_view.json shows the underlying data:

  ```
    "rows": [
      [
        "hello",
        "HELLO"
      ],
      [
        "world",
        "WORLD"
      ],
      [
        "",
        ""
      ]
    ]
  ```

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/314/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MTk1NDEzMjQx
title: Streaming mode for downloading all rows as a CSV
state: closed
created_at: 2018-06-18T03:06:59Z
updated_at: 2018-06-18T03:29:13Z
closed_at: 2018-06-18T03:21:02Z
author_association: OWNER
pull_request: simonw/datasette/pulls/315
body: Refs #266
type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/315/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMzMyMzg5MzI=
title: datasette inspect takes a very long time on large dbs
state: closed
created_at: 2018-06-18T11:56:27Z
updated_at: 2019-05-11T18:26:25Z
closed_at: 2019-05-11T18:26:25Z
author_association: NONE
type: issue
state_reason: completed
body:
  Hi,

  I want to expose data in a very large sqlite database (~600Gb) to the web.  I have used datasette with success on smaller test databases with the same schema - it works very well (thanks!).  However, using the full db, both `datasette inspect` and `datasette serve` seem to hang or pause for a very long time (tens of minutes) on startup.  Is this expected behaviour?

  (I noticed that the output of `datasette inspect` includes row counts for each table.   Simply counting the rows in this db will take a long time (tens of millions of rows across each of ~10 tables), so I wondered if this is the source of the problem.)

  Any help on a workaround would be appreciated.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/316/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMzMzMjYxMDc=
title: Travis CI fails to upload new releases to PyPI
state: closed
created_at: 2018-06-18T15:44:26Z
updated_at: 2018-06-21T15:45:47Z
closed_at: 2018-06-21T15:45:47Z
author_association: OWNER
type: issue
state_reason: completed
body:
  https://travis-ci.org/simonw/datasette/jobs/393684139

  ```
  ...
  removing build/bdist.linux-x86_64/wheel
  Uploading distributions to https://upload.pypi.org/legacy/
  Uploading datasette-0.23-py3-none-any.whl
  100%|██████████| 201k/201k [00:00<00:00, 1.02MB/s]
  HTTPError: 403 Client Error: Invalid or non-existent authentication information. for url: https://upload.pypi.org/legacy/
  ```

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/317/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMzQxNDg2Njk=
title: Facets with value of 0 displayed incorrectly
state: closed
created_at: 2018-06-20T16:06:46Z
updated_at: 2019-05-29T21:39:12Z
closed_at: 2018-06-21T04:30:45Z
author_association: OWNER
type: issue
state_reason: completed
body:
  https://registry.datasette.io/registry-7d4f81f/tables?_facet=is_hidden#facet-is_hidden

  ![2018-06-20 at 9 05 am](https://user-images.githubusercontent.com/9599/41670448-2c06e642-7469-11e8-86be-4664269582b1.png)

  Displays correctly if you select it:

  https://registry.datasette.io/registry-7d4f81f/tables?_facet=is_hidden&is_hidden=0

  ![2018-06-20 at 9 06 am](https://user-images.githubusercontent.com/9599/41670471-3e61e486-7469-11e8-8710-5da90ef65787.png)

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/318/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMzQxNDk3MTc=
title: Incorrect display of compound primary keys with foreign key relationships
state: closed
created_at: 2018-06-20T16:09:36Z
updated_at: 2018-06-21T15:58:15Z
closed_at: 2018-06-21T14:56:41Z
author_association: OWNER
type: issue
state_reason: completed
body:
  https://registry.datasette.io/registry-7d4f81f/datasette_tags

  ![2018-06-20 at 9 07 am](https://user-images.githubusercontent.com/9599/41670542-68cc4dec-7469-11e8-9521-3bbc6465eccb.png)

  Underlying JSON looks [like this](https://registry.datasette.io/registry-7d4f81f/datasette_tags.json?_labels=on):

  ```
  {
      "database": "registry",
      "table": "datasette_tags",
      "is_view": false,
      "human_description_en": "",
      "rows": [
          {
              "datasette_id": {
                  "value": 1,
                  "label": "Global Power Plant Database"
              },
              "tag": {
                  "value": "geospatial",
                  "label": "geospatial"
              }
          },
  ````

  Bug is likely somewhere in here: https://github.com/simonw/datasette/blob/e04f5b0d348ef7275a0a5ab9eb53527105132885/datasette/views/table.py#L143-L207

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/319/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMzQxNjk5MzI=
title: Need unit tests covering the different states for the advanced export box
state: closed
created_at: 2018-06-20T17:03:40Z
updated_at: 2018-07-24T04:53:20Z
closed_at: 2018-07-24T03:38:40Z
author_association: OWNER
type: issue
state_reason: completed
body:
  There are quite a few variants of this box:

  ![2018-06-20 at 10 02 am](https://user-images.githubusercontent.com/9599/41673229-1d423adc-7471-11e8-99d4-4251f7d03aa5.png)

  Test coverage should exercise all of them, since the logic is a little unclear.

  https://github.com/simonw/datasette/blob/fdfbbbb9ee0d02fd4d43dfc42382252fa2287d6d/datasette/templates/table.html#L140-L159

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/320/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMzQxOTA5NTk=
title: Wildcard support in query parameters
state: closed
created_at: 2018-06-20T18:03:56Z
updated_at: 2018-06-21T17:00:10Z
closed_at: 2018-06-21T04:55:26Z
author_association: NONE
type: issue
state_reason: completed
body:
  I haven't found a way to get the wildcard (%) inserted automatically in to a query parameter.  This would be useful for cases the query parameter is followed by a LIKE clause.  Wrapping the parameter name using the wildcard character within the metadata file (ie - ...where xyz like %:querystring%) does not seem to work.  Can this be made possible?  Or if not, can the template be extended to provide a tip to the user that they need to insert the wildcard characters themselves?

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/321/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MTk2NTI2ODYx
title: Feature/in operator
state: closed
created_at: 2018-06-21T17:41:51Z
updated_at: 2018-06-21T17:45:25Z
closed_at: 2018-06-21T17:45:25Z
author_association: NONE
pull_request: simonw/datasette/pulls/322
type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/322/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMzQ2OTg5Njk=
title: Speed up Travis CI builds
state: closed
created_at: 2018-06-21T23:55:27Z
updated_at: 2018-07-10T15:03:37Z
closed_at: 2018-07-10T15:03:36Z
author_association: OWNER
type: issue
state_reason: completed
body:
  They've got a bit slow. Part of this is the Zeit Now deploy, but the build-and-test cycle is taking at least a couple of minutes.

  ![2018-06-21 at 4 54 pm](https://user-images.githubusercontent.com/9599/41751010-e48c823e-7573-11e8-88f3-7aa8a7e53917.png)

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/323/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MTk2NjI4MzA0
title: Speed up Travis by reusing pip wheel cache across builds
state: closed
created_at: 2018-06-22T03:20:08Z
updated_at: 2018-06-24T01:03:47Z
closed_at: 2018-06-24T01:03:47Z
author_association: OWNER
pull_request: simonw/datasette/pulls/324
body: From https://atchai.com/blog/faster-ci/ - refs #323
type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/324/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMzUwNjQ3Nzc=
title: Error on row page if table has slashes in the name and ends in .csv
state: closed
created_at: 2018-06-23T03:43:42Z
updated_at: 2018-07-09T17:28:27Z
closed_at: 2018-07-08T05:21:59Z
author_association: OWNER
type: issue
state_reason: completed
body:
  https://v0-23-1.datasette.io/fixtures-e14e080/table%252Fwith%252Fslashes.csv/3

  > no such table: table%252Fwith%252Fslashes.csv

  From clicking the row link on https://v0-23-1.datasette.io/fixtures-e14e080/table%2Fwith%2Fslashes.csv

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/325/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMzUxNDE0MzQ=
title: CSV should respect --cors and return cors headers
state: closed
created_at: 2018-06-24T00:44:07Z
updated_at: 2021-06-17T18:14:24Z
closed_at: 2018-06-24T00:59:45Z
author_association: OWNER
body: Otherwise tools like Vega can't load data via CSV.
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/326/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMzUyMDAxMzY=
title: Explore if SquashFS can be used to shrink size of packaged Docker containers
state: open
created_at: 2018-06-24T18:15:16Z
updated_at: 2022-02-17T23:37:24Z
author_association: OWNER
type: issue
body:
  Inspired by this article: https://cldellow.com/2018/06/22/sqlite-parquet-vtable.html#sqlite-database-indexed--squashed

  https://en.wikipedia.org/wiki/SquashFS is "a compressed read-only file system for Linux" - which means it could be a really nice fit for Datasette and its read-only SQLite databases.

  It would be interesting to explore a Dockerfile recipe that used SquashFS to compress the SQLite database file that was bundled up by `datasette package` and friends.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/327/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMzY0NjQ3MzM=
title: Installation instructions, including how to use the docker image
state: closed
created_at: 2018-06-28T03:59:33Z
updated_at: 2023-09-05T14:10:39Z
closed_at: 2018-06-28T04:02:10Z
author_association: OWNER
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/328/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMzY0NjUwMTg=
title: Travis should push tagged images to Docker Hub for each release
state: closed
created_at: 2018-06-28T04:01:31Z
updated_at: 2018-11-05T06:54:10Z
closed_at: 2018-11-05T06:53:28Z
author_association: OWNER
body: https://sebest.github.io/post/using-travis-ci-to-build-docker-images/
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/329/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMzY5MjQxOTk=
title: Limit text display in cells containing large amounts of text
state: closed
created_at: 2018-06-29T09:15:22Z
updated_at: 2018-07-24T04:53:20Z
closed_at: 2018-07-10T16:20:48Z
author_association: CONTRIBUTOR
type: issue
state_reason: completed
body:
  The default preview of a database shows all columns (is the row count limited?) which is fine in many cases but can take a long time to load / offer a large overhead if the table is a SpatiaLite table containing geometry columns that include large shapefiles.

  Would it make sense to have a setting that can limit the amount of text displayed in any given cell in the table preview, or (less useful?) suppress (with notification) the display of overlong columns unless enabled by the user?

  An issue then arises if a user does want to see all the text in a cell:

    1) for a particular cell;
    2) for every cell in the table;
    3) for all cells in a particular column or columns

  (I haven't checked but what if a column contains e.g. raw image data? Does this display as raw data? Or can this be rendered in a context aware way as an image preview? I guess a custom template would be one way to do that?)

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/330/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMzY5MzYwMTA=
title: Datasette throws error when loading spatialite db without extension loaded
state: closed
created_at: 2018-06-29T09:51:14Z
updated_at: 2022-01-20T21:29:40Z
closed_at: 2018-07-10T15:13:36Z
author_association: CONTRIBUTOR
type: issue
state_reason: completed
body:
  When starting datasette on a SpatialLite database *without* loading the SpatiaLite extension (using eg `--load-extension=/usr/local/lib/mod_spatialite.dylib`) an error is thrown and the server fails to start:

  ```
  datasette -p 8003 adminboundaries.db 
  Serve! files=('adminboundaries.db',) on port 8003
  Traceback (most recent call last):
    File "/Users/ajh59/anaconda3/bin/datasette", line 11, in <module>
      sys.exit(cli())
    File "/Users/ajh59/anaconda3/lib/python3.6/site-packages/click/core.py", line 722, in __call__
      return self.main(*args, **kwargs)
    File "/Users/ajh59/anaconda3/lib/python3.6/site-packages/click/core.py", line 697, in main
      rv = self.invoke(ctx)
    File "/Users/ajh59/anaconda3/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
      return _process_result(sub_ctx.command.invoke(sub_ctx))
    File "/Users/ajh59/anaconda3/lib/python3.6/site-packages/click/core.py", line 895, in invoke
      return ctx.invoke(self.callback, **ctx.params)
    File "/Users/ajh59/anaconda3/lib/python3.6/site-packages/click/core.py", line 535, in invoke
      return callback(*args, **kwargs)
    File "/Users/ajh59/anaconda3/lib/python3.6/site-packages/datasette/cli.py", line 552, in serve
      ds.inspect()
    File "/Users/ajh59/anaconda3/lib/python3.6/site-packages/datasette/app.py", line 273, in inspect
      "tables": inspect_tables(conn, self.metadata.get("databases", {}).get(name, {}))
    File "/Users/ajh59/anaconda3/lib/python3.6/site-packages/datasette/inspect.py", line 79, in inspect_tables
      "PRAGMA table_info({});".format(escape_sqlite(table))
  sqlite3.OperationalError: no such module: VirtualSpatialIndex
  ``` 

  It would be nice to trap this and return a message saying something like:

  ```
  It looks like you're trying to load a SpatiaLite database? Make sure you load in the SpatiaLite extension when starting datasette.

  Read more: https://datasette.readthedocs.io/en/latest/spatialite.html
  ```

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/331/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMzcxNDExMDg=
title: Sanely handle Infinity/-Infinity values in JSON using ?_json_infinity=1
state: closed
created_at: 2018-06-29T21:21:27Z
updated_at: 2018-07-24T04:53:20Z
closed_at: 2018-07-24T03:08:30Z
author_association: OWNER
type: issue
state_reason: completed
body:
  It turns out if you load this CSV using `csvs-to-sqlite` you get an Infinity value in SQLite:

  ```
  name,num
  sasha,10
  terry,Inf
  cathy,0.5
  ```

  `csvs-to-sqlite infinity-bug.csv infinity-bug.db`

  I deployed this using:

  ```
  datasette publish now infinity-bug.db --name=datasette-infinity-bug --install=datasette-vega
  ```

  Datasette outputs that as `Infinity` in the JSON format, which causes JavaScript errors.

  Demo
  * https://datasette-infinity-bug.now.sh/infinity-bug-0d0224e/infinity-bug - HTML view works
  * https://datasette-infinity-bug.now.sh/infinity-bug-0d0224e/infinity-bug.json?_shape=array - this outputs the following:

  ```
  [
      {
          "rowid": 1,
          "name": "sasha",
          "num": 10.0
      },
      {
          "rowid": 2,
          "name": "terry",
          "num": Infinity
      },
      {
          "rowid": 3,
          "name": "cathy",
          "num": 0.5
      }
  ]
  ```

  But... in Firefox that gets rendered like this:

  ![2018-06-29 at 4 20 pm](https://user-images.githubusercontent.com/9599/42115408-5d30f630-7bb8-11e8-8370-c8484801c49b.png)

  And if you click the "Show charting options" button you get this error in the console:

  ```
  SyntaxError: JSON.parse: unexpected character at line 1 column 83 of the JSON data
  ```

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/332/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMzg3Njg1NTE=
title: Datasette on Zeit Now returns http URLs for facet and next links
state: closed
created_at: 2018-07-06T00:40:49Z
updated_at: 2018-07-24T04:53:20Z
closed_at: 2018-07-24T01:51:53Z
author_association: OWNER
type: issue
state_reason: completed
body:
  e.g. on https://fivethirtyeight.datasettes.com/fivethirtyeight-ac35616/nba-elo%2Fnbaallelo.json?_facet=lg_id&_size=0
  ```
  {
    "facet_results": {
      "lg_id": {
        "name": "lg_id",
        "results": [
          {
            "value": "NBA",
            "label": "NBA",
            "count": 118016,
            "toggle_url": "http://fivethirtyeight.datasettes.com/fivethirtyeight-ac35616/nba-elo%2Fnbaallelo.json?_facet=lg_id&_size=1&lg_id=NBA",
            "selected": false
          },
          {
            "value": "ABA",
            "label": "ABA",
            "count": 8298,
            "toggle_url": "http://fivethirtyeight.datasettes.com/fivethirtyeight-ac35616/nba-elo%2Fnbaallelo.json?_facet=lg_id&_size=1&lg_id=ABA",
            "selected": false
          }
        ],
        "truncated": false
      }
    },
    "suggested_facets": [
      {
        "name": "_iscopy",
        "toggle_url": "/fivethirtyeight-ac35616/nba-elo%2Fnbaallelo.json?_facet=lg_id&_size=1&_facet=_iscopy"
      }
    ],
    "next_url": "http://fivethirtyeight.datasettes.com/fivethirtyeight-ac35616/nba-elo%2Fnbaallelo.json?_facet=lg_id&_size=1&_next=1",
  }
  ```
  `next_url` and `facet_results` both link to `http://` when they should link to `https://`.

  Note that suggested facets doesn't include the full URL at all, which is a consistency bug.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/333/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMzkwOTU5NzY=
title: extra_options not passed to heroku publisher
state: closed
created_at: 2018-07-06T23:26:12Z
updated_at: 2018-07-24T04:53:21Z
closed_at: 2018-07-10T01:46:04Z
author_association: NONE
type: issue
state_reason: completed
body:
  I might be wrong but I was not able to publish to `heroku` with `--extra-options`, I think `extra_options` is not being used in this function [here](https://github.com/simonw/datasette/blob/master/datasette/utils.py#L369). 

  Any help appreciated! 

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/334/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzMzk1MDUyMDQ=
title: Package datasette for installation using homebrew
state: closed
created_at: 2018-07-09T15:45:03Z
updated_at: 2020-08-11T16:54:06Z
closed_at: 2020-08-11T16:54:06Z
author_association: OWNER
type: issue
state_reason: completed
body:
  https://docs.brew.sh/Python-for-Formula-Authors describes how.

  > Applications should be installed into a Python virtualenv environment rooted in libexec. This prevents the app’s Python modules from contaminating the system site-packages and vice versa.

  It recommends using https://github.com/tdsmith/homebrew-pypi-poet

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/335/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzNDAwMzk0MDk=
title: Ensure --help examples in docs are always up to date
state: closed
created_at: 2018-07-10T23:20:01Z
updated_at: 2018-07-24T16:01:29Z
closed_at: 2018-07-24T16:01:29Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Ideally I would automatically generate the --help output shown in our docs, but I don't think I can get that working with readthedocs.

  Instead, I'm going to add a unit test that checks that those extracts in the documentation match the current output of the --help command.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/336/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzNDAwNjUzNzQ=
title: Documentation for datasette publish and datasette package
state: closed
created_at: 2018-07-11T02:04:06Z
updated_at: 2018-07-11T02:07:32Z
closed_at: 2018-07-11T02:05:56Z
author_association: OWNER
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/337/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzNDAyODI3OTY=
title: Only load vegaEmbed if charting tools are enabled
state: closed
created_at: 2018-07-11T15:02:14Z
updated_at: 2018-07-11T15:21:47Z
closed_at: 2018-07-11T15:21:47Z
author_association: OWNER
type: issue
state_reason: completed
body:
  vegaEmbed is a LOT of code (it bundles d3)

  Inspired by this tweet: https://twitter.com/thelarkinn/status/1017053567641948162 - it would be great if we loaded that code on demand the first time the "Show chart options" button was clicked, or when the page loads with #g. options in the URL.

  Even better: avoid the overhead if loading React unless the chart options need to be displayed. This would be a pretty major refactoring though.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/338/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzNDAzOTYyNDc=
title: Expose SANIC_RESPONSE_TIMEOUT config option in a sensible way
state: closed
created_at: 2018-07-11T20:38:06Z
updated_at: 2022-03-21T22:22:40Z
closed_at: 2022-03-21T22:22:34Z
author_association: NONE
type: issue
state_reason: completed
body:
  Is it possible to configure the sql_time_limit_ms beyond 60 seconds?  It seems queries are still timing out at 60 seconds when sql_time_limit_ms is set to 180000.  We have a very large data set and often encounter timeouts when testing new queries from the datasette UI.  We are optimizing our database as much as we can, but still may require more than 60 seconds for complex queries.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/339/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzNDA3MzA5NjE=
title: Embrace black
state: closed
created_at: 2018-07-12T17:32:29Z
updated_at: 2019-06-24T06:50:27Z
closed_at: 2019-06-24T06:50:26Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Run [black](https://github.com/ambv/black) against everything. Then set up CI to fail if code doesn't conform to black's style.

  Here's how Starlette does this:

  * https://github.com/encode/starlette/blob/e3d090b3597167f7b3a4f76e4bb3c0d3e94be61a/.travis.yml#L14
  * https://github.com/encode/starlette/blob/e3d090b3597167f7b3a4f76e4bb3c0d3e94be61a/scripts/lint - essentially runs `black starlette tests --check`

  And here's an example of a test run that failed:  https://travis-ci.org/encode/starlette/jobs/403172478

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/340/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MjAxMDc1NTMy
title: Bump aiohttp to fix compatibility with Python 3.7
state: closed
created_at: 2018-07-12T17:41:24Z
updated_at: 2018-07-12T18:07:38Z
closed_at: 2018-07-12T18:07:38Z
author_association: OWNER
pull_request: simonw/datasette/pulls/341
body: Tests failed here: https://travis-ci.org/simonw/datasette/jobs/403223333
type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/341/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzNDExMjMzNTU=
title: Requesting support for query description
state: closed
created_at: 2018-07-13T18:50:16Z
updated_at: 2018-07-24T04:53:21Z
closed_at: 2018-07-16T02:33:54Z
author_association: NONE
type: issue
state_reason: completed
body:
  It would be great if the metadata file allowed you to enter a description for the query.  We have a lot of pre-defined queries that can only be so descriptive by their name.  It would be nice if an optional description could be included underneath the name within the UI, or on hover where it currently shows the SQL.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/342/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzNDEyMjg4NDY=
title: Render boolean fields better by default
state: open
created_at: 2018-07-14T11:10:29Z
updated_at: 2018-07-14T14:17:14Z
author_association: CONTRIBUTOR
body: These show up as 0 or 1 because sqlite. I think Yes/No would be fine in most cases?
type: issue
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/343/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzNDEyMjkxMTM=
title: datasette publish heroku fails without name provided
state: closed
created_at: 2018-07-14T11:15:56Z
updated_at: 2018-07-14T13:00:48Z
closed_at: 2018-07-14T13:00:48Z
author_association: CONTRIBUTOR
type: issue
state_reason: completed
body:
  It fails with the following JSON traceback if the `-n` option isn't provided, despite the fact that the command line help says that's not needed for heroku publishes.

  <details>

  ```
  Traceback (most recent call last):
    File "/usr/local/bin/datasette", line 11, in <module>
      sys.exit(cli())
    File "/usr/local/lib/python3.6/site-packages/click/core.py", line 722, in __call__
      return self.main(*args, **kwargs)
    File "/usr/local/lib/python3.6/site-packages/click/core.py", line 697, in main
      rv = self.invoke(ctx)
    File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
      return _process_result(sub_ctx.command.invoke(sub_ctx))
    File "/usr/local/lib/python3.6/site-packages/click/core.py", line 895, in invoke
      return ctx.invoke(self.callback, **ctx.params)
    File "/usr/local/lib/python3.6/site-packages/click/core.py", line 535, in invoke
      return callback(*args, **kwargs)
    File "/usr/local/lib/python3.6/site-packages/datasette/cli.py", line 265, in publish
      app_name = json.loads(create_output)["name"]
    File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/json/__init__.py", line 354, in loads
      return _default_decoder.decode(s)
    File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/json/decoder.py", line 339, in decode
      obj, end = self.raw_decode(s, idx=_w(s, 0).end())
    File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/json/decoder.py", line 357, in raw_decode
      raise JSONDecodeError("Expecting value", s, err.value) from None
  json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
  ```

  </details>

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/344/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MjAxNDUxMzMy
title: Allow app names for `datasette publish heroku`
state: closed
created_at: 2018-07-14T13:12:34Z
updated_at: 2018-07-14T14:09:54Z
closed_at: 2018-07-14T14:04:44Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/345
type: pull
body:
  Lets you supply the `-n` parameter for Heroku deploys, which also lets you update existing Heroku deployments.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/345/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzNDM3Mjg3NTQ=
title: Logo design for DATASETTE
state: closed
created_at: 2018-07-23T17:40:17Z
updated_at: 2018-08-02T02:31:59Z
closed_at: 2018-08-02T02:31:59Z
author_association: NONE
type: issue
state_reason: completed
body:
  Hello :) , I'm a graphic designer, I'm interested in collaborating with open source projects, besides this helps me expand my portfolio. I would like to design a logo for your project. I will be happy to collaborate with you :). 

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/346/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzNDQ2NTQ2MjM=
title: Rename "datasette package" to "datasette publish docker"
state: open
created_at: 2018-07-26T00:42:46Z
updated_at: 2018-07-26T00:42:46Z
author_association: OWNER
type: issue
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/347/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzNDQ2NTYxMTQ=
title: Unit tests for "datasette publish"
state: closed
created_at: 2018-07-26T00:52:23Z
updated_at: 2018-07-26T05:46:10Z
closed_at: 2018-07-26T05:46:10Z
author_association: OWNER
type: issue
state_reason: completed
body:
  The datasette publish family of commands all work by shelling out to heroku/now/docker

      from subprocess import call, check_output

  So in tests I should be able to mock those calls:

      @mock.patch('subprocess.call')

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/348/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MjA0MDI5MTQy
title: publish_subcommand hook + default plugins mechanism, used for publish heroku/now
state: closed
created_at: 2018-07-26T05:03:22Z
updated_at: 2018-07-26T05:28:54Z
closed_at: 2018-07-26T05:16:00Z
author_association: OWNER
pull_request: simonw/datasette/pulls/349
type: pull
body:
  This change introduces a new plugin hook, publish_subcommand, which can be
  used to implement new subcommands for the "datasette publish" command family.

  I've used this new hook to refactor out the "publish now" and "publish heroku"
  implementations into separate modules. I've also added unit tests for these
  two publishers, mocking the subprocess.call and subprocess.check_output
  functions.

  As part of this, I introduced a mechanism for loading default plugins. These
  are defined in the new "default_plugins" list inside datasette/app.py

  Closes #217 (Plugin support for "datasette publish")
  Closes #348 (Unit tests for "datasette publish")
  Refs #14, #59, #102, #103, #146, #236, #347

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/349/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzNDQ3MDE3NTU=
title: Don't list default plugins on /-/plugins
state: closed
created_at: 2018-07-26T05:38:00Z
updated_at: 2018-08-28T17:13:50Z
closed_at: 2018-08-28T16:48:19Z
author_association: OWNER
type: issue
state_reason: completed
body:
  https://dbbe707.datasette.io/-/plugins is showing "datasette.publish.now" and "datasette.publish.heroku"

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/350/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzNDU0NjkzNTU=
title: Automatically create a GitHub release linking to release notes for every tagged release
state: closed
created_at: 2018-07-28T18:31:12Z
updated_at: 2020-05-28T18:56:16Z
closed_at: 2020-05-28T18:56:15Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Can use this API called from Travis: https://developer.github.com/v3/repos/releases/#create-a-release

  The release it generates should look like this one: https://github.com/simonw/datasette/releases/tag/0.24

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/351/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzNDU4MjE1MDA=
title: render_cell(value) plugin hook
state: closed
created_at: 2018-07-30T15:56:20Z
updated_at: 2020-02-10T16:18:58Z
closed_at: 2018-08-05T00:14:57Z
author_association: OWNER
type: issue
state_reason: completed
body:
  To allow plugins to customize how values matching a specific pattern are displayed in the HTML table view.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/352/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MjA0ODUxNTEx
title: render_cell(value) plugin hook
state: closed
created_at: 2018-07-30T15:57:08Z
updated_at: 2018-08-05T00:14:57Z
closed_at: 2018-08-05T00:14:57Z
author_association: OWNER
pull_request: simonw/datasette/pulls/353
body: Closes #352.
type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/353/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzNDYwMjY4Njk=
title: Handle many-to-many relationships
state: open
created_at: 2018-07-31T04:03:13Z
updated_at: 2020-11-24T19:51:18Z
author_association: OWNER
body: This is a master tracking ticket for various many-2-many features.
type: issue
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/354/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzNDYwMjcwNDA=
title: Table view should support filtering via many-to-many relationships
state: open
created_at: 2018-07-31T04:04:16Z
updated_at: 2019-05-23T06:04:03Z
author_association: OWNER
body: Parent: #354
type: issue
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/355/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzNDYwMjg2NTU=
title: Ability to display facet counts for many-to-many relationships
state: closed
created_at: 2018-07-31T04:14:26Z
updated_at: 2019-05-29T21:39:12Z
closed_at: 2019-05-25T16:30:09Z
author_association: OWNER
body: Parent: #354
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/356/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MjA1NzcwOTk2
title: Make .indexes compatible with older SQLite versions
state: closed
created_at: 2018-08-02T15:17:05Z
updated_at: 2018-08-02T15:17:30Z
closed_at: 2018-08-02T15:17:30Z
author_association: OWNER
pull_request: simonw/sqlite-utils/pulls/1
body: Older SQLite versions return a different set of columns from the PRAGMA we are using.
type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/1/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzNDgwNDM4ODQ=
title: Plugin hook for loading metadata.json
state: open
created_at: 2018-08-06T19:00:01Z
updated_at: 2020-06-21T22:19:58Z
author_association: OWNER
type: issue
body:
  For https://github.com/simonw/russian-ira-facebook-ads-datasette/tree/af6d956995e14afd585c35a6a06bb01da32043ba I wrote a script to convert YAML to JSON because YAML is a better format for embedding multi-line HTML descriptions and canned SQL statements.

  Example yaml metadata file: https://github.com/simonw/russian-ira-facebook-ads-datasette/blob/af6d956995e14afd585c35a6a06bb01da32043ba/russian-ads-metadata.yaml

  It would be useful if Datasette could be fed a YAML file directly:

      datasette -m metadata.yaml

  Question is... should this be a native feature (hence adding a YAML dependency) or should it be handled by a `datasette-metadata-yaml` plugin, using a new plugin hook for loading metadata? If so, what would other use-cases for that plugin hook be?

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/357/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MjA2ODYzODAz
title: Bump versions of pytest, pluggy and beautifulsoup4
state: closed
created_at: 2018-08-08T00:44:38Z
updated_at: 2018-08-08T01:11:13Z
closed_at: 2018-08-08T01:11:13Z
author_association: OWNER
pull_request: simonw/datasette/pulls/358
type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/358/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzNDk4Mjc2NDA=
title: Faceted browse against a JSON list of tags
state: closed
created_at: 2018-08-12T17:01:14Z
updated_at: 2019-05-29T21:39:12Z
closed_at: 2019-05-03T00:21:44Z
author_association: OWNER
type: issue
state_reason: completed
body:
  If a table has a `["foo", "bar", "baz"]` JSON column allow that to be faceted against.

  - [x] Support `?column__arraycontains=x` filter queries
  - [x] Support `?_facet_array=column` faceting

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/359/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzNDk4NTA2ODc=
title: Mechanism for adding foreign keys to an existing table
state: closed
created_at: 2018-08-12T22:50:56Z
updated_at: 2019-02-24T21:34:41Z
closed_at: 2019-02-24T21:34:41Z
author_association: OWNER
type: issue
state_reason: completed
body:
  SQLite does not have ALTER TABLE support for adding new foreign keys... but it turns out it's possible to make these changes without having to duplicate the entire table by carefully running `UPDATE sqlite_master SET sql=... WHERE type='table' AND name='X';`

  Here's how Django does it: https://github.com/django/django/blob/d3449faaa915a08c275b35de01e66a7ef6bdb2dc/django/db/backends/sqlite3/schema.py#L103-L125

  And here's the official documentation about this: https://sqlite.org/lang_altertable.html#otheralter (scroll to the very bottom of the page)

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/2/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzNTEwMTcxMjk=
title: Use pysqlite3 if available
state: closed
created_at: 2018-08-16T00:50:45Z
updated_at: 2018-08-16T01:50:42Z
closed_at: 2018-08-16T00:58:58Z
author_association: OWNER
type: issue
state_reason: completed
body:
  [pysqlite3](https://github.com/coleifer/pysqlite3) is a way to provide access to a more recent version of SQLite than the standard library `sqlite3` module (which tends to use the version provided with the operating system - which on e.g. the Travis CI Ubuntu build environment can be as old as 3.8.0).

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/360/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MjA4NzE5MDQz
title: Import pysqlite3 if available, closes #360
state: closed
created_at: 2018-08-16T00:52:21Z
updated_at: 2018-08-16T00:58:57Z
closed_at: 2018-08-16T00:58:57Z
author_association: OWNER
pull_request: simonw/datasette/pulls/361
type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/361/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzNTE4NDU0MjM=
title: Experiment with contentless FTS tables
state: closed
created_at: 2018-08-18T19:31:01Z
updated_at: 2019-07-22T20:58:55Z
closed_at: 2019-07-22T20:58:55Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Could greatly reduce size of resulting database for large datasets: http://cocoamine.net/blog/2015/09/07/contentless-fts4-for-large-immutable-documents/

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/3/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzNTI3NjgwMTc=
title: Add option to include/exclude columns in search filters
state: open
created_at: 2018-08-22T01:32:08Z
updated_at: 2020-11-03T19:01:59Z
author_association: NONE
type: issue
body:
  I have a dataset with many columns, of which only some are likely to be of interest for searching.

  It would be great for usability if the search filters in the UI could be configured to include/exclude columns.

  See also: https://github.com/simonw/datasette/issues/292

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/362/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MjExODYwNzA2
title: Search all apps during heroku publish
state: open
created_at: 2018-08-29T19:25:10Z
updated_at: 2018-08-31T14:39:45Z
author_association: FIRST_TIME_CONTRIBUTOR
pull_request: simonw/datasette/pulls/363
body: Adds the `-A` option to include apps from all organizations when searching app names for publish.
type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/363/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MjE0NjUzNjQx
title: Support for other types of databases using external connectors
state: open
created_at: 2018-09-11T14:31:47Z
updated_at: 2018-09-11T14:31:47Z
author_association: FIRST_TIME_CONTRIBUTOR
pull_request: simonw/datasette/pulls/364
type: pull
body:
  This PR is related to #293, but now all commits have been merged.

  The purpose is to support other file formats that aren't SQLite, like files with PyTables format. I've tried to accomplish that using external connectors published with entry points.

  The modifications in the original datasette code are minimal and many are in a separated file.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/364/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MjE2NjUxMzE3
title: fix small doc typo
state: closed
created_at: 2018-09-19T14:02:02Z
updated_at: 2019-12-19T02:30:33Z
closed_at: 2018-09-19T17:15:43Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/365
type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/365/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzNjk3MTYyMjg=
title: Default built image size over Zeit Now 100MiB limit
state: closed
created_at: 2018-10-12T21:27:17Z
updated_at: 2018-11-05T06:23:32Z
closed_at: 2018-11-05T06:23:32Z
author_association: CONTRIBUTOR
type: issue
state_reason: completed
body:
  Using `dataset publish now` with no other custom options on a small (43KB) sqlite database leads to the error "The built image size (373.5M) exceeds the 100MiB limit". I think this is because of a recent Zeit change: https://github.com/zeit/now-cli/issues/1523

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/366/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MjI2MzE0ODYy
title: Mark codemirror files as vendored
state: closed
created_at: 2018-10-27T18:41:25Z
updated_at: 2019-05-03T21:12:09Z
closed_at: 2019-05-03T21:11:20Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/367
type: pull
body:
  GitHub lists datasette as a Javascript project, primarily because of the vendored codemirror files. This is somewhat confusing when you're looking for datasette, knowing it's written in Python.

  Luckily it's possible exclude certain files from GitHub's code statistics: https://github.com/github/linguist#using-gitattributes

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/367/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MjI2MzE1NTEz
title: Update installation instructions
state: closed
created_at: 2018-10-27T18:52:31Z
updated_at: 2019-05-03T18:18:43Z
closed_at: 2019-05-03T18:18:42Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/368
type: pull
body:
  I was writing this as a response to your tweet, but decided I might just make it a pull request.

  I feel like it might be confusing to those unfamiliar with Python's `-m` flag and the built-in `venv` module to omit the space between the flag and its argument. By adding a space and prefixing the second occurrence of `venv` with a `./` it's maybe a bit clearer what the arguments are and what they do.

  By also using `python3 -m pip` it becomes even clearer that `-m` is a special flag that makes the python executable do neat things.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/368/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzNzQ5NTMwMDY=
title: Interface should show same JSON shape options for custom SQL queries
state: open
created_at: 2018-10-29T10:39:15Z
updated_at: 2020-05-30T17:24:06Z
author_association: CONTRIBUTOR
type: issue
body:
  At the moment the page returning a custom SQL query shows the JSON and CSV APIs, but not the multiple JSON shapes. However, adding the `_shape` parameter to the JSON API URL manually still works, so perhaps there should be consistency in the interface by having the same "Advanced Export" box for custom SQL queries.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/369/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzNzcxNTUzMjA=
title: Integration with JupyterLab
state: open
created_at: 2018-11-04T13:57:13Z
updated_at: 2022-09-29T08:17:47Z
author_association: CONTRIBUTOR
type: issue
body:
  I just watched a demo video for the [JupyterLab Chart Editor](https://www.crowdcast.io/e/introducing-JupyterLab-Chart-Editor/) which wraps the plotly chart editor app in a JupyterLab panel and lets you open a plotly chart JSON file in that editor.  Essentially, it pops an HTML app into a panel in JupyterLab, and I think registers the app as a file viewer for a particular file type. (I'm not completely taken by it, tbh, because it means you can do irreproducible things to the chart definition file, but that's another issue).

  JupyterLab extensions can also open files from a dialogue as the iframe/html previewer shows: https://github.com/timkpaine/jupyterlab_iframe.

  This made me wonder about what `datasette` integration with JupyterLab might do.

  For example, by right-clicking on a CSV file (for which there is already a CSV table view) in the file browser, offer a *View / Run as datasette* file viewer option that will:

  - run the CSV file through `csvs-to-sqlite`;
  - launch the `datasette` server and display the `datasette` view in a JupyterLab panel.

  (? Create a new SQLite db for each CSV file and launch each datasette view on a new port? Or have a JupyterLab (session?) SQLite db that stores all `datasette` viewed CSVs and runs on a single port?) 

  As a freebie, the `datasette` API would allow you to run efficient SQL queries against the file eg using using `pandas.read_sql()` queries in a notebook in the same space.

  Related:

  - [JupyterLab extensions docs](https://jupyterlab.readthedocs.io/en/stable/user/extensions.html)
  -  a [cookiecutter for wrting JupyterLab extensions using Javascript](https://github.com/jupyterlab/extension-cookiecutter-js)
  - a [cookiecutter for writing JupyterLab extensions using Typescript](https://github.com/jupyterlab/extension-cookiecutter-ts)
  - tutorial: [Let’s Make an xkcd JupyterLab Extension](https://jupyterlab.readthedocs.io/en/stable/developer/xkcd_extension_tutorial.html)

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/370/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzNzcxNTYzMzk=
title: datasette publish digitalocean plugin
state: closed
created_at: 2018-11-04T14:07:41Z
updated_at: 2021-01-04T20:14:28Z
closed_at: 2021-01-04T20:14:28Z
author_association: CONTRIBUTOR
type: issue
state_reason: completed
body:
  Provide support for launching `datasette` on Digital Ocean.

  Example: [Deploy Docker containers into Digital Ocean](https://blog.machinebox.io/deploy-machine-box-in-digital-ocean-385265fbeafd).

  Digital Ocean also has a preconfigured VM running Docker that can be launched from the command line via the Digital Ocean API: [Docker One-Click Application](https://www.digitalocean.com/docs/one-clicks/docker/).

  Related:
  - Launching containers in Digital Ocean servers running docker: [How To Provision and Manage Remote Docker Hosts with Docker Machine on Ubuntu 16.04](https://www.digitalocean.com/community/tutorials/how-to-provision-and-manage-remote-docker-hosts-with-docker-machine-on-ubuntu-16-04)
  - [How To Use Doctl, the Official DigitalOcean Command-Line Client](https://www.digitalocean.com/community/tutorials/how-to-use-doctl-the-official-digitalocean-command-line-client)

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/371/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzNzcxNjY3OTM=
title: Docker build tools
state: open
created_at: 2018-11-04T16:02:35Z
updated_at: 2018-11-04T16:02:35Z
author_association: CONTRIBUTOR
type: issue
body:
  In terms of small pieces lightly joined, I note that there are several tools starting to appear for building generating Dockerfiles and building Docker containers from simpler components such as `requirements.txt` files.

  If plugin/extensions builders want to include additional packages, then things like incremental builds of composable builds that add additional items into a base `datasette` container may be required.

  Examples of Dockerfile generators / container builders:

  - [openshift/source-to-image (s2i)](https://github.com/openshift/source-to-image)
  - [jupyter/repo2docker](https://github.com/jupyter/repo2docker)
  - [stencila/dockter](https://github.com/stencila/dockter)

  Discussions / threads  (via Binderhub gitter) on:
  - [why `repo2docker` not `s2i`](http://words.yuvi.in/post/why-not-s2i/)
  - [why `dockter` not `repo2docker`](https://twitter.com/choldgraf/status/1058499607309647872)
  - [composability in `s2i`](https://trello.com/c/AexIVZNf/1008-8-composable-builds-builds-evg)

  Relates to things like:

  - https://github.com/simonw/datasette/pull/280

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/372/reactions", "total_count": 2, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 2, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzNzcyNjYzNTE=
title: Views should be shown on root/index page along with tables
state: closed
created_at: 2018-11-05T06:28:41Z
updated_at: 2019-05-16T00:29:22Z
closed_at: 2019-05-16T00:29:22Z
author_association: CONTRIBUTOR
type: issue
state_reason: completed
body:
  At the moment the number of views is given on a datasette "homepage", but not links to any views themselves

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/373/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzNzc1MTg0OTk=
title: Get Datasette working with Zeit Now v2's 100MB image size limit
state: closed
created_at: 2018-11-05T18:08:29Z
updated_at: 2018-12-19T01:35:59Z
closed_at: 2018-12-19T01:35:59Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Follow-on from #366

  Zeit Now's v2 cloud has a 100MB size limit on Docker images, in order to support much faster wake-ups of new instances.

  Fitting Datasette AND the SQLite database it is hosting in here is going to be a challenge.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/374/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MjMyMTcyMTA2
title: Bump dependency versions
state: closed
created_at: 2018-11-20T02:23:12Z
updated_at: 2019-11-13T19:13:41Z
closed_at: 2019-11-13T19:13:41Z
author_association: OWNER
pull_request: simonw/datasette/pulls/389
type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/389/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MjM1MTk0Mjg2
title: tiny typo in customization docs
state: closed
created_at: 2018-12-01T13:44:42Z
updated_at: 2019-12-19T02:30:35Z
closed_at: 2018-12-16T21:32:56Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/390
body: was looking to add some custom templates to my use of datasette and saw this small typo.
type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/390/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzOTI2MTA4MDM=
title: Google Trends example doesn’t work
state: closed
created_at: 2018-12-19T13:51:38Z
updated_at: 2019-01-02T19:45:13Z
closed_at: 2019-01-02T19:45:12Z
author_association: NONE
body:
  https://google-trends.datasettes.com/

  I see a cloud flare error.

type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/391/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MjQxNDE4NDQz
title: Fix some regex DeprecationWarnings
state: closed
created_at: 2018-12-29T02:10:28Z
updated_at: 2018-12-29T02:22:28Z
closed_at: 2018-12-29T02:22:28Z
author_association: OWNER
pull_request: simonw/datasette/pulls/392
type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/392/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzOTUyMzYwNjY=
title: CSV export in "Advanced export" pane doesn't respect query
state: closed
created_at: 2019-01-02T12:39:41Z
updated_at: 2021-06-17T18:14:24Z
closed_at: 2019-01-03T02:44:10Z
author_association: NONE
type: issue
state_reason: completed
body:
  It looks like there's an inconsistency when exporting to CSV via the the web interface. Say I'm looking at [songs released in 1989](https://fivethirtyeight.datasettes.com/fivethirtyeight-c300360/classic-rock%2Fclassic-rock-song-list?Release+Year__exact=1989) in the `classic-rock/classic-rock-song-list` table from the Five Thirty Eight data. The JSON and CSV export links at the top of the page both give me filtered data using `Release+Year__exact=1989` in the URL. In the `Advanced export` tab, though, the CSV option gives me the whole data set, while the JSON options preserve the query.

  It may be that this is intended behaviour related to the streaming CSV stuff [discussed here](https://github.com/simonw/datasette/issues/266), but if that's the case then I think it should be a little clearer.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/393/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzOTYyMTIwMjE=
title: base_url configuration setting
state: closed
created_at: 2019-01-05T23:48:48Z
updated_at: 2020-06-11T09:15:20Z
closed_at: 2020-03-25T00:18:45Z
author_association: OWNER
type: issue
state_reason: completed
body:
  I've identified a couple of use-cases for running Datasette in a way that over-rides the default way that internal URLs are generated.

  1. Running behind a reverse proxy. I tried running Datasette behind a proxy and found that some of the generated internal links incorrectly referenced `http://127.0.0.1:8001/fixtures/...` - when they should have been referencing `http://my-host.my-domain.com/fixtures/...` - this is a problem both for links within the HTML interface but also for the `toggle_url` keys returned in the JSON as part of the facets datastructure.
  2. I would like it to be possible to host a Datasette instance at e.g. `https://www.mynewspaper.com/interactives/2018/election-results/` - either through careful HTTP proxying or, once Datasette has been ported to ASGI, by mounting a Datasette ASGI instance deep within an existing set of URL routes.

  I'm going to add a `url_prefix` configuration option. This will default to `""`, which means Datasette will behave as it does at the moment - it will use `/` for most URL prefixes in the HTML version, and an absolute URL derived from the incoming `Host` header for URLs that are returned as part of the JSON output.

  If `url_prefix` is set to another value (either a full URL or a path) then this path will be appended to all generated URLs.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/394/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzOTYyMTUwNDM=
title: Find a cleaner pattern for fixtures with arguments
state: closed
created_at: 2019-01-06T00:31:22Z
updated_at: 2020-06-07T21:23:22Z
closed_at: 2020-06-07T21:23:22Z
author_association: OWNER
type: issue
state_reason: completed
body:
  A lot of Datasette tests look like this:

  https://github.com/simonw/datasette/blob/b65d97792a53f78cb14b226231063209d22c4602/tests/test_api.py#L438-L444

  The loop here isn't actually expected to loop - it's there because the `make_app_client` function yields a value and then cleans it up afterwards.

  This pattern works, but it is a little confusing. It would be nice to replace it with something less strange looking.

  The answer may be to switch to the "factories as fixtures" pattern described here: https://docs.pytest.org/en/latest/fixture.html#factories-as-fixtures

  In particular some variant of this example:

  ```
  @pytest.fixture
  def make_customer_record():

      created_records = []

      def _make_customer_record(name):
          record = models.Customer(name=name, orders=[])
          created_records.append(record)
          return record

      yield _make_customer_record

      for record in created_records:
          record.destroy()


  def test_customer_records(make_customer_record):
      customer_1 = make_customer_record("Lisa")
      customer_2 = make_customer_record("Mike")
      customer_3 = make_customer_record("Meredith")
  ```

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/395/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzOTcwOTg4ODI=
title: Add pragma compile_options output to /-/versions
state: closed
created_at: 2019-01-08T21:43:54Z
updated_at: 2019-01-11T00:55:22Z
closed_at: 2019-01-11T00:44:56Z
author_association: OWNER
type: issue
state_reason: completed
body:
  ```
  sqlite> pragma compile_options
     ...> ;
  BUG_COMPATIBLE_20160819
  COMPILER=clang-9.0.0
  DEFAULT_CACHE_SIZE=2000
  DEFAULT_CKPTFULLFSYNC
  DEFAULT_JOURNAL_SIZE_LIMIT=32768
  DEFAULT_PAGE_SIZE=4096
  DEFAULT_SYNCHRONOUS=2
  DEFAULT_WAL_SYNCHRONOUS=1
  ENABLE_API_ARMOR
  ENABLE_COLUMN_METADATA
  ENABLE_DBSTAT_VTAB
  ENABLE_FTS3
  ENABLE_FTS3_PARENTHESIS
  ENABLE_FTS3_TOKENIZER
  ENABLE_FTS4
  ENABLE_FTS5
  ENABLE_JSON1
  ENABLE_LOCKING_STYLE=1
  ENABLE_PREUPDATE_HOOK
  ENABLE_RTREE
  ENABLE_SESSION
  ENABLE_SNAPSHOT
  ENABLE_SQLLOG
  ENABLE_UNKNOWN_SQL_FUNCTION
  ENABLE_UPDATE_DELETE_LIMIT
  HAVE_ISNAN
  MAX_LENGTH=2147483645
  MAX_MMAP_SIZE=1073741824
  MAX_VARIABLE_NUMBER=500000
  OMIT_AUTORESET
  OMIT_LOAD_EXTENSION
  STMTJRNL_SPILL=131072
  THREADSAFE=2
  USE_URI
  sqlite>
  ```

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/396/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzOTcxMjk1NjQ=
title: Update official datasetteproject/datasette Docker container to SQLite 3.26.0
state: closed
created_at: 2019-01-08T22:51:50Z
updated_at: 2019-01-11T01:25:33Z
closed_at: 2019-01-11T00:56:18Z
author_association: NONE
type: issue
state_reason: completed
body:
  I try to start datasette on a database that contains the below view

  It fails in a way that makes me think it does not support the window functions SQL syntax.


  ```
  create view general_ledger as
  select transactions.account_number, strftime("%Y-%m-%d", verifications.verification_date) as verification_date, verifications.verification_number, verifications.verification_text, 
  case when transactions.centi_amount >= 0 and verifications.verification_number > 0 then printf("%.2f", (transactions.centi_amount/100.0))
  end as debit,
  case when transactions.centi_amount <= 0 and verifications.verification_number > 0 then printf("%.2f", (transactions.centi_amount/100.0)) 
  end as credit,
  printf("%.2f", sum(transactions.centi_amount) over (partition by transactions.account_number 
      order by verifications.verification_number range between unbounded preceding and current row)/100.0)
  from verifications inner join transactions on transactions.verification_id = verifications.id 
  order by transactions.account_number, verifications.verification_number;
  ```

  ```
  docker run -p 8001:8001 -v `pwd`:/mnt     datasetteproject/datasette     datasette -p 8001 -h 0.0.0.0 /mnt/ledger.db
  Serve! files=('/mnt/ledger.db',) on port 8001
  Traceback (most recent call last):
    File "/usr/local/bin/datasette", line 11, in <module>
      sys.exit(cli())
    File "/usr/local/lib/python3.6/site-packages/click/core.py", line 722, in __call__
      return self.main(*args, **kwargs)
    File "/usr/local/lib/python3.6/site-packages/click/core.py", line 697, in main
      rv = self.invoke(ctx)
    File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
      return _process_result(sub_ctx.command.invoke(sub_ctx))
    File "/usr/local/lib/python3.6/site-packages/click/core.py", line 895, in invoke
      return ctx.invoke(self.callback, **ctx.params)
    File "/usr/local/lib/python3.6/site-packages/click/core.py", line 535, in invoke
      return callback(*args, **kwargs)
    File "/usr/local/lib/python3.6/site-packages/datasette/cli.py", line 375, in serve
      ds.inspect()
    File "/usr/local/lib/python3.6/site-packages/datasette/app.py", line 308, in inspect
      "views": inspect_views(conn),
    File "/usr/local/lib/python3.6/site-packages/datasette/inspect.py", line 30, in inspect_views
      return [v[0] for v in conn.execute('select name from sqlite_master where type = "view"')]
  sqlite3.DatabaseError: malformed database schema (general_ledger) - near "over": syntax error

  ```

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/397/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzOTgwMTE2NTg=
title: Ensure downloading a 100+MB SQLite database file works
state: closed
created_at: 2019-01-10T20:57:52Z
updated_at: 2020-12-05T19:36:27Z
closed_at: 2020-12-05T19:36:27Z
author_association: OWNER
body: I've seen attempted downloads of large files fail after about ten seconds.
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/398/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzOTgwODkwODk=
title: /-/versions for official Docker image returns wrong Datasette version
state: closed
created_at: 2019-01-11T01:19:58Z
updated_at: 2019-01-13T23:31:59Z
closed_at: 2019-01-13T23:10:45Z
author_association: OWNER
type: issue
state_reason: completed
body:
  ```
  docker run -p 8001:8001 datasetteproject/datasette datasette -p 8001 -h 0.0.0.0
  ```
  http://0.0.0.0:8001/-/versions returns this:
  ```
  {
      "datasette": {
          "version": "0+unknown"
      },
      ...
  ```
  This is because the Docker image is built by copying in the Datasette source code, which confuses versioneer. Maybe the Docker image should install the code using a wheel or similar?

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/399/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWUzOTg1NTkxOTU=
title: datasette publish cloudrun plugin
state: closed
created_at: 2019-01-12T14:35:11Z
updated_at: 2019-05-03T16:57:35Z
closed_at: 2019-05-03T16:57:35Z
author_association: CONTRIBUTOR
type: issue
state_reason: completed
body:
  Google announced that they may launch a simple service for running Docker containers (previously serverless containers, now called "cloud run" -- link to alpha [here](https://services.google.com/fb/forms/serverlesscontainers/)). If/when this happens, it might be a good fit for publishing datasettes? (at least using the current version, manually publishing a datasette seems relatively painless).

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/400/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU0MDAyMjk5ODQ=
title: How to pass configuration to plugins?
state: closed
created_at: 2019-01-17T11:20:41Z
updated_at: 2019-01-18T11:48:13Z
closed_at: 2019-01-18T06:49:07Z
author_association: NONE
type: issue
state_reason: completed
body:
  Hi,
  Firstly, thanks for your work on datasette, it is a hugely useful tool!

  I've been working on a fork [https://github.com/dazzag24/datasette-cluster-map] of datasette-cluster-map to allow the tileserver to be easily switched. Primarily because the tiles being served in the current version use localised text for labels and I'd like to have English used for these names instead.

  It uses http://leaflet-extras.github.io/leaflet-providers/preview/ to allow you to simply set the tile provider using a call like so:
  ```    
  let tiles = L.tileLayer.provider('Esri.WorldTopoMap');
  ```
  instead of the current:
  ```
  let tiles = L.tileLayer('https://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png', {
          maxZoom: 19,
          detectRetina: true,
          attribution: '&copy; <a href="https://www.openstreetmap.org/copyright">OpenStreetMap</a> contributors'
      }),
    ```
  However I've got stuck in trying to work out how to pass the provider string to the plugin.
  In the documentation: https://datasette.readthedocs.io/en/stable/plugins.html you discuss configuration of plugins and use an example of passing in which latitude and longitude columns should be used.  However I cannot seem to see anywhere in the current datasette-cluster-map code where these config params are passed in or used.

  Can you please point me to an example or how to pass configuration from the metadata.json down into a plugin.  Once I've over come this issue I was wondering if you would be interested in taking this change into your version?

  Many thanks
  Darren

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/401/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU0MDAzNDA5MDU=
title: Use SQLITE_DBCONFIG_DEFENSIVE plus other recommendations from SQLite security docs
state: open
created_at: 2019-01-17T15:52:28Z
updated_at: 2019-01-17T16:15:21Z
author_association: OWNER
type: issue
body:
  > Was just having a skim through the datasette source. Given that the vuln impacts shadow tables, wasn't sure whether these are also covered by the immutable flag. Latest release introduced a SQLITE_DBCONFIG_DEFENSIVE flag that they recommend setting: https://sqlite.org/security.html

  https://twitter.com/ignoredambience/status/1085926961413869568

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/402/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU0MDA1MTEyMDY=
title: How does persistence work?
state: closed
created_at: 2019-01-17T23:41:57Z
updated_at: 2019-01-19T05:47:55Z
closed_at: 2019-01-18T06:51:14Z
author_association: NONE
type: issue
state_reason: completed
body:
  I was under the impression that now.sh is for stateless microservices. So where are these SQLite databases stored and when do they get created and destroyed?

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/403/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MjQ3NTc2OTQy
title: Fts5
state: closed
created_at: 2019-01-25T06:54:05Z
updated_at: 2019-01-25T06:54:33Z
closed_at: 2019-01-25T06:54:33Z
author_association: OWNER
pull_request: simonw/sqlite-utils/pulls/4
type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/4/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MjQ3ODYxNDE5
title: Run Travis tests against Python 3.8-dev
state: closed
created_at: 2019-01-26T02:30:55Z
updated_at: 2019-01-26T02:37:54Z
closed_at: 2019-01-26T02:37:54Z
author_association: OWNER
pull_request: simonw/sqlite-utils/pulls/5
type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/5/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MjQ3OTIzMzQ3
title: Experiment: run Jinja in async mode
state: closed
created_at: 2019-01-27T00:28:44Z
updated_at: 2019-11-12T05:02:18Z
closed_at: 2019-11-12T05:02:13Z
author_association: OWNER
pull_request: simonw/datasette/pulls/404
type: pull
body:
  See http://jinja.pocoo.org/docs/2.10/api/#async-support

  Tests all pass. Have not checked performance difference yet.

  Creating pull request to run tests in Travis. This is not ready to merge - I'm not yet sure if this is a good idea.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/404/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU0MDM2MTc4ODE=
title: .json?_nl=on option for exporting newline-delimited JSON
state: closed
created_at: 2019-01-28T01:10:45Z
updated_at: 2019-01-28T01:49:00Z
closed_at: 2019-01-28T01:48:37Z
author_association: OWNER
type: issue
state_reason: completed
body:
  The neat thing about newline-delimited JSON is that you don't have to read an entire array (of potentially thousands of objects) into memory in order to parse it - you can parse things a line at a time instead.

  It will look like this:

  `https://latest.datasette.io/fixtures/facetable.json?_shape=array&_nl=on`
  ```
  {"pk": 1, "planet_int": 1, "on_earth": 1, "state": "CA", "city_id": 1, "neighborhood": "Mission"}
  {"pk": 2, "planet_int": 1, "on_earth": 1, "state": "CA", "city_id": 1, "neighborhood": "Dogpatch"}
  {"pk": 3, "planet_int": 1, "on_earth": 1, "state": "CA", "city_id": 1, "neighborhood": "SOMA"}
  {"pk": 4, "planet_int": 1, "on_earth": 1, "state": "CA", "city_id": 1, "neighborhood": "Tenderloin"}
  {"pk": 5, "planet_int": 1, "on_earth": 1, "state": "CA", "city_id": 1, "neighborhood": "Bernal Heights"}
  ```

  I added this as part of the `sqlite-utils json` CLI command is this commit - I think Datasette should offer it as well: https://github.com/simonw/sqlite-utils/commit/5466c9745dfef858286146ea158ffd5a71391d10

  It can be offered alongside `_stream=on` (which currently only works for CSV, but it could work for JSON as well thanks to this trick).

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/405/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU0MDM2MjQwOTA=
title: "sqlite-utils insert" should support newline-delimited JSON
state: closed
created_at: 2019-01-28T02:00:02Z
updated_at: 2019-01-28T02:17:45Z
closed_at: 2019-01-28T02:17:45Z
author_association: OWNER
type: issue
state_reason: completed
body:
  We can already export newline delimited JSON. We should learn to import it as well.

  The neat thing about importing it is that you can import GBs of data without having to read the whole lot into memory in order to decode the wrapping JSON array.

  Datasette can export it now: https://github.com/simonw/datasette/issues/405

  Demo: https://latest.datasette.io/fixtures/facetable.json?_shape=array&_nl=on

  It should be possible to do this:

      $ curl "https://latest.datasette.io/fixtures/facetable.json?_shape=array&_nl=on" \
          | sqlite-utils insert data.db facetable - --nl

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/6/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU0MDM2MjU2NzQ=
title: .insert_all() should accept a generator and process it efficiently
state: closed
created_at: 2019-01-28T02:11:58Z
updated_at: 2019-01-28T06:26:53Z
closed_at: 2019-01-28T06:26:53Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Right now you have to load every record into memory before passing the list to `.insert_all()` and friends.

  If you want to process millions of rows, this is inefficient. Python has generators - we should use them!

  The only catch here is that part of the magic of `sqlite-utils` is that it guesses the column types and creates the table for you. This code will need to be updated to notice if the table needs creating and, if it does, create it using the first X (where x=1,000 but can be customized) records.

  If a record outside of those first 1,000 has a rogue column, we can crash with an error.

  This will free us up to make the `--nl` option added in #6 much more efficient.

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/7/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU0MDM5MjI2NDQ=
title: Problems handling column names containing spaces or -
state: closed
created_at: 2019-01-28T17:23:28Z
updated_at: 2019-04-14T15:29:33Z
closed_at: 2019-02-23T21:09:03Z
author_association: NONE
type: issue
state_reason: completed
body:
  Irrrespective of whether using column names containing a space or - character is good practice, SQLite does allow it, but `sqlite-utils` throws an error in the following cases:

  ```python
  from sqlite_utils import Database

  dbname = 'test.db'
  DB = Database(sqlite3.connect(dbname))

  import pandas as pd
  df = pd.DataFrame({'col1':range(3), 'col2':range(3)})

  #Convert pandas dataframe to appropriate list/dict format
  DB['test1'].insert_all( df.to_dict(orient='records') )
  #Works fine
  ```

  However:

  ```python
  df = pd.DataFrame({'col 1':range(3), 'col2':range(3)})
  DB['test1'].insert_all(df.to_dict(orient='records'))
  ```

  throws:

  ```
  ---------------------------------------------------------------------------
  OperationalError                          Traceback (most recent call last)
  <ipython-input-27-070b758f4f92> in <module>()
        1 import pandas as pd
        2 df = pd.DataFrame({'col 1':range(3), 'col2':range(3)})
  ----> 3 DB['test1'].insert_all(df.to_dict(orient='records'))

  /usr/local/lib/python3.7/site-packages/sqlite_utils/db.py in insert_all(self, records, pk, foreign_keys, upsert, batch_size, column_order)
      327                     jsonify_if_needed(record.get(key, None)) for key in all_columns
      328                 )
  --> 329             result = self.db.conn.execute(sql, values)
      330             self.db.conn.commit()
      331             self.last_id = result.lastrowid

  OperationalError: near "1": syntax error
  ```

  and:

  ```python
  df = pd.DataFrame({'col-1':range(3), 'col2':range(3)})
  DB['test1'].upsert_all(df.to_dict(orient='records'))
  ```

  results in:

  ```
  ---------------------------------------------------------------------------
  OperationalError                          Traceback (most recent call last)
  <ipython-input-28-654523549d20> in <module>()
        1 import pandas as pd
        2 df = pd.DataFrame({'col-1':range(3), 'col2':range(3)})
  ----> 3 DB['test1'].insert_all(df.to_dict(orient='records'))

  /usr/local/lib/python3.7/site-packages/sqlite_utils/db.py in insert_all(self, records, pk, foreign_keys, upsert, batch_size, column_order)
      327                     jsonify_if_needed(record.get(key, None)) for key in all_columns
      328                 )
  --> 329             result = self.db.conn.execute(sql, values)
      330             self.db.conn.commit()
      331             self.last_id = result.lastrowid

  OperationalError: near "-": syntax error
  ```

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/8/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MjQ5NjgwOTQ0
title: :pencil: Updates my_database.py to my_database.db
state: closed
created_at: 2019-02-01T17:35:43Z
updated_at: 2019-02-24T03:55:04Z
closed_at: 2019-02-24T03:55:04Z
author_association: CONTRIBUTOR
pull_request: simonw/sqlite-utils/pulls/9
body: I noticed that both `.py` and `.db` were used in the docs and assumed you'd prefer `.db`.
type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/9/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU0MDYwNTUyMDE=
title: Support nullable foreign keys in _labels mode
state: closed
created_at: 2019-02-03T05:34:20Z
updated_at: 2019-11-02T22:39:28Z
closed_at: 2019-11-02T22:30:27Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Currently if there's a null in a foreign key we get "None" displayed in the inflated view:

  <img width="412" alt="screen shot 2019-02-02 at 9 33 37 pm" src="https://user-images.githubusercontent.com/9599/52173123-46221e80-2732-11e9-8dcb-f6f4bc768c33.png">

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/406/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MjUwNjI4Mjc1
title: Heroku --include-vcs-ignore
state: closed
created_at: 2019-02-06T04:06:20Z
updated_at: 2019-02-06T04:31:30Z
closed_at: 2019-02-06T04:15:47Z
author_association: OWNER
pull_request: simonw/datasette/pulls/407
type: pull
body:
  Should mean `datasette publish heroku` can work under Travis, unlike this failure:

  https://travis-ci.org/simonw/fivethirtyeight-datasette/builds/488047550

  ```
  2.25s$ datasette publish heroku fivethirtyeight.db -m metadata.json -n fivethirtyeight-datasette
  tar: unrecognized option '--exclude-vcs-ignores'
  Try 'tar --help' or 'tar --usage' for more information.
   ▸    Command failed: tar cz -C /tmp/tmpuaxm7i8f --exclude-vcs-ignores --exclude
   ▸    .git --exclude .gitmodules . >
   ▸    /tmp/f49440e0-1bf3-4d3f-9eb0-fbc2967d1fd4.tar.gz
   ▸    tar: unrecognized option '--exclude-vcs-ignores'
   ▸    Try 'tar --help' or 'tar --usage' for more information.
   ▸    
  The command "datasette publish heroku fivethirtyeight.db -m metadata.json -n fivethirtyeight-datasette" exited with 0.
  ```

  The fix for that issue is to call the heroku command like this:

      heroku builds:create -a app_name --include-vcs-ignore

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/407/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU0MDcxNzQxNzM=
title: Show metadata info (e.g. license, source) on custom SQL query pages
state: closed
created_at: 2019-02-06T10:43:34Z
updated_at: 2019-10-14T03:53:22Z
closed_at: 2019-10-14T03:53:22Z
author_association: NONE
type: issue
state_reason: completed
body:
  Currently metadata info is not displayed on custom SQL pages.

  E.g. compare the footer of [this normal table page](https://register-of-members-interests.datasettes.com/regmem-98dc8b7/categories) with the footer [this custom SQL page](https://register-of-members-interests.datasettes.com/regmem-98dc8b7?sql=select+*+from+categories).

  This is important in order to adhere to attribution license requirements.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/408/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU0MDgzNzY4MjU=
title: Zeit API v1 does not work for new users - need to migrate to v2
state: closed
created_at: 2019-02-09T00:50:33Z
updated_at: 2020-04-06T15:44:46Z
closed_at: 2020-04-06T15:44:46Z
author_association: NONE
type: issue
state_reason: completed
body:
  Hello there,

  This looks like a great tool. Thanks. 

  Unfortunately, I hit the following error:

  ```
  michael@hazel ~/src/cc-datasette/data/out datasette publish now cc-datasette.db
  > WARN! You are using an old version of the Now Platform. More: https://zeit.co/docs/v1-upgrade
  > Deploying /tmp/tmpjtrxwsyf/datasette under michaelmcandrew
  > Using project datasette
  > Error! You tried to create a Now 1.0 deployment. Please use Now 2.0 instead: https://zeit.co/upgrade
  ```
  I'm guessing you might not hit this because you are not a 'new user' of Zeit (https://github.com/zeit/now-cli/issues/1805#issuecomment-452470953).

  Would it be a lot of work to upgrade to the new Zeit API, do you think?

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/409/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU0MDg1MTgwMjQ=
title: How to setup a multi database environment?
state: closed
created_at: 2019-02-10T09:39:24Z
updated_at: 2019-04-12T04:42:28Z
closed_at: 2019-04-12T04:42:27Z
author_association: NONE
type: issue
state_reason: completed
body:
  Hi,
  first of all I need to write that Simon Willison and datasette are really great.

  I have probably a stupid question, but it seems to me that I do not have the reply in the documentation.

  I have installed datasette and run it with `datasette mydb.db`, and I can reach it  on `http://127.0.0.1:8001`.

  But how to work with more than one db? Imagine I have ten sqlite databases, and that I need to explore/query these via datasette, how to run datasette? Is it possibile to create a sort of db index and than run `datasette serve myindex`?

  Thank you

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/410/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU0MTAzODQ5ODg=
title: How to pass named parameter into spatialite MakePoint() function
state: closed
created_at: 2019-02-14T16:30:22Z
updated_at: 2023-10-25T13:23:04Z
closed_at: 2019-05-05T12:25:04Z
author_association: NONE
type: issue
state_reason: completed
body:
  Hi,
  datasette version: "0.26.2"
  extensions: 
         spatialite: "4.4.0-RC0"
  sqlite version: "3.22.0"

  I have a table of airports with latitude and longitude columns.  I've added spatialite (with KNN support).  After creating the db using csvs-to-sqlit, I run these commands to setup the spatialite tables:

  ```
  conn.execute('SELECT InitSpatialMetadata(1)')

  conn.execute("SELECT AddGeometryColumn('airports', 'point_geom', 4326, 'POINT', 2);")

  conn.execute('''UPDATE airports SET point_geom = GeomFromText('POINT('||"longitude"||' '||"latitude"||')',4326);''')

  conn.execute("SELECT CreateSpatialIndex('airports', 'point_geom');")
  ```

  I'm attempting to create a canned query and have this in my metadata.json file:
  ```
  "find_airports_nearest_to_point":{
      "sql":"SELECT a.pos AS rank, b.id, b.name, b.country, b.latitude AS latitude, b.longitude AS longitude, a.distance / 1000.0 AS dist_km FROM KNN AS a JOIN airports AS b ON (b.rowid = a.fid) WHERE f_table_name = \"airports\" AND ref_geometry = MakePoint( :Long , :Lat ) AND max_items = 10;"}
  ```
  which doesn't seem to perform the templating of the name parameters correctly and I get no results.  

  Have also tired:
  ```
  MakePoint( || :Long || , || :Lat || )
  ```
  which returns this error:
  ```
  near "||": syntax error
  ```

  However I cannot seem to find the correct combination of named parameter syntax (:Lat) or sqlite concatenation operator to make it work.  Any ideas if using named parameters inside functions is supported?

  Thanks
  Darren

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/411/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU0MTEwNjY3MDA=
title: Error in upsert if column named 'order'
state: closed
created_at: 2019-02-16T12:05:18Z
updated_at: 2019-02-24T16:55:38Z
closed_at: 2019-02-24T16:55:37Z
author_association: NONE
type: issue
state_reason: completed
body:
  The following works fine:
  ```
  connX = sqlite3.connect('DELME.db', timeout=10)

  dfX=pd.DataFrame({'col1':range(3),'col2':range(3)})
  DBX = Database(connX)
  DBX['test'].upsert_all(dfX.to_dict(orient='records'))
  ```

  But if a column is named `order`:
  ```
  connX = sqlite3.connect('DELME.db', timeout=10)

  dfX=pd.DataFrame({'order':range(3),'col2':range(3)})
  DBX = Database(connX)
  DBX['test'].upsert_all(dfX.to_dict(orient='records'))
  ```

  it throws an error:

  ```
  ---------------------------------------------------------------------------
  OperationalError                          Traceback (most recent call last)
  <ipython-input-130-7dba33cd806c> in <module>
        3 dfX=pd.DataFrame({'order':range(3),'col2':range(3)})
        4 DBX = Database(connX)
  ----> 5 DBX['test'].upsert_all(dfX.to_dict(orient='records'))

  /usr/local/lib/python3.7/site-packages/sqlite_utils/db.py in upsert_all(self, records, pk, foreign_keys, column_order)
      347             foreign_keys=foreign_keys,
      348             upsert=True,
  --> 349             column_order=column_order,
      350         )
      351 

  /usr/local/lib/python3.7/site-packages/sqlite_utils/db.py in insert_all(self, records, pk, foreign_keys, upsert, batch_size, column_order)
      327                     jsonify_if_needed(record.get(key, None)) for key in all_columns
      328                 )
  --> 329             result = self.db.conn.execute(sql, values)
      330             self.db.conn.commit()
      331             self.last_id = result.lastrowid

  OperationalError: near "order": syntax error
  ```

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/10/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU0MTEyNTc5ODE=
title: Linked Data(sette)
state: open
created_at: 2019-02-18T00:38:14Z
updated_at: 2019-03-19T10:09:46Z
author_association: NONE
type: issue
body:
  I've a radical feature idea (possible first as an extension in order to experiment?): 

  I'd like to link to a remote table from a remote database, e.g. with a function "linked_datasette()". So one could do following query:
  ```
  SELECT foo.id, foo.a, remote_party.b
  FROM foo
  JOIN linked_datasette("https://parlgov.datasettes.com/parlgov-b42a2f2") AS remote_party 
    ON foo.id=remote_party.id
  ```
  This is inspired by SPARQL's SERVICE keyword for remote RDF "endpoints".

  There's a foundation in the SQL Standard called SQL/MED (https://rhaas.blogspot.com/2011/01/why-sqlmed-is-cool.html ).

  And here's an implementation from me in Postgres FDW to connect another Postgres "endpoint": https://pastebin.com/Fz2v64Cz .

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/412/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU0MTM3NDA2ODQ=
title: Detect numpy types when creating tables
state: closed
created_at: 2019-02-23T21:09:35Z
updated_at: 2019-02-24T04:02:20Z
closed_at: 2019-02-24T04:02:20Z
author_association: OWNER
body: Inspired by #8
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/11/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MjU1NjU4MTEy
title: Support for numpy types, closes #11
state: closed
created_at: 2019-02-24T03:57:32Z
updated_at: 2019-02-24T04:02:20Z
closed_at: 2019-02-24T04:02:20Z
author_association: OWNER
pull_request: simonw/sqlite-utils/pulls/12
type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/12/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU0MTM3NzkyMTA=
title: Ability to automatically create IDs from content hash of row
state: closed
created_at: 2019-02-24T04:07:08Z
updated_at: 2019-02-24T04:36:48Z
closed_at: 2019-02-24T04:36:48Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Sometimes when you are importing data the underlying source provides records without IDs that can be uniquely identified by their contents.

  A utility mechanism for calculating a sha1 hash of the contents and using that as a unique ID would be useful.

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/13/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU0MTM4NDI2MTE=
title: Utilities for adding indexes
state: closed
created_at: 2019-02-24T16:57:28Z
updated_at: 2019-02-24T19:11:28Z
closed_at: 2019-02-24T19:11:28Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Both in the Python API and the CLI tool. For the CLI tool this should work:

      $ sqlite-utils create-index mydb.db mytable col1 col2

  This will create a compound index across col1 and col2. The name of the index will be automatically chosen unless you use the `--name=...` option.

  Support a `--unique` option too.

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/14/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU0MTM4NTcyNTc=
title: Ability to add columns to tables
state: closed
created_at: 2019-02-24T19:20:51Z
updated_at: 2019-02-24T20:04:40Z
closed_at: 2019-02-24T20:04:40Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Makes sense to do this before foreign keys in #2

  Python:

      db["table"].add_column("new_column", int)

  CLI:

      $ sqlite-utils add-column table new_column INTEGER

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/15/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU0MTM4Njc1Mzc=
title: add_column() should support REFERENCES {other_table}({other_column})
state: closed
created_at: 2019-02-24T21:00:45Z
updated_at: 2019-05-29T05:17:59Z
closed_at: 2019-05-29T04:56:18Z
author_association: OWNER
body: Related to #2
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/16/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU0MTM4Njg0NTI=
title: Improve and document foreign_keys=... argument to insert/create/etc
state: closed
created_at: 2019-02-24T21:09:11Z
updated_at: 2019-02-24T23:45:48Z
closed_at: 2019-02-24T23:45:48Z
author_association: OWNER
type: issue
state_reason: completed
body:
  The `foreign_keys=` argument to `table.insert_all()` and friends can be used to specify foreign key relationships that should be created.

  It is not yet documented. It also requires you to specify the SQLite type of each column, even though this can be detected by introspecting the referenced table:

      cols = [c for c in self.db[other_table].columns if c.name == other_column]
      cols[0].type

  Relates to #2 

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/17/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU0MTM4NzEyNjY=
title: .insert/.upsert/.insert_all/.upsert_all should add missing columns
state: closed
created_at: 2019-02-24T21:36:11Z
updated_at: 2019-05-25T00:42:11Z
closed_at: 2019-05-25T00:42:11Z
author_association: OWNER
type: issue
state_reason: completed
body:
  This is a larger change, but it would be incredibly useful: if you attempt to insert or update a document with a field that does not currently exist in the underlying table, sqlite-utils should add the appropriate column for you.

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/18/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MjU1NzI1MDU3
title: Update spatialite.rst
state: closed
created_at: 2019-02-25T00:08:35Z
updated_at: 2019-03-15T05:06:45Z
closed_at: 2019-03-15T05:06:45Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/413
body: a line of sql added to create the idx_<table_name> in the python recipe
type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/413/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU0MTU1NzU2MjQ=
title: datasette requires specific version of Click
state: closed
created_at: 2019-02-28T11:24:59Z
updated_at: 2019-03-15T04:42:13Z
closed_at: 2019-03-15T04:42:13Z
author_association: CONTRIBUTOR
type: issue
state_reason: completed
body:
  Is `datasette` beholden to version `click==6.7`?

  Current release is at 7.0. Can the requirement be liberalised, eg to `>=6.7`?

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/414/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU0MTgzMjk4NDI=
title: Add query parameter to hide SQL textarea
state: closed
created_at: 2019-03-07T14:11:30Z
updated_at: 2019-03-15T09:30:57Z
closed_at: 2019-03-15T05:22:43Z
author_association: NONE
type: issue
state_reason: completed
body:
  It would be cool if there was a query parameter to hide / remove the SQL textarea. Then I could simply save a bookmark for a certain query and open it to see the data without having to scroll below the (long) SQL query first.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/415/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MjYxNDE4Mjg1
title: URL hashing now optional: turn on with --config hash_urls:1 (#418)
state: closed
created_at: 2019-03-15T04:26:06Z
updated_at: 2019-03-17T22:55:04Z
closed_at: 2019-03-17T22:55:04Z
author_association: OWNER
pull_request: simonw/datasette/pulls/416
type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/416/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU0MjE1NDY5NDQ=
title: Datasette Library
state: open
created_at: 2019-03-15T14:30:22Z
updated_at: 2020-12-29T14:34:50Z
author_association: OWNER
type: issue
body:
  The ability to run Datasette in a mode where it automatically picks up new (or modified) files in a directory tree without needing to restart the server.

  Suggested command:

      datasette library /path/to/mydbs/

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/417/reactions", "total_count": 8, "+1": 8, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU0MjE1NDg4ODE=
title: Hashed URLs should be optional
state: closed
created_at: 2019-03-15T14:34:12Z
updated_at: 2019-05-16T15:12:26Z
closed_at: 2019-05-16T15:12:26Z
author_association: OWNER
type: issue
state_reason: completed
body:
  The cute performance hack where a hash of the DB contents is included in the URL makes a lot less sense when serving files that frequently change. It's also difficult to explain to people.

  It should be optional and default to "off".

  Needed for #417 and #419

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/418/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU0MjE1NTE0MzQ=
title: Default to opening files in mutable mode, special option for immutable files
state: closed
created_at: 2019-03-15T14:39:27Z
updated_at: 2019-05-16T15:14:32Z
closed_at: 2019-05-16T15:14:31Z
author_association: OWNER
type: issue
state_reason: completed
body:
  One of the original ideas behind Datasette was that serving immutable data makes everything way easier. Two examples: You don't have to worry about SQLite concurrency and you can bundle the database inside a Docker container and deploy it to immutable hosting. See [The interesting ideas in Datasette](https://simonwillison.net/2018/Oct/4/datasette-ideas/) for more on this.

  I'm beginning to see a much stronger case for being able to serve mutable data as well.

  SQLite is actually perfectly capable of handling reads against a database that is also being written to, even if the writes are coming from another process. https://www.sqlite.org/wal.htm

  There are all kinds of interesting use-cases which Datasette is currently unsuitable for due to its insistence on immutable databases. Some examples:

  * Continually run Datasette against a SQLite database updated by another process, e.g. Firefox bookmarks
  * Projects where a cron runs every X minutes and writes new entries gathered from other sources to SQLite
  * Tail a log file, write those log updates to a SQLite file, view recent log entries in Datasette

  This is also relevant to #417, Datasette Library.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/419/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU0MjE5NzEzMzk=
title: Fix all the places that currently use .inspect() data
state: closed
created_at: 2019-03-17T20:54:37Z
updated_at: 2019-05-19T19:58:31Z
closed_at: 2019-05-02T01:13:46Z
author_association: OWNER
type: issue
state_reason: completed
body:
  See #419: if Datasette is going to work against mutable SQLite files it can no longer assume that the `.inspect()` method will have cached the correct schema for all tables in all attached databases.

  So everywhere in the code at the moment that relies on `.inspect()` data needs to be modified to use live introspection of the schema instead.

  From [a comment later on](https://github.com/simonw/datasette/issues/420#issuecomment-474398127): here are the uses I need to fix as a checklist:

  - [x] `table_exists()`
  - [x] `info["file"]` in `.execute()`
  - [x] `resolve_db_name()`
  - [x] `.database_url(database)`
  - [x] `DatabaseDownload` file path
  - [x] `sortable_columns_for_table()` uses it to find the columns in a table
  - [x] `expandable_columns()` uses it to find foreign keys
  - [x] `expand_foreign_keys()` uses it to find foreign keys
  - [x] `display_columns_and_rows()` uses it to find primary keys and foreign keys... but also has access to a cursor.description which it uses to list the columns
  - [x] `TableView.data` uses it to lookup columns and primary keys and the table_rows_count (used if the thing isn't a view) and probably a few more things, this method is huge!
  - [x] `RowView.data` uses it for primary keys
  - [x] `foreign_key_tables()` uses it for foreign keys
  - [x] `DatabaseView` list of tables
  - [x] `IndexView`

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/420/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU0MjE5ODU2ODU=
title: Documentation for ?_hash=1 and Datasette's hashed URL caching
state: closed
created_at: 2019-03-17T23:08:36Z
updated_at: 2019-05-19T05:32:37Z
closed_at: 2019-05-19T05:31:27Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Follow on from #418 - the Datasette documentation needs an entire section (probably a new page) describing exactly how the hash-in-URL caching mechanism works.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/421/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU0MjMzMTY0MDM=
title: Figure out what to do about table counts in a mutable world
state: closed
created_at: 2019-03-20T15:27:15Z
updated_at: 2019-05-02T05:43:11Z
closed_at: 2019-05-02T05:43:11Z
author_association: OWNER
type: issue
state_reason: completed
body:
  In moving away from the existing static inspect method (see #420 and #419) the biggest thing lost is full table row counts. These can be expensive against large tables, but currently Datasette runs the `count (*) from x` query once at inspection time and then reuses it for every page.

  We can run those counts with a timelimit, but this means that for larger tables we won't be able to show a count at all, which is disappointing.

  Is there a way we can find an approximate or lower bound count for a table?

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/422/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU0MjY3MjIyMDQ=
title: ?_search_col=X not reflected correctly in the UI
state: open
created_at: 2019-03-28T21:48:19Z
updated_at: 2020-11-03T19:01:59Z
author_association: OWNER
type: issue
body:
  e.g. https://latest.datasette.io/fixtures/searchable?_search_text1=barry

  ![2019-03-28 at 2 47 PM](https://user-images.githubusercontent.com/9599/55195035-84ebb800-5168-11e9-910b-fc9868bcd93e.png)

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/423/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MjY2MDM1Mzgy
title: Column types in inspected metadata
state: closed
created_at: 2019-03-31T18:46:33Z
updated_at: 2019-04-29T18:30:50Z
closed_at: 2019-04-29T18:30:46Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/424
type: pull
body:
  This PR does two things:

  * Adds the sqlite column type for each column to the inspected table info.
  * Stops binary columns from being rendered to HTML, unless a plugin handles it.

  There's a bit more detail in the changeset descriptions.

  These changes are intended as a precursor to a plugin which adds first-class support for Spatialite geographic primitives, and perhaps more useful geo-stuff.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/424/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU0MzAxMDM0NTA=
title: Submitting SQL on hide page is broken
state: closed
created_at: 2019-04-07T04:21:31Z
updated_at: 2019-04-12T05:12:13Z
closed_at: 2019-04-12T05:00:53Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Clicking the submit button here doesn't work correctly: https://3a208a4.datasette.io/fixtures?sql=select+%2A+from+compound_three_primary_keys+order+by+pk1%2C+pk2%2C+pk3+limit+101&_hide_sql=1

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/425/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MjY5MzY0OTI0
title: Upgrade to Jinja2==2.10.1
state: closed
created_at: 2019-04-10T23:03:08Z
updated_at: 2019-04-22T21:23:22Z
closed_at: 2019-04-10T23:13:31Z
author_association: OWNER
pull_request: simonw/datasette/pulls/426
type: pull
body:
  https://nvd.nist.gov/vuln/detail/CVE-2019-10906

  This is only a security issue of concern if evaluating templates from untrusted sources, which isn't something I would ever expect a Datasette user to do.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/426/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU0MzE4MDAyODY=
title: New design for facet abstraction, including querystring and metadata.json
state: closed
created_at: 2019-04-11T02:24:15Z
updated_at: 2019-05-29T21:39:12Z
closed_at: 2019-05-03T00:11:29Z
author_association: OWNER
type: issue
state_reason: completed
body:
  I need a better design for query strings for facets (and for how facets are enabled in `metadata.json`).

  Think of all of the potential kinds of facets:

  * `?_facet_array=tags` where tags is a JSON array of values
  * `_facet_date=datetimecol` - faceted by date part of a datetime
  * `_facet_bins=numeric_column` - can I do some kind of fancy binning here? Might need to take an argument
  * `?_facet_bins=numeric_column:5` - could be a way to take an argument. We’ll ignore columns with a : in their name.
  * `?_facet_json=jsoncol:jsonpath` - could use a JSON path to extract out something to facet on?
  * `?_facet_percentile=numericcolumn` - could this work?
  * `?_facet_function=column:sqlfunctionname` - maybe this could be interesting? Would allow for e.g. facet by soundex
  * `?_facet_prefix=column:prefix` - facet by terms but only if they start with a specific prefix
  * `?_facet_substring=column:3,6` - facet by a substr(column, 3, 6)

  Maybe bundling JSON in querystrings is a way to do options?

  `?_facet_distance={"latitude_column":"x",...}`

  Could detect values starting with `{` - and if for some weird reason you have a column starting with that character you can pass this instead: `?_facet_percentile={"column": "{value}"}`

  This could even be the mechanism that allows us to extend regular facets to support additional options like adding a sum or max to each one.

  Problem: it’s not obvious what the name associated with these facets should be. What if one column is faceted multiple times using multiple facet variants?

  Maybe just number them? name1=… name2=… etc?

  Other option is to use Solr style querystring syntax for notation. Solr does this: `?f.price.facet.range.gap=100&f.age.facet.range.gap=10`

  So how about this:

  `?_facet_range=age&_facet_range.span=5`

  Related: #359

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/427/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU0MzIyMTc2MjU=
title: Incorrect help text for enable-fts command
state: closed
created_at: 2019-04-11T19:46:44Z
updated_at: 2019-05-25T00:44:31Z
closed_at: 2019-05-25T00:44:31Z
author_association: OWNER
type: issue
state_reason: completed
body:
  I clearly copied-and-pasted this from the `tables` command without updating it:

  https://github.com/simonw/sqlite-utils/blob/0b1af42ead3b3902347951180b3364ce1942da6e/sqlite_utils/cli.py#L216-L222

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/19/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU0MzIzNzE3NjI=
title: Make ?_fts_table=x and ?_fts_pk=y available as URL parameters on table view
state: closed
created_at: 2019-04-12T03:30:55Z
updated_at: 2019-04-12T04:30:29Z
closed_at: 2019-04-12T04:21:25Z
author_association: OWNER
type: issue
state_reason: completed
body:
  These can currently only be set using `metadata.json`: https://datasette.readthedocs.io/en/0.27/full_text_search.html#configuring-full-text-search-for-a-table-or-view

  There's no reason not to support these as URL parameters as well. That way it would be easy to use FTS search against a view without having to use `metadata.json`.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/428/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU0MzI2MzY0MzI=
title: ?_where=sql-fragment parameter for table views
state: closed
created_at: 2019-04-12T15:58:51Z
updated_at: 2019-04-15T10:48:01Z
closed_at: 2019-04-13T01:37:25Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Only available if arbitrary SQL is enabled (the default).

  `?_where=id in (1,2,3)&_where=id in (select tag_id from tags)`

  Allows any table (or view) page to have arbitrary additional `extra_where` clauses defined using the URL!

  This would be extremely useful for building JavaScript applications against the Datasette API that only need on extra tiny bit of SQL but still want to benefit from other table view features like faceting.

  Would be nice if this could take `:named` parameters and have them filled in via querystring as well.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/429/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU0MzI3Mjc2ODU=
title: JSON column values get extraneously quoted
state: closed
created_at: 2019-04-12T20:15:30Z
updated_at: 2019-05-25T00:57:19Z
closed_at: 2019-05-25T00:57:19Z
author_association: NONE
type: issue
state_reason: completed
body:
  If the input to `sqlite-utils insert` includes a column that is a JSON array or object, `sqlite-utils query` will introduce an extra level of quoting on output:

  ```
  # echo '[{"key": ["one", "two", "three"]}]' | sqlite-utils insert t.db t -

  # sqlite-utils t.db 'select * from t'
  [{"key": "[\"one\", \"two\", \"three\"]"}]

  # sqlite3 t.db 'select * from t'
  ["one", "two", "three"]
  ```

  This might require an imperfect solution, since sqlite3 doesn't have a JSON type. Perhaps fields that start with `["` or `{"` and end with `"]` or `"}` could be detected, with a flag to turn off that behavior for weird text fields (or vice versa).

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/20/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MjcwMTkxMDg0
title: ?_where= parameter on table views, closes #429
state: closed
created_at: 2019-04-13T01:15:09Z
updated_at: 2019-04-13T01:37:23Z
closed_at: 2019-04-13T01:37:23Z
author_association: OWNER
pull_request: simonw/datasette/pulls/430
type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/430/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU0MzI4NzAyNDg=
title: Datasette doesn't reload when database file changes
state: closed
created_at: 2019-04-13T16:50:43Z
updated_at: 2019-05-02T05:13:55Z
closed_at: 2019-05-02T05:13:54Z
author_association: CONTRIBUTOR
type: issue
state_reason: completed
body:
  My understanding of the `--reload` option was that if the database file changed `datasette` would automatically reload.

  I'm running on a Mac and from the `datasette` UI queries don't seem to be picking up data in a newly changed db (I checked the db timestamp - it certainly updated).

  I was also expecting to see some sort of log statement in the datasette logging to say that it had detected a file change and restarted, but don't see anything there?

  Will try to check on an Ubuntu box when I get a chance to see if this is a Mac thing.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/431/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MjcwMjUxMDIx
title: Refactor facets to a class and new plugin, refs #427
state: closed
created_at: 2019-04-13T20:04:45Z
updated_at: 2019-05-03T00:04:24Z
closed_at: 2019-05-03T00:04:24Z
author_association: OWNER
pull_request: simonw/datasette/pulls/432
body: WIP for #427
type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/432/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU0MzMyOTc5ODk=
title: ?column__in=value1,value2,value3 filter
state: closed
created_at: 2019-04-15T13:58:24Z
updated_at: 2019-04-15T23:00:20Z
closed_at: 2019-04-15T23:00:20Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Support for the SQL `where column in (...)` construct, inspired by the new design for facet configuration in #427

  `?column__in=value1,value2,value3` will map to `where column in ("value1", "value2", "value3")`

  If comma separation won't work (because the values themselves contain commas) you can do this instead:

  `?column__in=["value1","value2","value3,with-comma"]`

  See also #288

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/433/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0MjcxMzM4NDA1
title: "datasette publish cloudrun" command to publish to Google Cloud Run
state: closed
created_at: 2019-04-17T14:41:18Z
updated_at: 2019-05-03T21:50:44Z
closed_at: 2019-05-03T13:59:02Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/434
type: pull
body:
  This is a very rough draft to start a discussion on a possible datasette cloud run publish plugin (see issue #400).

  The main change was to dynamically set the listening port in `make_dockerfile` to satisfy cloud run's [requirements](https://cloud.google.com/run/docs/reference/container-contract).

  This was done by running `datasette` through `sh` to get environment variable substitution. Not sure if that's the right approach?

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/434/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU0MzU1MzEwMzQ=
title: Tracing support for seeing what SQL queries were executed
state: closed
created_at: 2019-04-21T17:37:37Z
updated_at: 2019-05-11T20:32:21Z
closed_at: 2019-05-11T19:07:42Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Features like faceting, foreign key expansions and now the inspect-less index view mean Datasette can end up executing a surprisingly large number of SQL queries to render a single page.

  Past experience with projects like [tikbar](https://github.com/simonw/tikibar) have shown that being able to see what actually went into rendering a page can be critical for optimizing performance and generally understanding how everything works.

  Support a tracing mode (probably via a `?_trace=1` querystring) which adds information about what is actually going on to both the HTML and the JSON.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/435/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU0MzU4MTkzMjE=
title: 400 Error when trying to register new user via https://publish.datasettes.com/
state: closed
created_at: 2019-04-22T17:55:00Z
updated_at: 2021-01-04T20:15:42Z
closed_at: 2021-01-04T20:15:41Z
author_association: NONE
type: issue
state_reason: completed
body:
  Behavior: When registering a new user via Zeit - confirmation is sent and screen acknowledges registered user... When clicking grant access the next screen is a white 400 error message. 

  Replicated: Chrome and Firefox; 2 different email accounts

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/436/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0Mjc0MTc0NjE0
title: Add inspect and prepare_sanic hooks
state: closed
created_at: 2019-04-28T11:53:34Z
updated_at: 2019-06-24T16:38:57Z
closed_at: 2019-06-24T16:38:56Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/437
type: pull
body:
  This adds two new plugin hooks:

  The `inspect` hook allows plugins to add data to the inspect dictionary.

  The `prepare_sanic` hook allows plugins to hook into the web router. I've attached a warning to this hook in the docs in light of #272 but I want this hook now...

  On quick inspection, I don't think it's worthwhile to try and make this hook independent of the web framework (but it looks like Starlette would make the hook implementation a bit nicer).

  Ref #14

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/437/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU0MzgyMDA1Mjk=
title: Plugins are loaded when running pytest
state: closed
created_at: 2019-04-29T08:25:58Z
updated_at: 2019-05-02T05:09:18Z
closed_at: 2019-05-02T05:09:11Z
author_association: CONTRIBUTOR
type: issue
state_reason: completed
body:
  If I have a datasette plugin installed on my system, its hooks are called when running the main datasette tests. This is probably undesirable, especially with the inspect hook in #437, as the plugin may rely on inspected state that the tests don't know about.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/438/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0Mjc0MzEzNjI1
title: [WIP] Add primary key to the extra_body_script hook arguments
state: closed
created_at: 2019-04-29T10:08:23Z
updated_at: 2019-05-01T09:58:32Z
closed_at: 2019-05-01T09:58:30Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/439
type: pull
body:
  This allows the row to be identified on row pages. The context here is that I want to access the row's data to plot it on a map.

  I considered passing the entire template context through to the hook function. This would expose the actual row data and potentially avoid a further fetch request in JS, but it does make the plugin API a lot more leaky. 

  (At any rate, using the selected row data is tricky in my case because of Spatialite's infuriating custom binary representation...)

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/439/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU0MzgyNTk5NDE=
title: Plugin hook for additional data export formats
state: closed
created_at: 2019-04-29T11:01:39Z
updated_at: 2019-05-01T23:01:57Z
closed_at: 2019-05-01T23:01:57Z
author_association: CONTRIBUTOR
type: issue
state_reason: completed
body:
  It would be nice to have a simple way for plugins to provide additional data export formats. Might require a bit of work on the internals. I can work around this at a lower level with the `prepare_sanic` hook from #437 in the mean time.

  I guess plugins should be able to register a function which takes a row or list of rows and returns the rendered data. They'll also need to provide a file extension and probably a Content-Type.

  Datasette could then automatically include this format in the list of export formats on each page.

  Looks like this is related to #119.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/440/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0Mjc0NDY4ODM2
title: Add register_output_renderer hook
state: closed
created_at: 2019-04-29T18:03:21Z
updated_at: 2019-05-01T23:01:57Z
closed_at: 2019-05-01T23:01:57Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/441
type: pull
body:
  This changeset refactors out the JSON renderer and then adds a hook and
  dispatcher system to allow custom output renderers to be registered.

  The CSV output renderer is untouched because supporting streaming
  renderers through this system would be significantly more complex, and
  probably not worthwhile.

  We can't simply allow hooks to be called at request time because we need
  a list of supported file extensions when the request is being routed in
  order to resolve ambiguous database/table names. So, renderers need to
  be registered at startup.

  I've tried to make this API independent of Sanic's request/response
  objects so that this can remain stable during the switch to ASGI. I'm
  using dictionaries to keep it simple and to make adding additional
  options in the future easy.

  Fixes #440

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/441/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0Mjc0NDc4NzYx
title: Suppress rendering of binary data
state: closed
created_at: 2019-04-29T18:36:41Z
updated_at: 2019-05-03T18:26:48Z
closed_at: 2019-05-03T16:44:49Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/442
type: pull
body:
  Binary columns (including spatialite geographies) get shown as ugly
  binary strings in the HTML by default. Nobody wants to see that mess.

  Show the size of the column in bytes instead. If you want to decode
  the binary data, you can use a plugin to do it.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/442/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0Mjc1Mjc1NjEw
title: Pass view_name to extra_body_script hook
state: closed
created_at: 2019-05-02T08:38:36Z
updated_at: 2019-05-03T13:12:20Z
closed_at: 2019-05-03T13:12:20Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/443
type: pull
body:
  At the moment it's not easy to tell whether the hook is being called
  in (for example) the row or table view, as in both cases the
  `database` and `table` parameters are provided.

  This passes the `view_name` added in #441 to the `extra_body_script`
  hook.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/443/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0Mjc1MjgxMzA3
title: Add a max-line-length setting for flake8
state: closed
created_at: 2019-05-02T08:58:57Z
updated_at: 2019-05-04T09:44:48Z
closed_at: 2019-05-03T13:11:28Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/444
type: pull
body:
  This stops my automatic editor linting from flagging lines which are too
  long. It's been lingering in my checkout for ages.

  160 is an arbitrary large number - we could alter it if we have any
  opinions (but I find the line length limit to be my least favourite part
  of PEP8).

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/444/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0Mjc1NTU4NjEy
title: Extract facet code out into a new plugin hook, closes #427
state: closed
created_at: 2019-05-03T00:02:41Z
updated_at: 2019-05-03T18:17:18Z
closed_at: 2019-05-03T00:11:27Z
author_association: OWNER
pull_request: simonw/datasette/pulls/445
type: pull
body:
  Datasette previously only supported one type of faceting: exact column value counting.

  With this change, faceting logic is extracted out into one or more separate classes which can implement other patterns of faceting - this is discussed in #427, but potential upcoming facet types include facet-by-date, facet-by-JSON-array, facet-by-many-2-many and more.

  A new plugin hook, register_facet_classes, can be used by plugins to add in additional facet classes.

  Each class must implement two methods: suggest(), which scans columns in the table to decide if they might be worth suggesting for faceting, and facet_results(), which executes the facet operation and returns results ready to be displayed in the UI.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/445/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU0NDAxMzQ3MTQ=
title: Define mechanism for plugins to return structured data
state: closed
created_at: 2019-05-03T17:00:16Z
updated_at: 2020-10-02T00:08:54Z
closed_at: 2020-10-02T00:08:47Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Several plugin hooks now expect plugins to return data in a specific shape -  notably the new output format hook and the custom facet hook.

  These use Python dictionaries right now but that's quite error prone: it would be good to have a mechanism that supported a more structured format.

  Full list of current hooks is here: https://datasette.readthedocs.io/en/latest/plugins.html#plugin-hooks

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/446/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0Mjc1ODAxNDYz
title: Use dist: xenial and python: 3.7 on Travis
state: closed
created_at: 2019-05-03T18:07:07Z
updated_at: 2019-05-03T18:17:05Z
closed_at: 2019-05-03T18:16:53Z
author_association: OWNER
pull_request: simonw/datasette/pulls/447
type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/447/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU0NDAyMjI3MTk=
title: _facet_array should work against views
state: closed
created_at: 2019-05-03T21:08:04Z
updated_at: 2021-11-16T01:32:05Z
closed_at: 2021-11-16T01:19:40Z
author_association: OWNER
type: issue
state_reason: completed
body:
  I created this view: https://json-view-facet-bug-demo-j7hipcg4aq-uc.a.run.app/russian-ads-8dbda00/ads_with_targets

  ```
  CREATE VIEW ads_with_targets as select ads.*, json_group_array(targets.name) as target_names from ads
    join ad_targets on ad_targets.ad_id = ads.id
    join targets on ad_targets.target_id = targets.id
    group by ad_targets.ad_id
  ```

  When I try to apply faceting by array it appears to work at first: https://json-view-facet-bug-demo-j7hipcg4aq-uc.a.run.app/russian-ads/ads_with_targets?_facet_array=target_names

  But actually it's doing the wrong thing - the SQL for the facets uses rowid, but rowid is not present on views at all! These results are incorrect, and clicking to select a facet will fail to produce any rows: https://json-view-facet-bug-demo-j7hipcg4aq-uc.a.run.app/russian-ads/ads_with_targets?_facet_array=target_names&target_names__arraycontains=people_who_match%3Ainterests%3AAfrican-American+Civil+Rights+Movement+%281954%E2%80%9468%29

  Here's the SQL it should be using when you select a facet (note that it does not use a rowid):

  https://json-view-facet-bug-demo-j7hipcg4aq-uc.a.run.app/russian-ads?sql=select+*+from+ads_with_targets+where+id+in+%28%0D%0A++++++++++++select+ads_with_targets.id+from+ads_with_targets%2C+json_each%28ads_with_targets.target_names%29+j%0D%0A++++++++++++where+j.value+%3D+%3Ap0%0D%0A++++++++%29+limit+101&p0=people_who_match%3Ainterests%3ABlack+%28Color%29

  So we need to do something a lot smarter here. I'm not sure what the fix will look like, or even if it's feasible given that views don't have a rowid to hook into so the JSON faceting SQL may have to be completely rewritten.

  ```
  datasette publish cloudrun \
      russian-ads.db \
      --name json-view-facet-bug-demo \
      --branch master \
      --extra-options "--config sql_time_limit_ms:5000 --config facet_time_limit_ms:5000"
  ```

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/448/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0Mjc1ODYxNTU5
title: Apply black to everything
state: closed
created_at: 2019-05-03T21:57:26Z
updated_at: 2019-05-04T02:17:14Z
closed_at: 2019-05-04T02:15:15Z
author_association: OWNER
pull_request: simonw/datasette/pulls/449
type: pull
body:
  I've been hesitating on this for literally months, because I'm not at all excited about the giant diff that will result. But I've been using black on many of my other projects (most actively [sqlite-utils](https://github.com/simonw/sqlite-utils)) and the productivity boost is undeniable: I don't have to spend a single second thinking about code formatting any more!

  So it's worth swallowing the one-off pain and moving on in a new, black-enabled world.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/449/reactions", "total_count": 4, "+1": 0, "-1": 0, "laugh": 0, "hooray": 4, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0Mjc1OTA5MTk3
title: Coalesce hidden table count to 0
state: closed
created_at: 2019-05-04T09:37:10Z
updated_at: 2019-05-11T18:10:09Z
closed_at: 2019-05-11T18:10:09Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/450
type: pull
body:
  For some reason I'm hitting a `None` here with a FTS table. I'm not
  entirely sure why but this makes the logic work the same as with
  non-hidden tables.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/450/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU0NDAzMTMyMDk=
title: Update README
state: closed
created_at: 2019-05-04T11:26:07Z
updated_at: 2019-05-19T22:23:43Z
closed_at: 2019-05-19T22:23:43Z
author_association: OWNER
type: issue
state_reason: completed
body:
  The README is quite out of date now. It includes out-dated copies of help files, promotes the old Zeit Now integration and duplicates a lot of material from the docs.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/451/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0Mjc1OTIzMDY2
title: SQL builder utility classes
state: open
created_at: 2019-05-04T13:57:47Z
updated_at: 2019-05-04T14:03:04Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/452
type: pull
body:
  This adds a straightforward set of classes to aid in the construction of
  SQL queries.

  My plan for this was to allow plugins to manipulate the
  Datasette-generated SQL in a more structured way. I'm not sure that's
  going to work, but I feel like this is still a step forward - it
  reduces the number of intermediate variables in `TableView.data` which
  aids readability, and also factors out a lot of the boring string
  concatenation.

  There are a fair number of minor structure changes in here too as I've
  tried to make the ordering of `TableView.data` a bit more logical. As
  far as I can tell, I haven't broken anything...

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/452/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU0NDAzMzI2MjE=
title: Error pages do not return CORS header with --cors
state: closed
created_at: 2019-05-04T15:07:44Z
updated_at: 2019-05-05T12:24:24Z
closed_at: 2019-05-05T12:11:33Z
author_association: OWNER
type: issue
state_reason: completed
body:
  This is very confusing. It means that if you send invalid SQL you will get back a CORS error, because the resulting 400 page cannot be accessed via JavaScript.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/453/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU0NDA0MzcwMzc=
title: Plugin for allowing CORS from specified hosts
state: closed
created_at: 2019-05-05T12:05:02Z
updated_at: 2019-10-03T23:59:57Z
closed_at: 2019-10-03T23:59:56Z
author_association: OWNER
type: issue
state_reason: completed
body:
  It would be useful if Datasette could be configured to allow CORS requests from one or more origins, as opposed to only allowing either none or `"*"`.

  This is slightly tricky because the `Access-Control-Allow-Origin: https://foo.example` header is only allowed to return one value per request - and according to https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS "The Access-Control-Allow-Origin header should contain the value that was sent in the request's Origin header."

  This means the application code needs to have a whitelist of allowed hosts and code that dynamically changes the outgoing `Access-Control-Allow-Origin` header based on the `Origin` header from the incoming request.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/454/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU0NDE4NTg3NDc=
title: Hidden tables shown on the index page
state: closed
created_at: 2019-05-08T18:02:13Z
updated_at: 2019-05-14T15:49:29Z
closed_at: 2019-05-14T15:48:08Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Minor bug in master right now.

  https://csvconf.now.sh/

  <img width="521" alt="csv_conf_v4__csvconf" src="https://user-images.githubusercontent.com/9599/57397073-b2bd1700-7180-11e9-82b8-677338c50987.png">

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/455/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU0NDIzMjc1OTI=
title: Installing installs the tests package
state: closed
created_at: 2019-05-09T16:35:16Z
updated_at: 2020-07-24T20:39:54Z
closed_at: 2020-07-24T20:39:54Z
author_association: CONTRIBUTOR
type: issue
state_reason: completed
body:
  Because `setup.py` uses `find_packages` and `tests` is on the top-level, `pip install datasette` will install a top-level package called `tests`, which is probably not desired behavior.

  The offending line is here:
  https://github.com/simonw/datasette/blob/bfa2ae0d16d39bb82dbe4da4f3fdc3c7f6257418/setup.py#L40

  And only `pip uninstall datasette` with a conflicting package would warn you by default; apparently another package had the same problem, which is why I get this message when uninstalling:

  ```
  $ pip uninstall datasette
  Uninstalling datasette-0.27:
    Would remove:
      /usr/local/bin/datasette
      /usr/local/lib/python3.7/site-packages/datasette-0.27.dist-info/*
      /usr/local/lib/python3.7/site-packages/datasette/*
      /usr/local/lib/python3.7/site-packages/tests/*
    Would not remove (might be manually added):
      [ .. snip .. ]
  Proceed (y/n)? 
  ```

  This should be a relatively simple fix, and I could drop a PR if desired!

  Cheers

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/456/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU0NDIzMzA1NjQ=
title: Ability to "publish cloudrun" with no user input
state: closed
created_at: 2019-05-09T16:42:51Z
updated_at: 2019-05-09T19:41:31Z
closed_at: 2019-05-09T16:45:08Z
author_association: OWNER
type: issue
state_reason: completed
body:
  If you attempt to deploy a new version of a cloudrun deployment, the script currently pauses and asks for user input for the service name like this:

  ```77d4d7de-3dfc-4acc-9a23-efe16230f318  2019-05-09T15:01:48+00:00  52S       gs://datasette-222320_cloudbuild/source/1557414063.1-3a82df8096e9434b93511b0588d8d155.tgz  gcr.io/datasette-222320/sf-trees (+1 more)  SUCCESS
  Service name: (sf-trees):  USER INPUT REQUIRED HERE
  Deploying container to Cloud Run service [sf-trees] in project [datasette-222320] region [us-central1]
  ✓ Deploying... Done.                                                                                                                                                     
    ✓ Creating Revision...                                                                                                                                                 
    ✓ Routing traffic...                                                                                                                                                   
    ✓ Setting IAM Policy...                                                                                                                                                
  ```
  This is incompatible with running under CI.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/457/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0Mjc3NTI0MDcy
title: setup: add tests to package exclusion
state: closed
created_at: 2019-05-09T19:47:21Z
updated_at: 2020-07-21T01:14:42Z
closed_at: 2019-05-10T01:54:51Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/458
body:
  This PR fixes #456 by adding `tests` to the package exclusion list.

  Cheers

type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/458/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU0NDMwMjAwNDg=
title: Fix the "datasette now publish ... --alias=x" option
state: closed
created_at: 2019-05-11T17:48:40Z
updated_at: 2019-05-11T20:22:08Z
closed_at: 2019-05-11T20:22:08Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Now have deprecated the mechanism we were using for this - running `now alias` without any parameters - in favour of something new:

  https://zeit.co/blog/automatic-aliasing

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/459/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU0NDMwMjA4MTA=
title: Design changes to homepage to support mutable files
state: closed
created_at: 2019-05-11T17:58:05Z
updated_at: 2019-05-16T03:34:09Z
closed_at: 2019-05-16T03:24:16Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Needed for #419 - since we can now start up Datasette with a whole bunch of large connected databases that are mutable we can no longer guarantee a quick count of rows across all of the tables.

  New proposed homepage tweaks:

  <img width="872" alt="Datasette__blah__commits__events__fixtures__fixtures_modifyme__modme__nerds__out__russian-ads__salaries__sf-trees__sortable__this-db-will-change" src="https://user-images.githubusercontent.com/9599/57573373-a1237b80-73db-11e9-9b7c-1842729bcde6.png">

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/460/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU0NDMwMjE1MDk=
title: Paginate + search for databases/tables on the homepage
state: open
created_at: 2019-05-11T18:05:34Z
updated_at: 2020-12-17T22:14:46Z
author_association: OWNER
type: issue
body:
  Split out from #460 - in order to support large numbers of connected databases the homepage needs to be paginated.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/461/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU0NDMwMjMzMDg=
title: Replace most of `.inspect()` (and `datasette inspect`) with table counting
state: closed
created_at: 2019-05-11T18:26:06Z
updated_at: 2019-05-16T14:31:05Z
closed_at: 2019-05-16T14:31:05Z
author_association: OWNER
type: issue
state_reason: completed
body:
  This is the last part of #419 - with the move to supporting mutable databases by default, the inspect-data mechanism currently in use no-longer makes much sense.

  The one optimization I think it's worth keeping for databases opened in immutable mode is the cached table counts. I think `datasette inspect` should cut down to only counting the rows in the tables - the other things done by inspect (figuring out columns, foreign key relationships, FTS etc) should all be fast enough that they can be reliably performed at runtime even against large databases.

  If performing them at run-time has performance issues, I would rather cache those results internally within Datasette after they are first calculated than continue to support them in the `datasette inspect` command - to keep things simpler.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/462/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


