issues

Table actions

24 rows where "created_at" is on date 2017-10-23 and user = 9599 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: milestone, comments, body, created_at (date), updated_at (date), closed_at (date)

state 2

type 1

repo 1

id node_id number title user state locked assignee milestone comments created_at updated_at ▲ closed_at author_association pull_request body repo type active_lock_reason performed_via_github_app reactions draft state_reason
267739593 MDU6SXNzdWUyNjc3Mzk1OTM= 18 See if I can get a websockets interface working simonw 9599 closed 0     1 2017-10-23T16:46:41Z 2021-01-04T20:05:52Z 2021-01-04T20:05:48Z OWNER  

Since I am already running on Sanic, how hard would it be to add a websocket ebdpoint that lets you talk to sqlite interactively?

Could this be used to efficiently support streaming in answers to giant queries?

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/18/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
267769431 MDU6SXNzdWUyNjc3Njk0MzE= 22 Refactor to use class based views simonw 9599 closed 0   Ship first public release 2857392 0 2017-10-23T18:26:22Z 2019-05-27T20:05:56Z 2017-10-24T02:25:53Z OWNER   datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/22/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
267707940 MDU6SXNzdWUyNjc3MDc5NDA= 14 Datasette Plugins simonw 9599 closed 0     22 2017-10-23T15:15:28Z 2019-05-13T18:58:20Z 2019-05-13T18:58:19Z OWNER  

It would be neat if additional functionality could be opted-in to the system in the form of easy-to-add plugins, hosted as separate packages. First example: a Google Analytics plugin, which adds GA tracking code with your tracking ID to the web interface for your dataset.

This may be an opportunity to experiment with entry points: http://amir.rachum.com/blog/2017/07/28/python-entry-points/

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/14/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
267515678 MDU6SXNzdWUyNjc1MTU2Nzg= 3 Make individual column valuables addressable, with smart content types simonw 9599 open 0     1 2017-10-23T01:11:32Z 2017-12-10T03:11:58Z   OWNER  

Some SQLite databases embed images in columns. It would be cool if these had URLs.

/database-name-7sha256/table-name/compound-pk/column
/database-name-7sha256/table-name/compound-pk/column.json
/database-name-7sha256/table-name/compound-pk/column.png
/database-name-7sha256/table-name/compound-pk/column.gif
/database-name-7sha256/table-name/compound-pk/column.txt

The one without an explicit file extension auto-detects the correct extension.

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/3/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
    
267759136 MDU6SXNzdWUyNjc3NTkxMzY= 20 Config file with support for defining canned queries simonw 9599 closed 0 simonw 9599 Custom templates edition 2949431 9 2017-10-23T17:53:06Z 2017-12-05T19:05:35Z 2017-12-05T17:44:09Z OWNER  

Probably using YAML because then we get support for multiline strings:

bats:
  db: bats.sqlite3
  name: "Bat sightings"
  queries:
    specific_row: |
      select * from Bats
      where a = 1;
datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/20/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
267523511 MDU6SXNzdWUyNjc1MjM1MTE= 12 Make it so you can override templates simonw 9599 closed 0   Custom templates edition 2949431 1 2017-10-23T02:25:35Z 2017-11-30T16:42:46Z 2017-11-30T16:38:34Z OWNER  

The app will ship with default templates but, just like with the Django admin, you will be able to override them using either explicit configuration settings or just by dropping in templates with certain file names.

Template inheritance should work here, both allowing you to override just the base template and allowing you to customize tiny bits of others.

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/12/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
267542338 MDU6SXNzdWUyNjc1NDIzMzg= 13 Add a syntax highlighting SQL editor simonw 9599 closed 0     1 2017-10-23T05:03:33Z 2017-11-15T02:04:51Z 2017-11-15T02:04:51Z OWNER  

https://ace.c9.io/#nav=embedding looks like a good option

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/13/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
267726219 MDU6SXNzdWUyNjc3MjYyMTk= 16 Default HTML/CSS needs to look reasonable and be responsive simonw 9599 closed 0   Ship first public release 2857392 6 2017-10-23T16:05:22Z 2017-11-11T20:19:07Z 2017-11-11T20:19:07Z OWNER  

Version one should have the following characteristics:

  • Looks OK
  • Works great on mobile
  • Loads extremely fast
  • No JavaScript! At least not in v1.
 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/16/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
267769034 MDU6SXNzdWUyNjc3NjkwMzQ= 21 Use Sanic configuration mechanism simonw 9599 closed 0   v1 stretch goals 2859414 1 2017-10-23T18:25:14Z 2017-11-10T20:45:42Z 2017-11-10T20:45:42Z OWNER   datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/21/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
267516066 MDU6SXNzdWUyNjc1MTYwNjY= 5 Implement sensible query pagination simonw 9599 closed 0   Ship first public release 2857392 3 2017-10-23T01:16:00Z 2017-11-10T20:41:39Z 2017-11-10T20:41:39Z OWNER  
datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/5/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
267517381 MDU6SXNzdWUyNjc1MTczODE= 10 Set up Travis simonw 9599 closed 0   v1 stretch goals 2859414 1 2017-10-23T01:29:07Z 2017-11-04T23:48:57Z 2017-11-04T23:48:57Z OWNER  
datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/10/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
267741262 MDU6SXNzdWUyNjc3NDEyNjI= 19 Efficient url for downloading the raw database file simonw 9599 closed 0   Ship first public release 2857392 1 2017-10-23T16:52:17Z 2017-10-25T15:21:16Z 2017-10-25T15:19:37Z OWNER   datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/19/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
267788884 MDU6SXNzdWUyNjc3ODg4ODQ= 23 Support Django-style filters in querystring arguments simonw 9599 closed 0   Ship first public release 2857392 6 2017-10-23T19:29:42Z 2017-10-25T04:23:03Z 2017-10-25T04:23:02Z OWNER  

e.g

/database/table?name__contains=Simon&age__gte=4

Same format as Django: double underscore as the split.

If you need to match against a column that happens to contain a double underscore in its official name, do this:

/database/table?weird__column__exact=Simon

__exact is the default operation if none is supplied.

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/23/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
267516329 MDU6SXNzdWUyNjc1MTYzMjk= 6 Better JSON response options simonw 9599 closed 0   Ship first public release 2857392 0 2017-10-23T01:18:47Z 2017-10-24T15:07:58Z 2017-10-24T15:07:58Z OWNER  

Default returns this:

{
    “Columns”: [“id”, “name”, “age”],
    “Rows”: [
         [45, “Simon”, 36]
    ]
}

.jsono instead returns a list of objects each duplicating the headers in its keys.

They both probably share the same pagination mechanism so it might not be a jsono flat list.

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/6/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
267828746 MDU6SXNzdWUyNjc4Mjg3NDY= 24 Implement full URL design simonw 9599 closed 0   Ship first public release 2857392 2 2017-10-23T21:49:05Z 2017-10-24T14:12:00Z 2017-10-24T14:12:00Z OWNER  

Full URL design:

/database-name
/database-name.json
/database-name-7sha256
/database-name-7sha256.json
/database-name/table-name
/database-name/table-name.json
/database-name-7sha256/table-name
/database-name-7sha256/table-name.json
/database-name-7sha256/table-name/compound-pk
/database-name-7sha256/table-name/compound-pk.json
datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/24/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
267513424 MDU6SXNzdWUyNjc1MTM0MjQ= 1 Addressable pages for every row in a table simonw 9599 closed 0   Ship first public release 2857392 6 2017-10-23T00:44:16Z 2017-10-24T14:11:04Z 2017-10-24T14:11:03Z OWNER   
/database-name-7sha256/table-name/compound-pk
/database-name-7sha256/table-name/compound-pk.json

Tricky part will be figuring out what the private key is - especially since it could be a compound primary key and it might involve different data types.

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
267517348 MDU6SXNzdWUyNjc1MTczNDg= 9 Initial test suite simonw 9599 closed 0   Ship first public release 2857392 2 2017-10-23T01:28:46Z 2017-10-24T05:55:33Z 2017-10-24T05:55:33Z OWNER  
datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/9/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
267515836 MDU6SXNzdWUyNjc1MTU4MzY= 4 Make URLs immutable simonw 9599 closed 0   Ship first public release 2857392 8 2017-10-23T01:13:30Z 2017-10-24T02:38:24Z 2017-10-24T02:38:24Z OWNER  

Absolutely everything should have a far-future expires header

Part of the URL will be the truncated sha1 hash of the database file itself, calculated at build time

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/4/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
267516650 MDU6SXNzdWUyNjc1MTY2NTA= 7 Framework where by every page is JSON plus a template simonw 9599 closed 0   Ship first public release 2857392 1 2017-10-23T01:22:03Z 2017-10-24T02:27:25Z 2017-10-24T02:27:25Z OWNER  

Every single page of my interface should be implemented as a function that returns JSON.

I can then build my jinja templates on top of the exact data that would be returned by the API version.

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/7/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
267732005 MDU6SXNzdWUyNjc3MzIwMDU= 17 In development mode, should still pick up new .db files simonw 9599 closed 0   Ship first public release 2857392 1 2017-10-23T16:22:40Z 2017-10-24T02:26:48Z 2017-10-24T02:26:47Z OWNER  

Follow on from #11

datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/17/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
267713226 MDU6SXNzdWUyNjc3MTMyMjY= 15 Support multiple databases simonw 9599 closed 0   Ship first public release 2857392 0 2017-10-23T15:29:51Z 2017-10-24T02:01:38Z 2017-10-24T02:01:38Z OWNER  

I'm going to loop through every database file in the app root directory and bundle all of them.

Each one will be accessible at /databasename

Note this is without the file extension, and we will disallow multiple files with the same name but different extensions.

Supported extensions to start with will be .db and .sqlite and .sqlite3

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/15/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
267522549 MDU6SXNzdWUyNjc1MjI1NDk= 11 Code that generates compile-time properties about the database simonw 9599 closed 0   Ship first public release 2857392 1 2017-10-23T02:18:24Z 2017-10-23T16:04:23Z 2017-10-23T16:04:23Z OWNER  

At a minimum this will include:

  • sha hash of each database file
  • list of tables with row counts for each database file
 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/11/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
267517314 MDU6SXNzdWUyNjc1MTczMTQ= 8 Attempting an INSERT or UPDATE should return a sane error message simonw 9599 closed 0   Ship first public release 2857392 1 2017-10-23T01:28:25Z 2017-10-23T15:28:12Z 2017-10-23T15:28:08Z OWNER  
datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/8/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
267513523 MDU6SXNzdWUyNjc1MTM1MjM= 2 Initial proof-of-concept simonw 9599 closed 0   Ship first public release 2857392 0 2017-10-23T00:45:37Z 2017-10-23T01:26:39Z 2017-10-23T00:45:53Z OWNER   datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issues] (
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [number] INTEGER,
   [title] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [state] TEXT,
   [locked] INTEGER,
   [assignee] INTEGER REFERENCES [users]([id]),
   [milestone] INTEGER REFERENCES [milestones]([id]),
   [comments] INTEGER,
   [created_at] TEXT,
   [updated_at] TEXT,
   [closed_at] TEXT,
   [author_association] TEXT,
   [pull_request] TEXT,
   [body] TEXT,
   [repo] INTEGER REFERENCES [repos]([id]),
   [type] TEXT
, [active_lock_reason] TEXT, [performed_via_github_app] TEXT, [reactions] TEXT, [draft] INTEGER, [state_reason] TEXT);
CREATE INDEX [idx_issues_repo]
                ON [issues] ([repo]);
CREATE INDEX [idx_issues_milestone]
                ON [issues] ([milestone]);
CREATE INDEX [idx_issues_assignee]
                ON [issues] ([assignee]);
CREATE INDEX [idx_issues_user]
                ON [issues] ([user]);