issue_comments

Table actions

20 rows where "updated_at" is on date 2017-11-19 and user = 9599 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: issue_url, created_at (date), updated_at (date)

user 1

  • simonw · 20 ✖

author_association 1

id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions issue performed_via_github_app
345559864 https://github.com/simonw/datasette/issues/86#issuecomment-345559864 https://api.github.com/repos/simonw/datasette/issues/86 MDEyOklzc3VlQ29tbWVudDM0NTU1OTg2NA== simonw 9599 2017-11-19T23:35:48Z 2017-11-19T23:35:48Z OWNER

I need a nicer abstraction around the concept of filters. It needs to be able to:

  • convert querystring parameters into filters
  • convert filters into a querystring
  • iterate through currently applied filters
  • convert selected filters into a human description (e.g. for a title)
  • expand filters that involve a foreign key
  • add filters
  • remove filters
  • define different types of filters

It should replace my current build_where_clauses implementation, in particular this bit:

https://github.com/simonw/datasette/blob/a5881e105a02830d26f07e98177248d5910893da/datasette/utils.py#L38-L56

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Filter UI on table page 273703829  
345552500 https://github.com/simonw/datasette/issues/122#issuecomment-345552500 https://api.github.com/repos/simonw/datasette/issues/122 MDEyOklzc3VlQ29tbWVudDM0NTU1MjUwMA== simonw 9599 2017-11-19T21:47:27Z 2017-11-19T21:47:27Z OWNER

To start with, I could just ditch the .jsono in favour of the new _shape argument.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Redesign JSON output, ditch jsono, offer variants controlled by parameter instead 275092453  
345552440 https://github.com/simonw/datasette/issues/122#issuecomment-345552440 https://api.github.com/repos/simonw/datasette/issues/122 MDEyOklzc3VlQ29tbWVudDM0NTU1MjQ0MA== simonw 9599 2017-11-19T21:46:43Z 2017-11-19T21:46:43Z OWNER

This calls for refactoring the code so the table view, the row view and the custom SQL view share as much logic as possible.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Redesign JSON output, ditch jsono, offer variants controlled by parameter instead 275092453  
345538016 https://github.com/simonw/datasette/issues/127#issuecomment-345538016 https://api.github.com/repos/simonw/datasette/issues/127 MDEyOklzc3VlQ29tbWVudDM0NTUzODAxNg== simonw 9599 2017-11-19T18:22:45Z 2017-11-19T18:22:45Z OWNER

I implemented a basic version of this in f59c840e7db8870afcdeba7a53bdea07bb674334 for custom SQL.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Filtered tables should show count of all matching rows, if fast enough 275135719  
345537315 https://github.com/simonw/datasette/issues/44#issuecomment-345537315 https://api.github.com/repos/simonw/datasette/issues/44 MDEyOklzc3VlQ29tbWVudDM0NTUzNzMxNQ== simonw 9599 2017-11-19T18:11:27Z 2017-11-19T18:11:27Z OWNER

This would enable faceted search - moving it to the search milestone.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 ?_group_count=country - return counts by specific column(s) 269731374  
345537268 https://github.com/simonw/datasette/issues/134#issuecomment-345537268 https://api.github.com/repos/simonw/datasette/issues/134 MDEyOklzc3VlQ29tbWVudDM0NTUzNzI2OA== simonw 9599 2017-11-19T18:10:48Z 2017-11-19T18:10:48Z OWNER

Dupe of #127 

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Filtered table view should show a count 275176094  
345533274 https://github.com/simonw/datasette/issues/131#issuecomment-345533274 https://api.github.com/repos/simonw/datasette/issues/131 MDEyOklzc3VlQ29tbWVudDM0NTUzMzI3NA== simonw 9599 2017-11-19T17:17:37Z 2017-11-19T17:18:05Z OWNER 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 UI support for running FTS searches 275166669  
345526517 https://github.com/simonw/datasette/issues/131#issuecomment-345526517 https://api.github.com/repos/simonw/datasette/issues/131 MDEyOklzc3VlQ29tbWVudDM0NTUyNjUxNw== simonw 9599 2017-11-19T15:48:28Z 2017-11-19T15:48:28Z OWNER

Since SQLite supports column specifications in the MATCH body itself, there's no need to provide a separate mechanism for specifying columns in the query string: https://sqlite.org/fts5.html#fts5_column_filters

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 UI support for running FTS searches 275166669  
345526171 https://github.com/simonw/datasette/issues/131#issuecomment-345526171 https://api.github.com/repos/simonw/datasette/issues/131 MDEyOklzc3VlQ29tbWVudDM0NTUyNjE3MQ== simonw 9599 2017-11-19T15:44:30Z 2017-11-19T15:44:30Z OWNER 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 UI support for running FTS searches 275166669  
345497689 https://github.com/simonw/datasette/issues/86#issuecomment-345497689 https://api.github.com/repos/simonw/datasette/issues/86 MDEyOklzc3VlQ29tbWVudDM0NTQ5NzY4OQ== simonw 9599 2017-11-19T07:27:40Z 2017-11-19T07:27:40Z OWNER

I'll have to refactor the foreign key annotating code to be usable in other contexts - at the moment it only works for annotating displays of rows, but I need to use it to resolve selected filters as well. 

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Filter UI on table page 273703829  
345497534 https://github.com/simonw/datasette/issues/86#issuecomment-345497534 https://api.github.com/repos/simonw/datasette/issues/86 MDEyOklzc3VlQ29tbWVudDM0NTQ5NzUzNA== simonw 9599 2017-11-19T07:23:33Z 2017-11-19T07:23:33Z OWNER

"Tablename: 3,567 rows where status = 3 (published) and n > 55"

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Filter UI on table page 273703829  
345497453 https://github.com/simonw/datasette/issues/86#issuecomment-345497453 https://api.github.com/repos/simonw/datasette/issues/86 MDEyOklzc3VlQ29tbWVudDM0NTQ5NzQ1Mw== simonw 9599 2017-11-19T07:21:22Z 2017-11-19T07:21:22Z OWNER

I'm going to be a bit classier about this and auto generate a title for the page that describes the currently applied filters.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Filter UI on table page 273703829  
345496540 https://github.com/simonw/datasette/issues/86#issuecomment-345496540 https://api.github.com/repos/simonw/datasette/issues/86 MDEyOklzc3VlQ29tbWVudDM0NTQ5NjU0MA== simonw 9599 2017-11-19T06:59:40Z 2017-11-19T06:59:40Z OWNER

OK,I've figured out how to do an initial version of this without JavaScript. I'll provide three form fields labell d "add filter":

  • a select box of all of the columns
  • a select box of the available operations
  • a value box

Submit those and the site will redirect you to a correctly populated querystring for that filter.

If you have filters applied, those will display as prepopulated form field triples.

For foreign key reference filters, I will display the resolved value next to the text box containing the numeric ID. In the future this can get a select2 style treatment.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Filter UI on table page 273703829  
345495046 https://github.com/simonw/datasette/issues/127#issuecomment-345495046 https://api.github.com/repos/simonw/datasette/issues/127 MDEyOklzc3VlQ29tbWVudDM0NTQ5NTA0Ng== simonw 9599 2017-11-19T06:17:42Z 2017-11-19T06:17:42Z OWNER

Maybe I should support &_count=1 to handle this - that would be easy to Ajax-in in conjenction with the other filters.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Filtered tables should show count of all matching rows, if fast enough 275135719  
345494971 https://github.com/simonw/datasette/issues/44#issuecomment-345494971 https://api.github.com/repos/simonw/datasette/issues/44 MDEyOklzc3VlQ29tbWVudDM0NTQ5NDk3MQ== simonw 9599 2017-11-19T06:15:39Z 2017-11-19T06:15:39Z OWNER

It would be great if this could support foreign key references and automatically resolve and hyperlink them if they are detected.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 ?_group_count=country - return counts by specific column(s) 269731374  
345494918 https://github.com/simonw/datasette/issues/86#issuecomment-345494918 https://api.github.com/repos/simonw/datasette/issues/86 MDEyOklzc3VlQ29tbWVudDM0NTQ5NDkxOA== simonw 9599 2017-11-19T06:14:17Z 2017-11-19T06:14:17Z OWNER

If the selected relationship is a foreign key reference, we should resolve that foreign key and display it on the page.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Filter UI on table page 273703829  
345494775 https://github.com/simonw/datasette/issues/86#issuecomment-345494775 https://api.github.com/repos/simonw/datasette/issues/86 MDEyOklzc3VlQ29tbWVudDM0NTQ5NDc3NQ== simonw 9599 2017-11-19T06:09:43Z 2017-11-19T06:09:43Z OWNER

Now that we have foreign key support (#85) this is even more important, since foreign key support actively encourages linking to filtered table views.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Filter UI on table page 273703829  
345494724 https://github.com/simonw/datasette/issues/85#issuecomment-345494724 https://api.github.com/repos/simonw/datasette/issues/85 MDEyOklzc3VlQ29tbWVudDM0NTQ5NDcyNA== simonw 9599 2017-11-19T06:08:19Z 2017-11-19T06:08:19Z OWNER

This is working really nicely now:

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Detect foreign keys and use them to link HTML pages together 273678673  
345494052 https://github.com/simonw/datasette/issues/105#issuecomment-345494052 https://api.github.com/repos/simonw/datasette/issues/105 MDEyOklzc3VlQ29tbWVudDM0NTQ5NDA1Mg== simonw 9599 2017-11-19T05:49:53Z 2017-11-19T05:49:53Z OWNER 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Consider data-package as a format for metadata 274314940  
345493344 https://github.com/simonw/datasette/issues/105#issuecomment-345493344 https://api.github.com/repos/simonw/datasette/issues/105 MDEyOklzc3VlQ29tbWVudDM0NTQ5MzM0NA== simonw 9599 2017-11-19T05:28:49Z 2017-11-19T05:28:49Z OWNER

Looks like there are a ton of interesting datasets packaged in this way at http://datahub.io/docs/core-data - see also https://github.com/datasets

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Consider data-package as a format for metadata 274314940  

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
, [performed_via_github_app] TEXT);
CREATE INDEX [idx_issue_comments_issue]
                ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
                ON [issue_comments] ([user]);