issues

Table actions

9 rows where "created_at" is on date 2018-05-16 and user = 9599 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: comments, created_at (date), updated_at (date), closed_at (date)

type 2

state 2

repo 1

id node_id number title user state locked assignee milestone comments created_at updated_at ▲ closed_at author_association pull_request body repo type active_lock_reason performed_via_github_app reactions draft state_reason
323658641 MDU6SXNzdWUzMjM2NTg2NDE= 262 Add ?_extra= mechanism for requesting extra properties in JSON simonw 9599 open 0   Datasette 1.0 3268330 27 2018-05-16T14:55:42Z 2023-03-29T06:22:22Z   OWNER  

Datasette views currently work by creating a set of data that should be returned as JSON, then defining an additional, optional template_data() function which is called if the view is being rendered as HTML.

This template_data() function calculates extra template context variables which are necessary for the HTML view but should not be included in the JSON.

Example of how that is used today: https://github.com/simonw/datasette/blob/2b79f2bdeb1efa86e0756e741292d625f91cb93d/datasette/views/table.py#L672-L704

With features like Facets in #255 I'm beginning to want to move more items into the template_data() - in the case of facets it's the suggested_facets array. This saves that feature from being calculated (involving several SQL queries) for the JSON case where it is unlikely to be used.

But... as an API user, I want to still optionally be able to access that information.

Solution: Add a ?_extra=suggested_facets&_extra=table_metadata argument which can be used to optionally request additional blocks to be added to the JSON API.

Then redefine as many of the current template_data() features as extra arguments instead, and teach Datasette to return certain extras by default when rendering templates.

This could allow the JSON representation to be slimmed down further (removing e.g. the table_definition and view_definition keys) while still making that information available to API users who need it.

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/262/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
    
323718842 MDU6SXNzdWUzMjM3MTg4NDI= 268 Mechanism for ranking results from SQLite full-text search simonw 9599 open 0     12 2018-05-16T17:36:40Z 2022-01-13T22:21:28Z   OWNER  

This isn't particularly straight-forward - all the more reason for Datasette to implement it for you. This article is helpful: http://charlesleifer.com/blog/using-sqlite-full-text-search-with-python/

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/268/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
    
323681589 MDU6SXNzdWUzMjM2ODE1ODk= 266 Export to CSV simonw 9599 closed 0     27 2018-05-16T15:50:24Z 2021-06-17T18:14:24Z 2018-06-18T06:05:25Z OWNER  

Datasette needs to be able to export data to CSV.

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/266/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
323671577 MDU6SXNzdWUzMjM2NzE1Nzc= 263 Facets should not execute for ?shape=array|object simonw 9599 closed 0     3 2018-05-16T15:26:13Z 2021-06-02T02:54:34Z 2021-06-02T02:54:34Z OWNER  

Split off from #255 - there's no point executing the facet SQL for the ?_shape=array and ?_shape=object API responses.

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/263/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
323716411 MDU6SXNzdWUzMjM3MTY0MTE= 267 Documentation for URL hashing, redirects and cache policy simonw 9599 closed 0     3 2018-05-16T17:29:01Z 2019-06-24T06:41:02Z 2019-06-24T06:41:02Z OWNER  

See my comments on #258 for a starting point

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/267/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
323677499 MDU6SXNzdWUzMjM2Nzc0OTk= 265 Add links to example Datasette instances to appropiate places in docs simonw 9599 closed 0     5 2018-05-16T15:40:20Z 2018-06-18T15:52:15Z 2018-06-18T15:52:15Z OWNER   datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/265/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
323673899 MDU6SXNzdWUzMjM2NzM4OTk= 264 Make it possible to customize various facet settings simonw 9599 closed 0     1 2018-05-16T15:31:34Z 2018-05-18T06:18:00Z 2018-05-18T05:11:52Z OWNER  

The new Facets implementation from #255 includes several hard-coded settings which should be made configurable somehow:

Number of rows to return in a facet (maybe this should also be an option that can be set via quersytring argument, e.g. ?_facet=qSpecies:40):

https://github.com/simonw/datasette/blob/9959a9e4deec8e3e178f919e8b494214d5faa7fd/datasette/views/table.py#L539

Time limit for executing a facet:

https://github.com/simonw/datasette/blob/9959a9e4deec8e3e178f919e8b494214d5faa7fd/datasette/views/table.py#L559-L562

Maximum unique values returned in order for a column to be suggested as a facet:

https://github.com/simonw/datasette/blob/9959a9e4deec8e3e178f919e8b494214d5faa7fd/datasette/views/table.py#L646-L647

Time limit for calculating if a column should be a suggested facet:

https://github.com/simonw/datasette/blob/9959a9e4deec8e3e178f919e8b494214d5faa7fd/datasette/views/table.py#L664-L667

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/264/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
323726888 MDU6SXNzdWUzMjM3MjY4ODg= 269 If a facet fails due to timing out, let the user know somehow simonw 9599 closed 0     0 2018-05-16T18:01:47Z 2018-05-18T06:11:46Z 2018-05-18T06:11:46Z OWNER  

Refs #255 - right now facets fail silently if the user requested them but they take longer than 200ms to calculate - see also #264

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/269/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
323459939 MDExOlB1bGxSZXF1ZXN0MTg4MzEyNDEx 261 Facets improvements plus suggested facets simonw 9599 closed 0     0 2018-05-16T03:52:39Z 2018-05-16T15:27:26Z 2018-05-16T15:27:25Z OWNER simonw/datasette/pulls/261

Refs #255

 datasette 107914493 pull     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/261/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 0  

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issues] (
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [number] INTEGER,
   [title] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [state] TEXT,
   [locked] INTEGER,
   [assignee] INTEGER REFERENCES [users]([id]),
   [milestone] INTEGER REFERENCES [milestones]([id]),
   [comments] INTEGER,
   [created_at] TEXT,
   [updated_at] TEXT,
   [closed_at] TEXT,
   [author_association] TEXT,
   [pull_request] TEXT,
   [body] TEXT,
   [repo] INTEGER REFERENCES [repos]([id]),
   [type] TEXT
, [active_lock_reason] TEXT, [performed_via_github_app] TEXT, [reactions] TEXT, [draft] INTEGER, [state_reason] TEXT);
CREATE INDEX [idx_issues_repo]
                ON [issues] ([repo]);
CREATE INDEX [idx_issues_milestone]
                ON [issues] ([milestone]);
CREATE INDEX [idx_issues_assignee]
                ON [issues] ([assignee]);
CREATE INDEX [idx_issues_user]
                ON [issues] ([user]);