home / github / issue_comments

Menu
  • Search all tables
  • GraphQL API

issue_comments: 389572201

This data as json

html_url issue_url id node_id user created_at updated_at author_association body reactions issue performed_via_github_app
https://github.com/simonw/datasette/issues/266#issuecomment-389572201 https://api.github.com/repos/simonw/datasette/issues/266 389572201 MDEyOklzc3VlQ29tbWVudDM4OTU3MjIwMQ== 9599 2018-05-16T15:58:43Z 2018-05-16T16:00:47Z OWNER

This will likely be implemented in the BaseView class, which needs to know how to spot the .csv extension, call the underlying JSON generating function and then return the columns and rows as correctly formatted CSV.

https://github.com/simonw/datasette/blob/9959a9e4deec8e3e178f919e8b494214d5faa7fd/datasette/views/base.py#L201-L207

This means it will take ALL arguments that are available to the .json view. It may ignore some (e.g. _facet= makes no sense since CSV tables don't have space to show the facet results).

In streaming mode, things will behave a little bit differently - in particular, if _stream=1 then _next= will be forbidden.

It can't include a length header because we don't know how many bytes it will be

CSV output will throw an error if the endpoint doesn't have rows and columns keys eg /-/inspect.json

So the implementation...

  • looks for the .csv extension
  • internally fetches the .json data instead
  • If no _stream it just transposes that JSON to CSV with the correct content type header
  • If _stream=1 - checks for _next= and throws an error if it was provided
  • Otherwise... fetch first page and emit CSV header and first set of rows
  • Then start async looping, emitting more CSV rows and following the _next= internal reference until done

I like that this takes advantage of efficient pagination. It may not work so well for views which use offset/limit though.

It won't work at all for custom SQL because custom SQL doesn't support _next= pagination. That's fine.

For views... easiest fix is to cut off after first X000 records. That seems OK. View JSON would need to include a property that the mechanism can identify.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
323681589  
Powered by Datasette · Queries took 0.999ms · About: github-to-sqlite