issue_comments

Table actions

14 rows where author_association = "OWNER" and issue = 725996507 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date)

user 1

issue 1

  • Make it possible to download BLOB data from the Datasette UI · 14 ✖

author_association 1

  • OWNER · 14 ✖
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions issue performed_via_github_app
762387875 https://github.com/simonw/datasette/issues/1036#issuecomment-762387875 https://api.github.com/repos/simonw/datasette/issues/1036 MDEyOklzc3VlQ29tbWVudDc2MjM4Nzg3NQ== simonw 9599 2021-01-18T17:36:36Z 2021-01-18T17:36:36Z OWNER

As you can see, I'm pretty paranoid about serving content with Content-Type HTTP headers because I'm so worried about execution vulnerabilities. I'm much more comfortable exploring that kind of thing in plugins, since that way people can opt-in to riskier features.

You found datasette-media which is my most comprehensive exploration of that idea so far - but there's definitely lots of room for more plugins along those lines.

Maybe even an output plugin? .jpg as an export format which returns the BLOB column for a row as a JPEG image with the correct content-type header (but first verifies that the binary content does indeed look like a real JPEG) could be interesting.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Make it possible to download BLOB data from the Datasette UI 725996507  
713899530 https://github.com/simonw/datasette/issues/1036#issuecomment-713899530 https://api.github.com/repos/simonw/datasette/issues/1036 MDEyOklzc3VlQ29tbWVudDcxMzg5OTUzMA== simonw 9599 2020-10-21T21:55:00Z 2020-10-21T21:55:00Z OWNER 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Make it possible to download BLOB data from the Datasette UI 725996507  
713821656 https://github.com/simonw/datasette/issues/1036#issuecomment-713821656 https://api.github.com/repos/simonw/datasette/issues/1036 MDEyOklzc3VlQ29tbWVudDcxMzgyMTY1Ng== simonw 9599 2020-10-21T19:22:45Z 2020-10-21T19:41:48Z OWNER

So for https://latest.datasette.io/fixtures/binary_data the BLOB download URLs would be:

https://latest.datasette.io/fixtures/-/blob/binary_data/1/data.blob - that last bit after the primary key is to indicate the data column

With these headers:

  • Content-Disposition: attachment; filename="binary_data-1-data.blob"
  • X-Content-Type-Options: nosniff
  • Content-Type: application/binary
 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Make it possible to download BLOB data from the Datasette UI 725996507  
713830842 https://github.com/simonw/datasette/issues/1036#issuecomment-713830842 https://api.github.com/repos/simonw/datasette/issues/1036 MDEyOklzc3VlQ29tbWVudDcxMzgzMDg0Mg== simonw 9599 2020-10-21T19:41:20Z 2020-10-21T19:41:20Z OWNER 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Make it possible to download BLOB data from the Datasette UI 725996507  
713829629 https://github.com/simonw/datasette/issues/1036#issuecomment-713829629 https://api.github.com/repos/simonw/datasette/issues/1036 MDEyOklzc3VlQ29tbWVudDcxMzgyOTYyOQ== simonw 9599 2020-10-21T19:38:43Z 2020-10-21T19:38:43Z OWNER

Should this work just for BLOB columns, or should it work for other columns too?

For the moment I'm going to restrict it to BLOBs, since data from other columns is available through the UI whereas BLOB columns are not.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Make it possible to download BLOB data from the Datasette UI 725996507  
713818817 https://github.com/simonw/datasette/issues/1036#issuecomment-713818817 https://api.github.com/repos/simonw/datasette/issues/1036 MDEyOklzc3VlQ29tbWVudDcxMzgxODgxNw== simonw 9599 2020-10-21T19:17:01Z 2020-10-21T19:17:01Z OWNER

Actually I like .blob

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Make it possible to download BLOB data from the Datasette UI 725996507  
713818178 https://github.com/simonw/datasette/issues/1036#issuecomment-713818178 https://api.github.com/repos/simonw/datasette/issues/1036 MDEyOklzc3VlQ29tbWVudDcxMzgxODE3OA== simonw 9599 2020-10-21T19:15:38Z 2020-10-21T19:16:34Z OWNER

What should the suggested filename be?

I think something that includes the table name, primary key and the name of the column would work.

How about a file extension? I guess .binary, then let the user rename it? Or .raw.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Make it possible to download BLOB data from the Datasette UI 725996507  
713278349 https://github.com/simonw/datasette/issues/1036#issuecomment-713278349 https://api.github.com/repos/simonw/datasette/issues/1036 MDEyOklzc3VlQ29tbWVudDcxMzI3ODM0OQ== simonw 9599 2020-10-21T03:42:29Z 2020-10-21T03:42:29Z OWNER

Possible URL for this: /db/table/-/blob/primary-keys - this would use the /db/table/-/ namespace proposed in #296.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Make it possible to download BLOB data from the Datasette UI 725996507  
713226726 https://github.com/simonw/datasette/issues/1036#issuecomment-713226726 https://api.github.com/repos/simonw/datasette/issues/1036 MDEyOklzc3VlQ29tbWVudDcxMzIyNjcyNg== simonw 9599 2020-10-21T01:04:25Z 2020-10-21T01:04:25Z OWNER

Extra security idea: a blob_download_host setting which can be used to indicate a host that should be used for downloads - for example datasettestatic.com. If this setting is populated then binary downloads are served from paths on that host only, and no other Datasette URLs from that host will be served.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Make it possible to download BLOB data from the Datasette UI 725996507  
713186189 https://github.com/simonw/datasette/issues/1036#issuecomment-713186189 https://api.github.com/repos/simonw/datasette/issues/1036 MDEyOklzc3VlQ29tbWVudDcxMzE4NjE4OQ== simonw 9599 2020-10-20T22:56:33Z 2020-10-20T22:56:33Z OWNER

I think this plus the binary-CSV stuff in #1034 will justify a dedicated section of the documentation to talk about how Datasette handles binary BLOB columns.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Make it possible to download BLOB data from the Datasette UI 725996507  
713185871 https://github.com/simonw/datasette/issues/1036#issuecomment-713185871 https://api.github.com/repos/simonw/datasette/issues/1036 MDEyOklzc3VlQ29tbWVudDcxMzE4NTg3MQ== simonw 9599 2020-10-20T22:55:36Z 2020-10-20T22:55:36Z OWNER

I can also use a Content-Disposition header to force a download. I'm reasonably confident that the combination of Content-Disposition and X-Content-Type-Options: nosniff and application/binary will let me allow users to download the contents of arbitrary BLOB columns without any XSS risk.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Make it possible to download BLOB data from the Datasette UI 725996507  
713185173 https://github.com/simonw/datasette/issues/1036#issuecomment-713185173 https://api.github.com/repos/simonw/datasette/issues/1036 MDEyOklzc3VlQ29tbWVudDcxMzE4NTE3Mw== simonw 9599 2020-10-20T22:53:41Z 2020-10-20T22:53:41Z OWNER

https://security.stackexchange.com/questions/12896/does-x-content-type-options-really-prevent-content-sniffing-attacks says:

In Tangled Web Michal Zalewski says:

Refrain from using Content-Type: application/octet-stream and use application/binary instead, especially for unknown document types. Refrain from returning Content-Type: text/plain.

For example, any code-hosting platform must exercise caution when returning executables or source archives as application/octet-stream, because there is a risk they may be misinterpreted as HTML and displayed inline.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Make it possible to download BLOB data from the Datasette UI 725996507  
713184374 https://github.com/simonw/datasette/issues/1036#issuecomment-713184374 https://api.github.com/repos/simonw/datasette/issues/1036 MDEyOklzc3VlQ29tbWVudDcxMzE4NDM3NA== simonw 9599 2020-10-20T22:51:22Z 2020-10-20T22:51:22Z OWNER

From https://hackerone.com/reports/126197:

archive.uber.com mirrors pypi. When downloading .tar.gz files from archive.uber.com, the MIME type is application/octet-stream. Injecting <html><script>alert(0)</script> into the start of the .tar.gz causes an XSS in Internet Explorer due to MIME sniffing.

So you do have to be careful not to open accidental XSS holes with application/octet-stream thanks to (presumably older) versions of IE.

From that thread it looks like the solution is to add a X-Content-Type-Options: nosniff header.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Make it possible to download BLOB data from the Datasette UI 725996507  
713183306 https://github.com/simonw/datasette/issues/1036#issuecomment-713183306 https://api.github.com/repos/simonw/datasette/issues/1036 MDEyOklzc3VlQ29tbWVudDcxMzE4MzMwNg== simonw 9599 2020-10-20T22:48:10Z 2020-10-20T22:48:10Z OWNER 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Make it possible to download BLOB data from the Datasette UI 725996507  

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
, [performed_via_github_app] TEXT);
CREATE INDEX [idx_issue_comments_issue]
                ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
                ON [issue_comments] ([user]);