home / github

Menu
  • Search all tables
  • GraphQL API

issue_comments

Table actions
  • GraphQL API for issue_comments

20 rows where "updated_at" is on date 2020-10-29 and user = 9599 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: issue_url, reactions, created_at (date), updated_at (date)

issue 8

  • Switch to .blob render extension for BLOB downloads 7
  • .blob output renderer 3
  • .csv should link to .blob downloads 3
  • Add template block prior to extra URL loaders 2
  • Better display of binary data on arbitrary query results page 2
  • Better way of representing binary data in .csv output 1
  • Document recommendations for plugin authors to design URLs 1
  • Refactor .csv to be an output renderer - and teach register_output_renderer to stream all rows 1

user 1

  • simonw · 20 ✖

author_association 1

  • OWNER 20
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions issue performed_via_github_app
719066706 https://github.com/simonw/datasette/issues/1063#issuecomment-719066706 https://api.github.com/repos/simonw/datasette/issues/1063 MDEyOklzc3VlQ29tbWVudDcxOTA2NjcwNg== simonw 9599 2020-10-29T22:46:28Z 2020-10-29T22:46:28Z OWNER

I'm not going to do the base64 thing unless someone asks for it.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
.csv should link to .blob downloads 732685643  
719053669 https://github.com/simonw/datasette/issues/1051#issuecomment-719053669 https://api.github.com/repos/simonw/datasette/issues/1051 MDEyOklzc3VlQ29tbWVudDcxOTA1MzY2OQ== simonw 9599 2020-10-29T22:12:16Z 2020-10-29T22:12:16Z OWNER

https://latest.datasette.io/fixtures?sql=select+rowid%2C+data+from+binary_data+order+by+rowid+limit+101 now looks like this:

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Better display of binary data on arbitrary query results page 729096595  
719050754 https://github.com/simonw/datasette/issues/1034#issuecomment-719050754 https://api.github.com/repos/simonw/datasette/issues/1034 MDEyOklzc3VlQ29tbWVudDcxOTA1MDc1NA== simonw 9599 2020-10-29T22:04:52Z 2020-10-29T22:04:52Z OWNER

I'm going to link to. the new .blob representation using the new ?_blob_hash=xxx argument to ensure that the content served is the expected binary blob.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Better way of representing binary data in .csv output 725184645  
719050390 https://github.com/simonw/datasette/issues/1063#issuecomment-719050390 https://api.github.com/repos/simonw/datasette/issues/1063 MDEyOklzc3VlQ29tbWVudDcxOTA1MDM5MA== simonw 9599 2020-10-29T22:04:00Z 2020-10-29T22:04:00Z OWNER

This will close #1034.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
.csv should link to .blob downloads 732685643  
719042601 https://github.com/simonw/datasette/pull/1061#issuecomment-719042601 https://api.github.com/repos/simonw/datasette/issues/1061 MDEyOklzc3VlQ29tbWVudDcxOTA0MjYwMQ== simonw 9599 2020-10-29T21:45:35Z 2020-10-29T21:50:42Z OWNER

Moving the CSV work to a separate issue.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
.blob output renderer 732634375  
719043108 https://github.com/simonw/datasette/issues/1063#issuecomment-719043108 https://api.github.com/repos/simonw/datasette/issues/1063 MDEyOklzc3VlQ29tbWVudDcxOTA0MzEwOA== simonw 9599 2020-10-29T21:46:48Z 2020-10-29T21:46:48Z OWNER

Remove this xfail and import pytest: https://github.com/simonw/datasette/blob/503a5b7b4080a26ef9ceb1ecd1a4a6f4ef4ffc59/tests/test_csv.py#L83-L96

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
.csv should link to .blob downloads 732685643  
719035336 https://github.com/simonw/datasette/pull/1061#issuecomment-719035336 https://api.github.com/repos/simonw/datasette/issues/1061 MDEyOklzc3VlQ29tbWVudDcxOTAzNTMzNg== simonw 9599 2020-10-29T21:29:29Z 2020-10-29T21:29:29Z OWNER

Those display_rows have already been processed by the render_cell plugin hook: https://github.com/simonw/datasette/blob/d6f9ff71378c4eab34dad181c23cfc143a4aef2d/datasette/views/database.py#L320-L346

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
.blob output renderer 732634375  
719033013 https://github.com/simonw/datasette/pull/1061#issuecomment-719033013 https://api.github.com/repos/simonw/datasette/issues/1061 MDEyOklzc3VlQ29tbWVudDcxOTAzMzAxMw== simonw 9599 2020-10-29T21:27:14Z 2020-10-29T21:27:14Z OWNER

Next challenge: link to .blob downloads from https://latest.datasette.io/fixtures?sql=select+rowid%2C+data+from+binary_data

This will be a bit tricky. Here's how that template works at the moment: https://github.com/simonw/datasette/blob/d6f9ff71378c4eab34dad181c23cfc143a4aef2d/datasette/templates/query.html#L69-L77

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
.blob output renderer 732634375  
719031901 https://github.com/simonw/datasette/issues/1062#issuecomment-719031901 https://api.github.com/repos/simonw/datasette/issues/1062 MDEyOklzc3VlQ29tbWVudDcxOTAzMTkwMQ== simonw 9599 2020-10-29T21:25:54Z 2020-10-29T21:25:54Z OWNER

Relevant code: https://github.com/simonw/datasette/blob/d6f9ff71378c4eab34dad181c23cfc143a4aef2d/datasette/views/base.py#L258-L345

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Refactor .csv to be an output renderer - and teach register_output_renderer to stream all rows 732674148  
719021514 https://github.com/simonw/datasette/issues/1050#issuecomment-719021514 https://api.github.com/repos/simonw/datasette/issues/1050 MDEyOklzc3VlQ29tbWVudDcxOTAyMTUxNA== simonw 9599 2020-10-29T21:05:08Z 2020-10-29T21:05:08Z OWNER

Idea: what if Datasette had a custom SQLite function that could be used to generate URLs to the row-level BLOB download for a value? Then custom SQL query authors could use that function to link to the relevant content.

This could be expanded to exposing other datasette.urls functionality as well.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Switch to .blob render extension for BLOB downloads 729057388  
719001701 https://github.com/simonw/datasette/issues/1050#issuecomment-719001701 https://api.github.com/repos/simonw/datasette/issues/1050 MDEyOklzc3VlQ29tbWVudDcxOTAwMTcwMQ== simonw 9599 2020-10-29T20:26:44Z 2020-10-29T20:26:44Z OWNER

I'll do the rest of the work on this in the pull request #1061.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Switch to .blob render extension for BLOB downloads 729057388  
718989895 https://github.com/simonw/datasette/issues/1050#issuecomment-718989895 https://api.github.com/repos/simonw/datasette/issues/1050 MDEyOklzc3VlQ29tbWVudDcxODk4OTg5NQ== simonw 9599 2020-10-29T20:04:15Z 2020-10-29T20:04:15Z OWNER

I'll use hashlib.sha256 for these hashes.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Switch to .blob render extension for BLOB downloads 729057388  
718987852 https://github.com/simonw/datasette/issues/1050#issuecomment-718987852 https://api.github.com/repos/simonw/datasette/issues/1050 MDEyOklzc3VlQ29tbWVudDcxODk4Nzg1Mg== simonw 9599 2020-10-29T20:00:32Z 2020-10-29T20:00:32Z OWNER

The reason I like the ?_blob_hash= solution is that it feels really misleading to provide a link to "download this binary" which could conceivably download some other data.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Switch to .blob render extension for BLOB downloads 729057388  
718980944 https://github.com/simonw/datasette/issues/1050#issuecomment-718980944 https://api.github.com/repos/simonw/datasette/issues/1050 MDEyOklzc3VlQ29tbWVudDcxODk4MDk0NA== simonw 9599 2020-10-29T19:46:19Z 2020-10-29T19:46:19Z OWNER

Had an idea in https://github.com/simonw/datasette/issues/1051#issuecomment-718980659

OK, alternative idea. The .blob output renderer from #1050 gets to see multiple rows at once.

For an arbitrary SQL query, how about if I link to this?

/db.blob?sql=...&_blob_column=data&_blob_hash=bc4c24181ed3ce666

Then the output renderer loops through all of the data results that are available to it and, if one of them hashes to that value, serves up that data?

If no matches are found it can show an error message telling you that the link has expired (presumably because the underlying database has changed since the link was generated).

I think this might be the best solution to the problem.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Switch to .blob render extension for BLOB downloads 729057388  
718980659 https://github.com/simonw/datasette/issues/1051#issuecomment-718980659 https://api.github.com/repos/simonw/datasette/issues/1051 MDEyOklzc3VlQ29tbWVudDcxODk4MDY1OQ== simonw 9599 2020-10-29T19:45:42Z 2020-10-29T19:45:42Z OWNER

OK, alternative idea. The .blob output renderer from #1050 gets to see multiple rows at once.

For an arbitrary SQL query, how about if I link to this?

/db.blob?sql=...&_blob_column=data&_blob_hash=bc4c24181ed3ce666

Then the output renderer loops through all of the data results that are available to it and, if one of them hashes to that value, serves up that data?

If no matches are found it can show an error message telling you that the link has expired (presumably because the underlying database has changed since the link was generated).

I think this might be the best solution to the problem.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Better display of binary data on arbitrary query results page 729096595  
718976679 https://github.com/simonw/datasette/issues/1053#issuecomment-718976679 https://api.github.com/repos/simonw/datasette/issues/1053 MDEyOklzc3VlQ29tbWVudDcxODk3NjY3OQ== simonw 9599 2020-10-29T19:37:57Z 2020-10-29T19:37:57Z OWNER

https://docs.datasette.io/en/latest/writing_plugins.html#designing-urls-for-your-plugin

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Document recommendations for plugin authors to design URLs 729604838  
718346019 https://github.com/simonw/datasette/issues/1050#issuecomment-718346019 https://api.github.com/repos/simonw/datasette/issues/1050 MDEyOklzc3VlQ29tbWVudDcxODM0NjAxOQ== simonw 9599 2020-10-29T04:05:07Z 2020-10-29T04:05:07Z OWNER

Yes, confirmed - this is a bug where if the BLOB column contains a null you get a nasty exception if you try to download it.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Switch to .blob render extension for BLOB downloads 729057388  
718342036 https://github.com/simonw/datasette/issues/1050#issuecomment-718342036 https://api.github.com/repos/simonw/datasette/issues/1050 MDEyOklzc3VlQ29tbWVudDcxODM0MjAzNg== simonw 9599 2020-10-29T03:49:57Z 2020-10-29T03:49:57Z OWNER

@thadk from that error it looks like the problem may have been that you had a BLOB column containing a null value? If so that's definitely a bug, I'll fix that.

{
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Switch to .blob render extension for BLOB downloads 729057388  
718340847 https://github.com/simonw/datasette/pull/1049#issuecomment-718340847 https://api.github.com/repos/simonw/datasette/issues/1049 MDEyOklzc3VlQ29tbWVudDcxODM0MDg0Nw== simonw 9599 2020-10-29T03:45:47Z 2020-10-29T03:48:26Z OWNER

thebe is the first time I've seen a library that requires you to set up some global JavaScript configuration before loading the script itself.

I'm hesitant to add an extra template block just to cover that one case since it's such a rare pattern. But it's important that thebelab can be used with Datasette.

Would this pattern work for you instead?

```html+jinja {% block extra_head %}

<script type="text/x-thebe-config"> { requestKernel: true, binderOptions: { repo: "binder-examples/requirements", }, } </script> <script src="https://unpkg.com/thebelab@latest/lib/index.js"></script>

{% endblock %} ```

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Add template block prior to extra URL loaders 729017519  
718341542 https://github.com/simonw/datasette/pull/1049#issuecomment-718341542 https://api.github.com/repos/simonw/datasette/issues/1049 MDEyOklzc3VlQ29tbWVudDcxODM0MTU0Mg== simonw 9599 2020-10-29T03:48:12Z 2020-10-29T03:48:12Z OWNER

You could use Datasette's new {{ urls.static_plugins(...) }} template option - see https://docs.datasette.io/en/latest/internals.html#internals-datasette-urls - to generate a link to code that was bundled with the plugin:

```html+jinja {% block extra_head %}

<script type="text/x-thebe-config"> { requestKernel: true, binderOptions: { repo: "binder-examples/requirements", }, } </script> <script src="{{ urls.static_plugins("datasette-thebelab-plugin", "thebelab-index.js")"></script>

{% endblock %} ```

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Add template block prior to extra URL loaders 729017519  

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options:

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
, [performed_via_github_app] TEXT);
CREATE INDEX [idx_issue_comments_issue]
                ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
                ON [issue_comments] ([user]);
Powered by Datasette · Queries took 544.39ms · About: github-to-sqlite