home / github

Menu
  • Search all tables
  • GraphQL API

issue_comments

Table actions
  • GraphQL API for issue_comments

15 rows where "updated_at" is on date 2020-10-25 sorted by updated_at descending

✖
✖

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: issue_url, created_at (date), updated_at (date)

issue 6

  • Better way of representing binary data in .csv output 8
  • Switch to .blob render extension for BLOB downloads 2
  • Better display of binary data on arbitrary query results page 2
  • Incorrect URLs when served behind a proxy with base_url set 1
  • Link to blob downloads in the right places 1
  • Add template block prior to extra URL loaders 1

user 3

  • simonw 13
  • psychemedia 1
  • codecov[bot] 1

author_association 3

  • OWNER 13
  • CONTRIBUTOR 1
  • NONE 1
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions issue performed_via_github_app
716204271 https://github.com/simonw/datasette/issues/1051#issuecomment-716204271 https://api.github.com/repos/simonw/datasette/issues/1051 MDEyOklzc3VlQ29tbWVudDcxNjIwNDI3MQ== simonw 9599 2020-10-25T20:08:04Z 2020-10-25T20:08:04Z OWNER

This is bad though, because if I want to provide binary data in CSV as requested in #1034 I need some way of providing that data.

Which suggests to me that the base64 option is the only one that can make sense for arbitrary SQL queries represented as CSV. Download links won't work.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Better display of binary data on arbitrary query results page 729096595  
716204090 https://github.com/simonw/datasette/issues/1051#issuecomment-716204090 https://api.github.com/repos/simonw/datasette/issues/1051 MDEyOklzc3VlQ29tbWVudDcxNjIwNDA5MA== simonw 9599 2020-10-25T20:06:42Z 2020-10-25T20:06:42Z OWNER

Providing a binary download link here is actually extremely difficult.

The problem is that the SQL query itself represents data that can change from one moment to the next. It's no good showing a "Binary: 55 bytes" message that links to that same SQL query but with a .blob extension and arguments to select the particular result, because the data may change in a way that causes that query to return a different row - at which point the download link will give you the wrong data, not the 55 bytes you asked for.

So providing a download link risks being misleading.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Better display of binary data on arbitrary query results page 729096595  
716174203 https://github.com/simonw/datasette/issues/1050#issuecomment-716174203 https://api.github.com/repos/simonw/datasette/issues/1050 MDEyOklzc3VlQ29tbWVudDcxNjE3NDIwMw== simonw 9599 2020-10-25T16:27:39Z 2020-10-25T16:53:27Z OWNER

Idea: .blob output rendererer, where you tell it which column you want using ?_blob_column=x.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Switch to .blob render extension for BLOB downloads 729057388  
716175236 https://github.com/simonw/datasette/issues/1050#issuecomment-716175236 https://api.github.com/repos/simonw/datasette/issues/1050 MDEyOklzc3VlQ29tbWVudDcxNjE3NTIzNg== simonw 9599 2020-10-25T16:35:20Z 2020-10-25T16:35:20Z OWNER

This is clearly a better solution than the one I implemented in #1040 - I don't have to add a new route, I don't have to implement permission checks, it reuses mechanism.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Switch to .blob render extension for BLOB downloads 729057388  
716146238 https://github.com/simonw/datasette/pull/1049#issuecomment-716146238 https://api.github.com/repos/simonw/datasette/issues/1049 MDEyOklzc3VlQ29tbWVudDcxNjE0NjIzOA== codecov[bot] 22429695 2020-10-25T13:13:32Z 2020-10-25T13:13:32Z NONE

Codecov Report

Merging #1049 into main will not change coverage. The diff coverage is n/a.

```diff @@ Coverage Diff @@

main #1049 +/-

======================================= Coverage 84.72% 84.72%
======================================= Files 28 28
Lines 3942 3942
======================================= Hits 3340 3340
Misses 602 602
```


Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 42f4851...50a743a. Read the comment docs.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Add template block prior to extra URL loaders 729017519  
716123598 https://github.com/simonw/datasette/issues/838#issuecomment-716123598 https://api.github.com/repos/simonw/datasette/issues/838 MDEyOklzc3VlQ29tbWVudDcxNjEyMzU5OA== psychemedia 82988 2020-10-25T10:20:12Z 2020-10-25T10:53:24Z CONTRIBUTOR

I'm trying to run something behind a MyBinder proxy, but seem to have something set up incorrectly and not sure what the fix is?

I'm starting datasette with jupyter-server-proxy setup:

```

init.py

def setup_nbsearch():

return {
    "command": [
        "datasette",
        "serve",
        f"{_NBSEARCH_DB_PATH}",
        "-p",
        "{port}",
        "--config",
        "base_url:{base_url}nbsearch/"
    ],
    "absolute_url": True,
    # The following needs a the labextension installing.
    # eg in postBuild: jupyter labextension install jupyterlab-server-proxy
    "launcher_entry": {
        "enabled": True,
        "title": "nbsearch",
    },
}

```

where the base_url gets automatically populated by the server-proxy. I define the loaders as:

```

init.py

from datasette import hookimpl

@hookimpl def extra_css_urls(database, table, columns, view_name, datasette): return [ "/-/static-plugins/nbsearch/prism.css", "/-/static-plugins/nbsearch/nbsearch.css", ] ``` but these seem to also need a base_url prefix set somehow?

Currently, the generated HTML loads properly but internal links are incorrect; eg they take the form <link rel="stylesheet" href="/-/static-plugins/nbsearch/prism.css"> which resolves to eg https://notebooks.gesis.org/hub/-/static-plugins/nbsearch/prism.css rather than required URL of form https://notebooks.gesis.org/binder/jupyter/user/ouseful-testing-nbsearch-0fx1mx67/nbsearch/-/static-plugins/nbsearch/prism.css.

The main css is loaded correctly: <link rel="stylesheet" href="/binder/jupyter/user/ouseful-testing-nbsearch-0fx1mx67/nbsearch/-/static/app.css?404439">

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Incorrect URLs when served behind a proxy with base_url set 637395097  
716078777 https://github.com/simonw/datasette/issues/1034#issuecomment-716078777 https://api.github.com/repos/simonw/datasette/issues/1034 MDEyOklzc3VlQ29tbWVudDcxNjA3ODc3Nw== simonw 9599 2020-10-25T01:25:11Z 2020-10-25T01:25:11Z OWNER

SQLite actually has APIs that could help here: https://www.sqlite.org/c3ref/column_database_name.html - for any given SQL query they identify the origin/table/column that is the source of each resulting column.

Those aren't exposed in the Python sqlite3 module though, so using them could be extremely tricky.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Better way of representing binary data in .csv output 725184645  
716078605 https://github.com/simonw/datasette/issues/1034#issuecomment-716078605 https://api.github.com/repos/simonw/datasette/issues/1034 MDEyOklzc3VlQ29tbWVudDcxNjA3ODYwNQ== simonw 9599 2020-10-25T01:22:22Z 2020-10-25T01:22:22Z OWNER

For arbitrary CSV the only solution I can think of is to embed the base64 value.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Better way of representing binary data in .csv output 725184645  
716078512 https://github.com/simonw/datasette/issues/1034#issuecomment-716078512 https://api.github.com/repos/simonw/datasette/issues/1034 MDEyOklzc3VlQ29tbWVudDcxNjA3ODUxMg== simonw 9599 2020-10-25T01:21:11Z 2020-10-25T01:21:11Z OWNER

What should happen for CSV export of arbitrary SQL queries, where there's no obvious BLOB to link to?

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Better way of representing binary data in .csv output 725184645  
716078420 https://github.com/simonw/datasette/issues/1034#issuecomment-716078420 https://api.github.com/repos/simonw/datasette/issues/1034 MDEyOklzc3VlQ29tbWVudDcxNjA3ODQyMA== simonw 9599 2020-10-25T01:20:00Z 2020-10-25T01:20:00Z OWNER

That documentation: https://docs.datasette.io/en/latest/internals.html#absolute-url-request-path

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Better way of representing binary data in .csv output 725184645  
716077541 https://github.com/simonw/datasette/issues/1034#issuecomment-716077541 https://api.github.com/repos/simonw/datasette/issues/1034 MDEyOklzc3VlQ29tbWVudDcxNjA3NzU0MQ== simonw 9599 2020-10-25T01:09:38Z 2020-10-25T01:10:04Z OWNER

I should turn datasette.absolute_url(...) into a documented internal API on https://docs.datasette.io/en/stable/internals.html#datasette-class

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Better way of representing binary data in .csv output 725184645  
716077508 https://github.com/simonw/datasette/issues/1034#issuecomment-716077508 https://api.github.com/repos/simonw/datasette/issues/1034 MDEyOklzc3VlQ29tbWVudDcxNjA3NzUwOA== simonw 9599 2020-10-25T01:09:17Z 2020-10-25T01:09:17Z OWNER

Here's how those absolute next_url values are generated: https://github.com/simonw/datasette/blob/5db7ae3ce165ded57c7fb1cfbdb3258b1cf06c10/datasette/views/table.py#L774-L776

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Better way of representing binary data in .csv output 725184645  
716077436 https://github.com/simonw/datasette/issues/1034#issuecomment-716077436 https://api.github.com/repos/simonw/datasette/issues/1034 MDEyOklzc3VlQ29tbWVudDcxNjA3NzQzNg== simonw 9599 2020-10-25T01:08:35Z 2020-10-25T01:08:42Z OWNER

This is actually a bit tricky to implement, for a few reasons:

  • Need to generate a full URL, including the https://host/ bit. I've done this for next_url in the JSON output before, thankfully.
  • This only makes sense for CSV output for tables. If it's the CSV output of an arbitrary query there's no /db/table/-/blob/pk/column.blob page for me to link to.
  • Need to generate those /.../-/blob/... URLs for the data that is being output as CSV.
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Better way of representing binary data in .csv output 725184645  
713277810 https://github.com/simonw/datasette/issues/1034#issuecomment-713277810 https://api.github.com/repos/simonw/datasette/issues/1034 MDEyOklzc3VlQ29tbWVudDcxMzI3NzgxMA== simonw 9599 2020-10-21T03:40:50Z 2020-10-25T01:01:23Z OWNER

Blocked awaiting #1036 (update: now unblocked)

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Better way of representing binary data in .csv output 725184645  
716071507 https://github.com/simonw/datasette/issues/1046#issuecomment-716071507 https://api.github.com/repos/simonw/datasette/issues/1046 MDEyOklzc3VlQ29tbWVudDcxNjA3MTUwNw== simonw 9599 2020-10-25T00:06:47Z 2020-10-25T00:06:47Z OWNER

I used https://primer.style/octicons/download-16 instead.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Link to blob downloads in the right places 728895193  

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options:

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
, [performed_via_github_app] TEXT);
CREATE INDEX [idx_issue_comments_issue]
                ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
                ON [issue_comments] ([user]);
Powered by Datasette · Queries took 515.107ms · About: github-to-sqlite
  • Sort ascending
  • Sort descending
  • Facet by this
  • Hide this column
  • Show all columns
  • Show not-blank rows