issue_comments

Table actions

10 rows where author_association = "OWNER", issue = 324188953 and "updated_at" is on date 2019-06-23 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date)

user 1

issue 1

  • Port Datasette to ASGI · 10 ✖

author_association 1

  • OWNER · 10 ✖
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions issue performed_via_github_app
504761039 https://github.com/simonw/datasette/issues/272#issuecomment-504761039 https://api.github.com/repos/simonw/datasette/issues/272 MDEyOklzc3VlQ29tbWVudDUwNDc2MTAzOQ== simonw 9599 2019-06-23T15:15:41Z 2019-06-23T15:18:36Z OWNER

And now the tests are all passing!

Still to do:

  • Use raw_path so table names containing / can work correctly
  • Get ?_trace=1 working again
  • Replacement for @app.listener("before_server_start")
  • Replace Sanic request object with my own request class, so I can remove Sanic dependency
 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Port Datasette to ASGI 324188953  
504761165 https://github.com/simonw/datasette/issues/272#issuecomment-504761165 https://api.github.com/repos/simonw/datasette/issues/272 MDEyOklzc3VlQ29tbWVudDUwNDc2MTE2NQ== simonw 9599 2019-06-23T15:17:07Z 2019-06-23T15:17:07Z OWNER

I'm going to move the remaining work into a pull request.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Port Datasette to ASGI 324188953  
504716988 https://github.com/simonw/datasette/issues/272#issuecomment-504716988 https://api.github.com/repos/simonw/datasette/issues/272 MDEyOklzc3VlQ29tbWVudDUwNDcxNjk4OA== simonw 9599 2019-06-23T03:43:46Z 2019-06-23T15:15:26Z OWNER

OK, it's beginning to shape up now. Next steps:

  • [x] Static file support (including for plugins) - plus tests
  • [x] Streaming support so the CSV tests will pass
  • [x] Ability to download the database file
  • [x] Implement missing-slash redirects
 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Port Datasette to ASGI 324188953  
504760061 https://github.com/simonw/datasette/issues/272#issuecomment-504760061 https://api.github.com/repos/simonw/datasette/issues/272 MDEyOklzc3VlQ29tbWVudDUwNDc2MDA2MQ== simonw 9599 2019-06-23T15:02:52Z 2019-06-23T15:02:52Z OWNER

Tests are failing on Python 3.5: https://travis-ci.org/simonw/datasette/jobs/549380098 - error is TypeError: the JSON object must be str, not 'bytes'

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Port Datasette to ASGI 324188953  
504759842 https://github.com/simonw/datasette/issues/272#issuecomment-504759842 https://api.github.com/repos/simonw/datasette/issues/272 MDEyOklzc3VlQ29tbWVudDUwNDc1OTg0Mg== simonw 9599 2019-06-23T15:00:06Z 2019-06-23T15:00:06Z OWNER

I also need to actually take advantage of raw_path such that pages like https://fivethirtyeight.datasettes.com/fivethirtyeight/twitter-ratio%2Fsenators can be correctly served.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Port Datasette to ASGI 324188953  
504759683 https://github.com/simonw/datasette/issues/272#issuecomment-504759683 https://api.github.com/repos/simonw/datasette/issues/272 MDEyOklzc3VlQ29tbWVudDUwNDc1OTY4Mw== simonw 9599 2019-06-23T14:57:50Z 2019-06-23T14:57:50Z OWNER

All of the tests are now passing!

I still need a solution for this:

https://github.com/simonw/datasette/blob/5bd510b01adae3f719e4426b9bfbc346a946ba5c/datasette/app.py#L706-L714

I think the answer is ASGI lifespan, which is supported by Uvicorn. https://asgi.readthedocs.io/en/latest/specs/lifespan.html#startup

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Port Datasette to ASGI 324188953  
504754552 https://github.com/simonw/datasette/issues/272#issuecomment-504754552 https://api.github.com/repos/simonw/datasette/issues/272 MDEyOklzc3VlQ29tbWVudDUwNDc1NDU1Mg== simonw 9599 2019-06-23T13:53:39Z 2019-06-23T13:53:39Z OWNER

Next test to fix (because by new test harness doesn't actually obey the allow_redirects= parameter): ``` ___ test_database_page_redirects_with_url_hash _____

app_client_with_hash = <tests.fixtures.TestClient object at 0x10981f240>

def test_database_page_redirects_with_url_hash(app_client_with_hash):
    response = app_client_with_hash.get("/fixtures", allow_redirects=False)
    assert response.status == 302
    response = app_client_with_hash.get("/fixtures")

  assert "fixtures" in response.text

E AssertionError: assert 'fixtures' in '' E + where '' = <tests.fixtures.TestResponse object at 0x10981f550>.text ```

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Port Datasette to ASGI 324188953  
504754433 https://github.com/simonw/datasette/issues/272#issuecomment-504754433 https://api.github.com/repos/simonw/datasette/issues/272 MDEyOklzc3VlQ29tbWVudDUwNDc1NDQzMw== simonw 9599 2019-06-23T13:51:53Z 2019-06-23T13:51:53Z OWNER

CSV tests all pass as of https://github.com/simonw/datasette/commit/ff9efa668ebc33f17ef9b30139960e29906a18fb

This code could be a lot neater though. At the very least I'm going to refactor datasette/utils.py into a datasette/utils package and put all of my new ASGI utilities in datasette/utils/asgi.py

The way I implemented streaming on top of a writer object (inspired by Sanic) is a bit of a weird hack. I think I'd rather use an abstraction where my view functions can yield chunks of body data.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Port Datasette to ASGI 324188953  
504711468 https://github.com/simonw/datasette/issues/272#issuecomment-504711468 https://api.github.com/repos/simonw/datasette/issues/272 MDEyOklzc3VlQ29tbWVudDUwNDcxMTQ2OA== simonw 9599 2019-06-23T01:36:33Z 2019-06-23T01:36:33Z OWNER

Published an in-progress demo:

datasette publish now fixtures.db -n datasette-asgi-early-demo --branch=asgi

Here it is: https://datasette-asgi-early-demo-qahhxctqpw.now.sh/

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Port Datasette to ASGI 324188953  
504710331 https://github.com/simonw/datasette/issues/272#issuecomment-504710331 https://api.github.com/repos/simonw/datasette/issues/272 MDEyOklzc3VlQ29tbWVudDUwNDcxMDMzMQ== simonw 9599 2019-06-23T01:08:45Z 2019-06-23T01:08:45Z OWNER

Lots still to do:

  • Static files are not being served
  • Streaming CSV files don't work
  • Tests all fail
  • Some URLs (e.g. the 'next' link on tables) are incorrect

I'm going to work on getting the unit test framework to be ASGI-compatible next.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Port Datasette to ASGI 324188953  

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
, [performed_via_github_app] TEXT);
CREATE INDEX [idx_issue_comments_issue]
                ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
                ON [issue_comments] ([user]);