home / github

Menu
  • Search all tables
  • GraphQL API

issue_comments

Table actions
  • GraphQL API for issue_comments

4 rows where author_association = "OWNER" and issue = 423316403 sorted by updated_at descending

✖
✖
✖

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date)

user 1

  • simonw 4

issue 1

  • Figure out what to do about table counts in a mutable world · 4 ✖

author_association 1

  • OWNER · 4 ✖
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions issue performed_via_github_app
488558896 https://github.com/simonw/datasette/issues/422#issuecomment-488558896 https://api.github.com/repos/simonw/datasette/issues/422 MDEyOklzc3VlQ29tbWVudDQ4ODU1ODg5Ng== simonw 9599 2019-05-02T05:43:11Z 2019-05-02T05:43:11Z OWNER

I'm doing this with time limits right now, which I think is good enough for the moment.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Figure out what to do about table counts in a mutable world 423316403  
475277571 https://github.com/simonw/datasette/issues/422#issuecomment-475277571 https://api.github.com/repos/simonw/datasette/issues/422 MDEyOklzc3VlQ29tbWVudDQ3NTI3NzU3MQ== simonw 9599 2019-03-21T15:30:03Z 2019-03-21T15:30:03Z OWNER

It would be useful to be able to detect if a table is a rowid table or not.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Figure out what to do about table counts in a mutable world 423316403  
474890920 https://github.com/simonw/datasette/issues/422#issuecomment-474890920 https://api.github.com/repos/simonw/datasette/issues/422 MDEyOklzc3VlQ29tbWVudDQ3NDg5MDkyMA== simonw 9599 2019-03-20T15:39:58Z 2019-03-20T15:39:58Z OWNER

The page with the most table counts on it is the index page: https://fivethirtyeight.datasettes.com/fivethirtyeight-b76415d

If I paginate this (which needs to happen anyway for Datasette Library #417) the impact here won't be as bad. I could even load in the table row counts asynchronously via JavaScript?

Bigger problem is this total summed count representation on the homepage:

I think that feature just won't be feasibly against large databases in a mutable world. Maybe we consider to show that total but only for immutable databases? May be easier just to drop it entirely (we will still show the table count).

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Figure out what to do about table counts in a mutable world 423316403  
474888132 https://github.com/simonw/datasette/issues/422#issuecomment-474888132 https://api.github.com/repos/simonw/datasette/issues/422 MDEyOklzc3VlQ29tbWVudDQ3NDg4ODEzMg== simonw 9599 2019-03-20T15:34:37Z 2019-03-20T15:34:37Z OWNER

Here's a trick for lower bound counts which looks like it might actually work. Consider the following queries:

select count(*) from ( select rowid from [most-common-name/surnames] limit 1000 ) https://fivethirtyeight.datasettes.com/fivethirtyeight-b76415d?sql=select+count%28*%29+from+%28%0D%0A++select+rowid+from+%5Bmost-common-name%2Fsurnames%5D+limit+1000%0D%0A%29

Takes 0.827ms (it took longer with select * from in the subquery).

Same query but with limit 10,000:

https://fivethirtyeight.datasettes.com/fivethirtyeight-b76415d?sql=select+count%28*%29+from+%28%0D%0A++select+rowid++from+%5Bmost-common-name%2Fsurnames%5D+limit+10000%0D%0A%29

Took 2.335ms

With 100,000 limit:

https://fivethirtyeight.datasettes.com/fivethirtyeight-b76415d?sql=select+count%28*%29+from+%28%0D%0A++select+rowid++from+%5Bmost-common-name%2Fsurnames%5D+limit+100000%0D%0A%29

Took 27.558ms

So one solution here would be to pick an upper bound (maybe 100,001) and use this query, which should give an accurate count below that upper bound but allow us to show "100,000+" as a count if the table exceeds that boundary.

Maybe the boundary is a config setting? Also, if a tighter timeout (maybe 20ms) is exceeded for that boundary we could halve it and try again.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Figure out what to do about table counts in a mutable world 423316403  

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options:

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
, [performed_via_github_app] TEXT);
CREATE INDEX [idx_issue_comments_issue]
                ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
                ON [issue_comments] ([user]);
Powered by Datasette · Queries took 23.8ms · About: github-to-sqlite
  • Sort ascending
  • Sort descending
  • Facet by this
  • Hide this column
  • Show all columns
  • Show not-blank rows