issue_comments

Table actions

7 rows where issue = 770598024 and "updated_at" is on date 2020-12-18 sorted by updated_at descending

✖
✖
✖

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date)

user 1

issue 1

  • Efficiently calculate list of databases/tables a user can view · 7 ✖

author_association 1

id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions issue performed_via_github_app
747921195 https://github.com/simonw/datasette/issues/1152#issuecomment-747921195 https://api.github.com/repos/simonw/datasette/issues/1152 MDEyOklzc3VlQ29tbWVudDc0NzkyMTE5NQ== simonw 9599 2020-12-18T07:31:25Z 2020-12-18T07:31:25Z OWNER

It's also a really good fit for the new mechanism that's coming together in #1150.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Efficiently calculate list of databases/tables a user can view 770598024  
747920852 https://github.com/simonw/datasette/issues/1152#issuecomment-747920852 https://api.github.com/repos/simonw/datasette/issues/1152 MDEyOklzc3VlQ29tbWVudDc0NzkyMDg1Mg== simonw 9599 2020-12-18T07:30:22Z 2020-12-18T07:30:22Z OWNER

Redefining all Datasette permissions in terms of SQL queries that return the set of databases and tables that the user is allowed to interact with does feel VERY Datasette-y.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Efficiently calculate list of databases/tables a user can view 770598024  
747920087 https://github.com/simonw/datasette/issues/1152#issuecomment-747920087 https://api.github.com/repos/simonw/datasette/issues/1152 MDEyOklzc3VlQ29tbWVudDc0NzkyMDA4Nw== simonw 9599 2020-12-18T07:27:58Z 2020-12-18T07:28:30Z OWNER

I want to keep the existing metadata.json "allow" blocks mechanism working. Note that if you have 1,000 tables and a permissions policy you won't be using "allow" blocks, you'll be using a more sophisticated permissions plugin instead.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Efficiently calculate list of databases/tables a user can view 770598024  
747919782 https://github.com/simonw/datasette/issues/1152#issuecomment-747919782 https://api.github.com/repos/simonw/datasette/issues/1152 MDEyOklzc3VlQ29tbWVudDc0NzkxOTc4Mg== simonw 9599 2020-12-18T07:27:01Z 2020-12-18T07:27:01Z OWNER

Perhaps this can be solved by keeping the existing plugin hooks and adding new, optional ones for bulk lookups.

If your plugin doesn't implement the bulk lookup hooks Datasette will do an inefficient loop through everything checking permissions on each one.

If you DO implement it you can speed things up dramatically.

Not sure if this would solve the homepage problem though, where you might need to run 1,000 table permission checks. That's more a case where you want to think in terms of a SQL where clause.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Efficiently calculate list of databases/tables a user can view 770598024  
747893423 https://github.com/simonw/datasette/issues/1152#issuecomment-747893423 https://api.github.com/repos/simonw/datasette/issues/1152 MDEyOklzc3VlQ29tbWVudDc0Nzg5MzQyMw== simonw 9599 2020-12-18T06:18:24Z 2020-12-18T06:18:24Z OWNER

What would Datasette's permission hooks look like if they all dealt with sets of items rather than individual items? So plugins could return a set of items that the user has permission to access, or even a WHERE clause?

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Efficiently calculate list of databases/tables a user can view 770598024  
747892731 https://github.com/simonw/datasette/issues/1152#issuecomment-747892731 https://api.github.com/repos/simonw/datasette/issues/1152 MDEyOklzc3VlQ29tbWVudDc0Nzg5MjczMQ== simonw 9599 2020-12-18T06:16:29Z 2020-12-18T06:16:29Z OWNER

One enormous advantage I have is that after #1150 I will have a database table full of databases and tables that I can execute queries against.

This means I could calculate visible tables using SQL where clauses, which should be easily fast enough even against ten thousand plus tables.

The catch is the permissions hooks. Since I haven't hit Datasette 1.0 yet maybe I should redesign those hooks to work against the new in-memory database schema stuff?

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Efficiently calculate list of databases/tables a user can view 770598024  
747891854 https://github.com/simonw/datasette/issues/1152#issuecomment-747891854 https://api.github.com/repos/simonw/datasette/issues/1152 MDEyOklzc3VlQ29tbWVudDc0Nzg5MTg1NA== simonw 9599 2020-12-18T06:14:09Z 2020-12-18T06:14:15Z OWNER

This is a classic challenge in permissions systems. If I want Datasette to be able to handle thousands of tables I need a reasonable solution for it.

Twitter conversation: https://twitter.com/simonw/status/1339791768842248192

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Efficiently calculate list of databases/tables a user can view 770598024  

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
, [performed_via_github_app] TEXT);
CREATE INDEX [idx_issue_comments_issue]
                ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
                ON [issue_comments] ([user]);