issue_comments

I landed canned query writes. This feature can now be considered complete: https://datasette.readthedocs.io/en/latest/authentication.html

The canned queries feature is gaining permissions support in #800.

I can close this issue once I've expanded out this page of documentation https://datasette.readthedocs.io/en/latest/authentication.html - and published at least one plugin and/or feature that takes advantage of this new mechanism.

Debugging tool idea: /-/permissions page which shows you the actor and lets you type in the strings for action, resource_type and resource_identifier - then shows you EVERY plugin hook that would have executed and what it would have said, plus when the chain would have terminated.

Bonus: if you're logged in as the root user (or a user that matches some kind of permission check, maybe a check for permissions_debug) you get to see a rolling log of the last 30 permission checks and what the results were across the whole of Datasette. This should make figuring out permissions policies a whole lot easier.

Plugin idea: datasette-allow-all - really simple plugin which just says "yes" to every permission check.

https://latest.datasette.io/-/actor is now live (it returns null because there's no current way to sign into the latest.datasette.io site - not even with a fake ds_actor cookie because there's no way to know what that site's random secret is).

Some next steps:

• Try out a branch of datasette-auth-github that builds on these new plugin hooks
• Build a datasette-api-tokens plugin which implements Authorization: bearer xxx token support for API access
• Maybe prototype up a datasette-user-accounts plugin which supports username/password accounts and allows an admin user to create/delete them
• Do more work on writable canned queries in #698 and see what they look like if they take advantage of the permissions hook (to restrict some to only allowing authenticated users)

I rebased in #783 so all of this is on master now.

I should add an entire page to the documentation describing Datasette authentication.

OK, the implementation in PR #783 is in a good state now - it implements the new plugin hooks with tests and documentation, plus it implements this:

$ datasette . --root
http://127.0.0.1:8001/-/auth-token?token=3ca9ee460a6451142389351d19b147bce27d2a785dfb6b5a74f82211be1ede49
...

That URL, when clicked, will set a cookie for the {"id": "root"} user. The cookie is respected and used to populate scope["actor"].

I'm going to merge that pull request and continue working on this stuff on master.

I quite like root - it supports the idea that best practice is to NOT do things as the root account, but to use a plugin to set up separate accounts for different purposes.

Maybe the default single account should be called something other than admin? The problem with admin is that it sounds like more of a role - in larger installations one can expect multiple admins. root may be better since there's clearly only one root account. Bit of a technical term though.

Again I will use exploratory prototyping to inform a decision on the minimum subset design for the actor dictionary.

Keeping the structure of the actor dictionary completely undefined doesn't make sense if Datasette is going to ship with a default authentication mechanism for admin users.

I'm going to define a small set of required keys for the actor dictionary, and enforce them in code.

But which keys? I feel I need a unique key representing the identity of the actor, plus a key that can be displayed in the "You are logged in as X" navigation. Maybe these are the same key?

So the single required key could be id. Problem is: is that a string or an integer? Some use-cases may call for an integer, which matches to how SQLite auto incrementing primary keys work. admin is a string.

Maybe id is required, name is optional - but if name is present then the "You are logged in as..." uses that in preference to id. id has to be a string, and if you want to store integer IDs in your database you need to remember to convert them to a string in your actor_from_request implementation.

This also means some writable canned queries can allow writes from unauthenticated users (for stuff like feedback forms), while others can require an authenticated user - all with core Datasette without any plugins needed.

I believe that this plugin hook design is flexible enough that role-based permissions could be built on top of it as a separate plugin.

Would be good to check that with a proof of concept though.

I'm going to draw the line here: default Datasette supports authentication but only for a single user account ("admin"). Plugins can then add support for multiple user accounts, social auth, SSO etc.

HTTP Basic auth would be a good default option. No need to build a custom login UI for it.

I think I want to keep full username/password authentication against a database table as a plugin. I'll experiment with Jupyter-style URLs as a starting point.

If Datasette were to support authentication out-of-the-box, without plugins (which makes more sense with writable canned queries, #698) what would that look like?

Some options:

• Jupyter notebook style: output a magic URL on the console with a one-time token to authenticate the user as an "admin"
• Really simple password authentication - via an environment variable perhaps?
• SQL based authentication: I was going to do this as a plugin, but maybe it should be default? A way of configuring a SQL query which can be used to authenticate a user based on their username and password.

Idea: add a /-/actor.json special page which JSON dumps out the current request.scope["actor"] - so you can easily test how your request has been authenticated.

Robust testing of permissions is really important. I should think about utilities I may be able to add to Datasette's unit testing tools that make it as easy as possible to confirm which permission checks were carried out on a specific HTTP request.

That way I can set a good example that any Datasette plugin which makes permission checks can follow.

The branch is now usable! Next step: write some experimental plugins that exercise some real authentication use-cases with it.

I'm going to add an awaitable utility method to the Datasette class for checking permissions:

await datasette.permission_allowed(actor, action, resource_type, resource_identifier)

The second two arguments will be optional.

Debugging permissions is going to be important. Optional tooling that supports the following would be useful:

• Log every check to permission_allowed to the console - optionally with tracebacks showing where in the code the check was made
• Log every check to the https://latest.datasette.io/?_trace=1 output
• A tool that shows you exactly what permissions the current authenticated user/entity has
• A tool showing all available permissions

That last one is tricky if permissions are just strings that might be passed to permission_allowed - so maybe there needs to be a plugin hook that lets plugins register their permissions, such that they can be introspected later on? A register_permission_actions() hook that returns a list of permission action strings (or objects of some sort) perhaps.

I could bake some permission checks into default Datasette, which are all treated as allow by default but can then be locked down by plugins. Maybe the following:

permission_allowed(request.actor, "execute-sql", "database", "name-of-database")

Checks that current user can execute arbitrary SQL queries against a specific database (or use the ?_where= feature). Equivalent to current allow_sql setting.

permission_allowed(request.actor, "download-database", "database", "name-of-database")

Can the user download the database file? Like allow_download.

Maybe one for allow_csv_stream too.

Having a permission check (defaulting to True) on every single "view" would be useful:

• view_index
• view_database
• view_table
• view_row
• view_query
• view_special (for /-/versions and so on)

I started sketching this out in the authentication branch. Here's the documentation so far: https://github.com/simonw/datasette/blob/8871c20/docs/plugins.rst#actor_from_requestdatasette-request

I think there are two hooks here:

actor_from_request(datasette, request) - returns None or a dictionary.

• datasette is a Datasette instance - useful for things like reading plugin configuration or executing queries
• request is a Request object - which means ASGI scope can be accessed as request.scope

A non-None value means the request is authenticated in some way. The shape of that dictionary is entirely undefined.

The second hook is for checking permissions. It can look something like this:

permission_allowed(actor, action, resource_type, resource_identifier)

• actor = the dictionary that was returned by actor_from_scope
• action = a string representing the action to be performed, e.g. edit-schema
• resource_type = a string representing the type of resource being acted on, e.g. table
• resource_identifier = a string (or maybe tuple?) representing the specific resource, e.g. the table name

I don't know if Datasette should provide default implementations of these hooks. It may be that leaving them completely up to plugins is the way to go.

I think I need to prototype this quickly to start feeling for how well it might work.

auth_from_scope(datasette, scope) needs to be able to return an awaitable which is then awaited - so it can execute database queries.

Maybe call that actor_from_request(datasette, request) instead.

I like "actor" better than "entity" to mean "the user or API key that is authenticated for this request".

I'm going to use "resource" instead of "subject" - updating the design comment again.

In AWS IAM world the following terminology is used: https://aws.amazon.com/iam/features/manage-permissions/

Permissions are granted to IAM entities (users, groups, and roles) [...]

To assign permissions to a user, group, role, or resource, you create a policy that lets you specify:

• Actions – Which AWS service actions you allow. For example, you might allow a user to call the Amazon S3 ListBucket action. Any actions that you don't explicitly allow are denied.
• Resources – Which AWS resources you allow the action on. For example, what Amazon S3 buckets will you allow the user to perform the ListBucket action on? Users cannot access any resources that you do not explicitly grant permissions to.
• Effect – Whether to allow or deny access. Because access is denied by default, you typically write policies where the effect is to allow.
• Conditions – Which conditions must be present for the policy to take effect. For example, you might allow access only to the specific S3 buckets if the user is connecting from a specific IP range or has used multi-factor authentication at login.

My usage of the term subject here to mean "the thing I am checking I have permission to interact with, e.g. a database table" may be misleading. https://stackoverflow.com/questions/4989063/what-is-the-meaning-and-difference-between-subject-user-and-principal for example shows that JAAS (Java Authentication and Authorization Service) defines subject as "The purpose of the Subject is to represent the authenticated user".

I'm changing auth to actor and updating the above design comment.

Very nice! Thank you for sharing that :+1: :) Will try it out!

I did have a bit of trouble with this one-off plugin getting it to load in the correct order - since I need authentication to work if EITHER the one-off plugin spots a token or my datasette-auth-github plugin authenticates the user.

That's why I want authentication as a core Datasette concept - so plugins like these can easily play together in a predictable manner.

I implemented bearer tokens in a private project of mine as a one-off plugin. I'm going to extract that out into a installable plugin soon. For the moment, my plugins/token_auth.py file looks like this: ```python from datasette import hookimpl import secrets

class TokenAuth: def init( self, app, secret, auth, ): self.app = app self.secret = secret self.auth = auth

async def __call__(self, scope, receive, send):
    if scope.get("type") != "http":
        return await self.app(scope, receive, send)

    authorization = dict(scope.get("headers") or {}).get(b"authorization") or b""
    expected = "Bearer {}".format(self.secret).encode("utf8")

    if secrets.compare_digest(authorization, expected):
        scope = dict(scope, auth=self.auth)

    return await self.app(scope, receive, send)

@hookimpl(trylast=True) def asgi_wrapper(datasette): config = datasette.plugin_config("token-auth") or {} secret = config.get("secret") auth = config.get("auth")

def wrap_with_asgi_auth(app):
    return TokenAuth(app, secret=secret, auth=auth,)

return wrap_with_asgi_auth

Then I have the following in `metadata.json`:json { "plugins": { "token-auth": { "auth": { "name": "token-bot" }, "secret": { "$env": "TOKEN_SECRET" } } } } `` And aTOKEN_SECRET` environment variable.

Authorization: bearer xxx auth for API keys is a plus plus for me. Looked into just adding this into your Flask logic but learned this project doesn't use flask. Interesting 🤔

So maybe this is all handled by plugin hooks?

auth_from_scope(datasette, scope) would be the main one - for deciding if a user should be authenticated based on data from the scope.

How would a permissions hook work though?

Another problem this would solve: if you want multiple authentication mechanisms - GitHub auth for users, Authorization: bearer xxx auth for API keys - the order in which they run might end up mattering.

I dealt with this a bit in https://github.com/simonw/datasette-auth-github/issues/59

But having an authentication plugin hook - where playing get to decide if a user should be authenticated based on the incoming ASGI scopes - would be neater.