I'm going to ignore the permissions issue for the moment - I'll allow people to select any permissions they like in any of the databases or tables that are visible to them (don't want to leak the existence of databases/tables to users who shouldn't be able to see them).

I think the value of getting this working outweights any potential confusion from not using finely grained permission checks to decide if the user should be able to apply a permission or not.

The tokens themselves won't be able to perform insert-row or similar if the user doesn't have the ability to do that, even if they selected that checkbox.

Here's the checkbox prototype: ```diff diff --git a/datasette/templates/create_token.html b/datasette/templates/create_token.html index a94881ed..1795ebaf 100644 --- a/datasette/templates/create_token.html +++ b/datasette/templates/create_token.html @@ -2,11 +2,20 @@

{% block title %}Create an API token{% endblock %}

+{% block extra_head %} +<style type="text/css"> +#restrict-permissions label { + display: inline; + width: 90%; +} +</style> +{% endblock %} + {% block content %}

Create an API token

-

This token will allow API access with the same abilities as your current user.

This token will allow API access with the same abilities as your current user, {{ request.actor.id }}

{% if errors %} {% for error in errors %} @@ -27,8 +36,39 @@ - + +

{% if token %}

Slightly tricky thing here is that it should only show permissions that the user themselves has - on databases and tables that they have permission to access.

I have a nasty feeling this may require looping through everything and running every permission check, which could get very expensive if there are plugins involved that do their own storage check to resolve a permission.

It's that classic permission system problem: how to efficiently iterate through everything the user has permission to do in one go?

Might be that I have to punt on that, and show the user a list of permissions to select that they might not actually have ability for.

Checkbox interface looks like this. It's not beautiful but it's good enough for the moment:

I think checkboxes will work well.

Here's the data I get back from them (as post_vars()):

{'all:debug-menu': 'on', 'all:insert-row': 'on', 'expire_duration': '', 'expire_type': '', 'table:fixtures:delete-row': 'on', 'table:fixtures:drop-table': 'on', 'table:fixtures:view-query': 'on'}

My <select multiple> prototype: ```diff diff --git a/datasette/templates/create_token.html b/datasette/templates/create_token.html index a94881ed..5bd641cc 100644 --- a/datasette/templates/create_token.html +++ b/datasette/templates/create_token.html @@ -6,7 +6,7 @@

Create an API token

-

This token will allow API access with the same abilities as your current user.

This token will allow API access with the same abilities as your current user, {{ request.actor.id }}

{% if errors %} {% for error in errors %} @@ -28,6 +28,36 @@ + +

{% if token %} diff --git a/datasette/views/special.py b/datasette/views/special.py index 30345d14..9d0fcd31 100644 --- a/datasette/views/special.py +++ b/datasette/views/special.py @@ -231,7 +231,17 @@ class CreateTokenView(BaseView): return await self.render( ["create_token.html"], request, - {"actor": request.actor}, + { + "actor": request.actor, + "all_permissions": self.ds.permissions.keys(), + "database_permissions": [key for key, value in self.ds.permissions.items() if value.takes_database], + "table_permissions": [key for key, value in self.ds.permissions.items() if value.takes_resource], + "databases": self.ds.databases.keys(), + "database_with_tables": [{ + "database": db.name, + "tables": await db.table_names(), + } for db in self.ds.databases.values()], + }, )

 async def post(self, request):

```

Got an option group thing working:

But... it strikes me that any time you're considering a <select multiple> like this a nested list of checkboxes would actually be better - easier for people to use.

I'm experimenting with a <select multiple> for this.

The usability for keyboards is still pretty awful, but it's a niche enough feature that maybe that's OK for the moment?

javascript var select = document.querySelector('select'); var selected = Array.from(temp0.options).filter(o => o.selected).map(o => o.value)

• [x] I should add a --database example to that help text.

https://latest.datasette.io/-/create-token currently looks like this:

As a reminder, the CLI options that this needs to provide an alternative to are:

https://github.com/simonw/datasette/blob/d4b98d3924dec625a99236e65b1b169ff957381f/docs/cli-reference.rst#L619-L638

Most places I use that exception at the moment set their own non-500 status error: ``` datasette % rg DatasetteError datasette/handle_exception.py 7:from .views.base import DatasetteError 33: elif isinstance(exception, DatasetteError):

datasette/filters.py 2:from datasette.views.base import DatasetteError 22: raise DatasetteError("_where= is not allowed", status=403) 141: raise DatasetteError(

datasette/views/table.py 34:from .base import BaseView, DataView, DatasetteError, ureg, _error 178: raise DatasetteError( 192: raise DatasetteError( 390: raise DatasetteError("Cannot use _sort and _sort_desc at the same time") 394: raise DatasetteError(f"Cannot sort table by {sort}") 400: raise DatasetteError(f"Cannot sort table by {sort_desc}")

datasette/views/base.py 39:class DatasetteError(Exception): 219: raise DatasetteError(str(e), title="Invalid SQL", status=400) 222: raise DatasetteError(str(e)) 224: except DatasetteError: 382: raise DatasetteError( 402: raise DatasetteError(str(e), title="Invalid SQL", status=400) 405: raise DatasetteError(str(e)) 407: except DatasetteError:

datasette/views/table2.py 28:from .base import DataView, DatasetteError, ureg 296: raise DatasetteError( 310: raise DatasetteError( 472: raise DatasetteError("Cannot use _sort and _sort_desc at the same time") 476: raise DatasetteError(f"Cannot sort table by {sort}") 482: raise DatasetteError(f"Cannot sort table by {sort_desc}")

datasette/views/database.py 31:from .base import BaseView, DatasetteError, DataView, _error 188: raise DatasetteError("Invalid database", status=404) 190: raise DatasetteError("Cannot download in-memory databases", status=404) 194: raise DatasetteError("Cannot download database", status=404) ```

https://github.com/simonw/datasette/blob/d4b98d3924dec625a99236e65b1b169ff957381f/datasette/views/table.py#L392-L400

When you run --root you need to follow the special link that gets output to the console:

% datasette --root http://127.0.0.1:8001/-/auth-token?token=036d8055cc8000e9667f21c1dd08722a9358c066463873ad9566d23d88765c52 INFO: Started server process [53934] INFO: Waiting for application startup. INFO: Application startup complete. That /-/auth-token?... link is the one that sets the cookie and lets you in.

I may have found the reason. I don't think the --root option is taking effect.

Visited: http://127.0.0.1:8001/-/permissions

Should make sure that every API that returns an object as the top level (that's almost all of them) includes "ok": true to indicate no errors.

I'm going to hide the options for reducing the scope of the token inside a details/summary element.

Also you should need update-row permission to use the "replace": true option - I should add that rule to /-/insert add well.

I landed this already: - #1636

I tested:

{"id": "root", "_r": {"a": "view-table"}}

New documentation: https://docs.datasette.io/en/latest/authentication.html#restricting-the-actions-that-a-token-can-perform

For the UI: I think I'm going to dump a whole bunch of form elements on the page (so you can set up to 3 of each category of limit without any JavaScript), then add JavaScript that hides all but one of the options and gives you a "add another" widget that adds multiple more.

Decided to do the /-/create-token UI in a separate ticket: - #1947

Demo of the new feature: % datasette create-token --secret s root dstok_eyJhIjoicm9vdCIsInRva2VuIjoiZHN0b2siLCJ0IjoxNjcwOTA1NjgwfQ.pqSWOwCSNp678hEWl9l5o7m1GaM % datasette --get /-/actor.json {"actor": null} % DATASETTE_SECRET=s datasette --get /-/actor.json --token dstok_eyJhIjoicm9vdCIsInRva2VuIjoiZHN0b2siLCJ0IjoxNjcwOTA1NjgwfQ.pqSWOwCSNp678hEWl9l5o7m1GaM {"actor": {"id": "root", "token": "dstok"}}

Two options:

• --header "Authorization: Bearer XXX" which can be used to send any headers
• --token XXX to specify the token, which is then sent using that header

I like the second option more, simply because there are currently no other headers that affect how Datasette works. --token feels obvious and easy to use.

For the tests for datasette create-token it would be useful if datasette --get had a mechanism for sending an Authorization: Bearer X header.

I'm going to move this code into datasette/cli.py - it's a bit unexpected having it live in default_permissions.py like this (I couldn't find the code when I went looking for it earlier).

Help looks like this:

``` Usage: datasette create-token [OPTIONS] ID

Create a signed API token for the specified actor ID

Example:

  datasette create-token root --secret mysecret

To only allow create-table:

  datasette create-token root --secret mysecret \
      --all create-table

Or to only allow insert-row against a specific table:

  datasette create-token root --secret myscret \
      --resource mydb mytable insert-row

Restricted actions can be specified multiple times using multiple --all, --database, and --resource options.

Add --debug to see a decoded version of the token.

Options: --secret TEXT Secret used for signing the API tokens [required] -e, --expires-after INTEGER Token should expire after this many seconds -a, --all ACTION Restrict token to this action -d, --database DB ACTION Restrict token to this action on this database -r, --resource DB RESOURCE ACTION Restrict token to this action on this database resource (a table, SQL view or named query) --debug Show decoded token --plugins-dir DIRECTORY Path to directory containing custom plugins --help Show this message and exit. ```

I just noticed this in the existing code:

https://github.com/simonw/datasette/blob/c5d30b58a1cd1c66bbddcf3561db005543ecaf25/datasette/default_permissions.py#L195-L203

Hard-coding those action names should not be necessary any more, especially now we have datasette.permissions for looking up metadata about the permissions.

Initial prototype of the create-token command changes:

diff diff --git a/datasette/default_permissions.py b/datasette/default_permissions.py index 406dae40..bbe1247e 100644 --- a/datasette/default_permissions.py +++ b/datasette/default_permissions.py @@ -278,17 +278,55 @@ def register_commands(cli): help="Token should expire after this many seconds", type=int, ) + @click.option( + "alls", + "-a", + "--all", + type=str, + multiple=True, + help="Restrict token to this permission", + ) + @click.option( + "databases", + "-d", + "--database", + type=(str, str), + multiple=True, + help="Restrict token to this permission on this database", + ) + @click.option( + "resources", + "-r", + "--resource", + type=(str, str, str), + multiple=True, + help="Restrict token to this permission on this database resource (a table, SQL view or named query)", + ) @click.option( "--debug", help="Show decoded token", is_flag=True, ) - def create_token(id, secret, expires_after, debug): + def create_token(id, secret, expires_after, alls, databases, resources, debug): "Create a signed API token for the specified actor ID" ds = Datasette(secret=secret) bits = {"a": id, "token": "dstok", "t": int(time.time())} if expires_after: bits["d"] = expires_after + if alls or databases or resources: + bits["_r"] = {} + if alls: + bits["_r"]["a"] = list(alls) + if databases: + bits["_r"]["d"] = {} + for database, action in databases: + bits["_r"]["d"].setdefault(database, []).append(action) + if resources: + bits["_r"]["r"] = {} + for database, table, action in resources: + bits["_r"]["r"].setdefault(database, {}).setdefault( + table, [] + ).append(action) token = ds.sign(bits, namespace="token") click.echo("dstok_{}".format(token)) if debug: Still needs tests, plus I'd like it to use abbreviations if available to keep the token length shorter.

I'm going to rename "t" in the magic format to "r" for resource.

I was going to have the CLI command throw an error if you attempt to use a permission that isn't registered with Datasette, but then I remembered that one of the uses for the CLI tool is to create signed tokens that will work against other Datasette instances (via the --secret option) that might have different plugins installed that register different permission names.

So I might have it output warnings instead.

The datasette create-token command will need to be able to do this too.

Right now that command looks like this: ``` % datasette create-token --help Usage: datasette create-token [OPTIONS] ID

Create a signed API token for the specified actor ID

Options: --secret TEXT Secret used for signing the API tokens [required] -e, --expires-after INTEGER Token should expire after this many seconds --debug Show decoded token --help Show this message and exit. % datasette create-token root --secret sec --debug -e 445 dstok_eyJhIjoicm9vdCIsInRva2VuIjoiZHN0b2siLCJ0IjoxNjY4NDA2MjEzLCJkIjo0NDV9.Hd6qRli6xRKkOIRQgZkPO5iN1wM

Decoded:

{ "a": "root", "token": "dstok", "t": 1668406213, "d": 445 } `` (The--debug` bit adds the decoded token.)

Syntax for adding "insert row" for everything, "update row" for all in the "data" database and "delete row" just for the docs / titles table: datasette create-token root --secret sec \ --all insert-row \ --database data update-row \ --table docs titles delete-row The ir / ur / dr options would work too. To add multiple permissions use these options multiple times: datasette create-token root --secret sec \ --all insert-row \ --all delete-row Short versions: -a and -d and -t.

UPDATE: I have decided to use the term resource in the user-facing elements of this feature instead of table, since that can refer to a SQL view and a canned query as well.

So --resource and -r, not -t.

The hardest piece here is the UI. I'm going to implement the CLI command first.

The thing I'm stuck on at the moment is how to implement it such that an allow block for create-table at the root of the metadata will be checked correctly.

Maybe the algorithm when _resolve_metadata_permissions_blocks(datasette, actor, action, resource) is called should do this:

1. If a root permission block matching that action exists, test with that
2. Next, if resource has been passed, check at the database level
3. If the resource included a table/query, check at that level too

So everything is keyed off the incoming action name.

The implementation for this will go here: https://github.com/simonw/datasette/blob/8bf06a76b51bc9ace7cf72cf0cca8f1da7704ea7/datasette/default_permissions.py#L81-L83

Here's the start of the tests (currently marked as xfail):

https://github.com/simonw/datasette/blob/8bf06a76b51bc9ace7cf72cf0cca8f1da7704ea7/tests/test_permissions.py#L652-L689

A bunch of the work for this just landed - in particular the new scheme is now documented (even though it doesn't work yet):

https://docs.datasette.io/en/latest/authentication.html#other-permissions-in-metadata

Documentation for the new hook: https://docs.datasette.io/en/latest/plugin_hooks.html#register-permissions-datasette

This URL is already used for the https://latest.datasette.io/-/permissions tool - but it could include a block on that page that tells you what permissions are available.

This PR ended up bundling part of the implementation of: - #1636

I'm going to be bad an NOT untangle that from this before I merge it.

Codecov Report

Base: 92.00% // Head: 92.03% // Increases project coverage by +0.02% :tada:

Coverage data is based on head (a1317ab) compared to base (e539c1c). Patch coverage: 100.00% of modified lines in pull request are covered.

:exclamation: Current head a1317ab differs from pull request most recent head 94e5c75. Consider uploading reports for the commit 94e5c75 to get more accurate results

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

Actually one last thing: I said that the error would only occur if the permissions differed in some way.

It's this change which triggers the failures: ```diff diff --git a/datasette/app.py b/datasette/app.py index 760063d5..defa9688 100644 --- a/datasette/app.py +++ b/datasette/app.py @@ -707,9 +707,12 @@ class Datasette: ) return crumbs

• async def permission_allowed(self, actor, action, resource=None, default=False):
• async def permission_allowed(self, actor, action, resource=None, default=None): """Check permissions using the permissions_allowed plugin hook""" result = None

• Use default from registered permission, if available

• if default is None and action in self.permissions:
• default = self.permissions[action].default for check in pm.hook.permission_allowed( datasette=self, actor=actor, ```

I'm going to revert that last commit, see if I can get the tests running again and then apply the changes a line at a time to figure out which ones broke things.