issue_comments

API design: (**UPDATE: this was later changed to POST /db/table/-/insert)

POST /db/table Authorization: Bearer xxx Content-Type: application/json { "row": { "id": 1, "name": "New record" } } Returns: 201 Created { "row": { "id": 1, "name": "New record" } } You can omit optional fields in the input, including the ID field. The returned object will always include all fields - and will even include rowid if your object doesn't have a primary key of its own.

I decided to use "row" as the key in both request and response, to preserve space for other future keys - one that tells you that the table has been created, for example.

I'm going to implement the first version of this token mechanism using permissions that exist already. Right now that's:

https://docs.datasette.io/en/0.62/authentication.html#built-in-permissions

Here are the shortcuts I'll use for them:

• view-instance - vi
• view-database - vd
• view-database-download - vdd
• view-table - vt
• view-query - vq
• execute-sql - es

Might be neat for API tokens to be signed with an additional secret than can be rotated independently of DATASETTE_SECRET itself, in order to invalidate all tokens without needing to invalidate logged in users too.

But again, I don't want to implement something like that until I see an actual need for it.

... also, maybe there should be a UI (perhaps on that page) for resetting the Datasette secret? Useful for emergency invalidation of all tokens.

No, I'm not going to build that unless someone asks for it. Restarting the server with a fresh secret should be easy enough.

If you start Datasette without providing a DATASETTE_SECRET environment variable of --secret option it creates a random signing secret that only lasts for the lifetime of the server.

This means any signed API tokens you create will stop working if the server restarts.

I think the /-/create-token UI should know when this happens and show a warning message about it, to avoid confusion.

Here's what that example looks like signed: python from datasette.app import Datasette ds = Datasette() ds.sign('{"t":{"a":["ir","ur","dr"],"d":{"fixtures":["ir","ur","dr"]},"t":{"fixtures":{"searchable":["ir"]}}}}') .eJxTqo5RKolRsgJSiUAqOkYpsyhGSSdGqRRCpQCpWBANUZOWWVFSWpRajFNprQ7cPCS1QF5xamJRckZiUk4qQm9sLRAoAQCC8yph.O0Gaej6-VOLbbtPq7xU6T77jEO0 That's 129 characters.

Note that Datasette doesn't have its own mechanism for signing things for a specific duration yet: https://docs.datasette.io/en/stable/internals.html#sign-value-namespace-default

So I'll need to add a "e": 1666739744 field with the UTC timestamp at which the token should expire.

Token design concept: json { "t": { "a": ["ir", "ur", "dr"], "d": { "fixtures": ["ir", "ur", "dr"] }, "t": { "fixtures": { "searchable": ["ir"] } } } } That JSON would be minified and signed.

Minified version of the above looks like this (101 characters):

{"t":{"a":["ir","ur","dr"],"d":{"fixtures":["ir","ur","dr"]},"t":{"fixtures":{"searchable":["ir"]}}}}

The "t" key shows this is a token that as a default API key.

"a" means "all" - these are permissions that have been granted on all tables and databases.

"d" means "databases" - this is a way to set permissions for all tables in a specific database.

"t" means "tables" - this lets you set permissions at a finely grained table level.

Then the permissions themselves are two character codes which are shortened versions - so:

• ir = insert-row
• ur = update-row
• dr = delete-row

Maybe these tokens can be restricted to specific databases and tables when they are first created?

Since they're signed tokens, I could bundle a bunch of extra stuff in them - this token is allowed to do these permissions against these tables/rows for example.

General wisdom seems to be that 8KB is a sensible maximum length for this kind of token, which is easily long enough to fit in a bunch of database / table / permissions.

Interesting open question: how should validation errors (if any) be returned?

The two forms of validation I can think of at first are:

• Missing keys which are marked as not null in the schema
• Keys that do not match to existing columns (if you didn't pass "alter": true, an option I am considering)

Here's the implementation of datasette-auth-tokens: https://github.com/simonw/datasette-auth-tokens/blob/main/datasette_auth_tokens/init.py

API authentication will be via Authorization: Bearer XXX request headers.

I'm inclined to add a default token mechanism to Datasette based on tokens that are signed with the DATASETTE_SECRET. Maybe the root user can access /-/create-token which provides a UI for generating a time-limited signed token? Could also have a datasette create-token command for creating such tokens at the command-line.

Plugins can then define alternative ways of creating tokens, such as the existing https://datasette.io/plugins/datasette-auth-tokens plugin.

It may turn out that it makes sense to also add a UI for these actions as part of this project. That's still to be determined.

This is going to need a whole bunch of new permissions.

To review: the existing set of permissions are listed here: https://docs.datasette.io/en/0.62/authentication.html#built-in-permissions

• view-instance
• view-database
• view-database-download
• view-table
• view-query
• execute-sql
• permissions-debug
• debug-menu

I'm going to reuse database terminology for the new permissions. So first draft of those is:

• insert-row
• update-row
• delete-row
• create-table
• drop-table
• alter-table

I'm going to treat this as a bit of a research spike, at least until I like the direction it is going enough to commit to it.

Refs: - #1831

Looks like the new breadcrumbs code can't handle the case where request is None.

Need a test that demonstrates this too.

https://latest.datasette.io/_internal now looks like this:

Tested my fix with this metadata.yml: yaml databases: fixtures: allow: id: root tables: 123_starts_with_digits: allow: true Signed in as root I saw this - showing that the 123_starts_with_digits table is public:

Here's the code at fault: https://github.com/simonw/datasette/blob/78dad236df730212aa7172f885fd8ec575f0d3ad/datasette/views/database.py#L67-L116

Those checks aren't doing the new cascading permissions thing added in #1829 which means they can't tell that an anonymous user would not be able to se those tbles and queries and views.

Should do something like this instead:

python view_visible, view_private = await self.ds.check_visibility( request.actor, permissions=[ ("view-table", (database, view_name)), ("view-database", database), "view-instance", ], )

Visit https://latest.datasette.io/login-as-root and then:

https://latest.datasette.io/

https://latest.datasette.io/_internal/columns

https://latest.datasette.io/_internal/columns/_internal,columns,cid

https://latest.datasette.io/_internal/from_hook

That's all as it should be.

Updated docs for check_visibility(): https://docs.datasette.io/en/latest/internals.html#await-check-visibility-actor-action-none-resource-none-permissions-none

I'm going to construct a metadata.yml which makes various databases and tables visible or invisible, then browse them using the root user.

block-instance.yml:

yaml allow: id: root block-database.yml: yaml databases: fixtures: allow: id: root block-table.yml: yaml databases: fixtures: tables: searchable: allow: id: root block-query.yml: yaml databases: fixtures: queries: two: sql: select 1 + 1 allow: id: root https://gist.github.com/simonw/2d007ebe43de46d44499c77a2a291756 - checkout that Gist to get all four.

I manually tested all four scenarios with root and non-root users and confirmed that they worked correctly and padlocks were shown in the right places.

Useful test: if you sign in as root to https://latest.datasette.io/_internal/columns/_internal,columns,database_name you can see there's no padlock icon on that page or on https://latest.datasette.io/_internal/columns - fixing this bug should fix that.

I need to do one last round of manual testing before I merge this.

This bug here: https://github.com/simonw/datasette/blob/85d5d2762c13d2b5a8bd9c5ec81c77fe6577121f/tests/test_permissions.py#L485

Turns out that was a bug I had introduced while working on that test, and it was the reason I was blocked on finishing work on: - #1829