issue_comments

I'm going to split the remaining work into separate issues:

• [x] #1856
• [ ] #1855

Documentation: https://docs.datasette.io/en/1.0-dev/authentication.html#api-tokens

Built a prototype of the actor_from_request() hook for this and now:

% curl http://127.0.0.1:8001/-/actor.json -H 'Authorization: Bearer dstok_eyJhIjoicm9vdCIsImUiOm51bGx9.6O1OxgNTFkAU6uw7xNcmXYX949A' {"actor": {"id": "root", "dstok": true}}

Implemented that dstok_ prefix and the thing where only the actor["id"] is copied to the "a" field.

Prefix: dstok_ - for Datasette signed token.

The tokens also need something that can be used to differentiate them from alternative token mechanisms that other plugins might provide.

Maybe a prefix before the signed value.

Prefixes are also useful for scanning to check they were not accidentally committed to source control.

Other options:

• --setting default_api_tokens off
• --setting signed_api_tokens off
• --setting allow_create_token off

These feel inconsistent because they don't use the allow_ prefix - but they're also a bit less ugly to look at.

I like that last one.

Rather than duplicating the entire actor into the "a" field, maybe just copy the actor ID?

Would need to restrict token creation to just actors with an ID set.

On that basis, I think the model described above where tokens mainly work to provide an "act on behalf of this actor" - but with optional additional constraints - is a good one.

This is what we do for Sandstorm essentially and I fully agree it's the right way to do API tokens in multiuser systems.

Constraints will definitely be important though. I know I want a token to submit error reports programmatically, but I wouldn't want that token to convey my right to delete tables and records, Little Bobby Tables is out there somewhere, and he's all grown up now.

Tests failed because I added a view without also adding documentation!

I forgot that the deploy still goes out for branches other than main even if the tests aren't passing: https://github.com/simonw/datasette/blob/c7dd76c26257ded5bcdfd0570e12412531b8b88f/.github/workflows/deploy-latest.yml#L34-L38

Another thought about tokens that can act on behalf of the user.

Imagine a user has permission to access a table. They create a token that can create that table... but then their permission is revoked. It would be bad if they could still use that token they created earlier to access that table!

On that basis, I think the model described above where tokens mainly work to provide an "act on behalf of this actor" - but with optional additional constraints - is a good one.

Next step: make these tokens actually do something.

If I'm going to change the naming conventions for settings I should do it before Datasette 1.0.

If I'm going to have a setting to disable this feature I need to decide what it will be called.

Closest existing setting is this one, since it's for a feature that is turned on by default:

datasette mydatabase.db --setting allow_download off

So maybe this?

datasette mydatabase.db --setting allow_signed_api_tokens off

I like allow_signed_api_tokens more than allow_api_tokens because if you install a plugin such as https://datasette.io/plugins/datasette-auth-tokens then API tokens will work even though you disabled this default signed token feature.

allow_signed_api_tokens does feel a bit clumsy/verbose though.

Demo is now live at https://latest-1-0-dev.datasette.io/-/create-token - visit https://latest-1-0-dev.datasette.io/login-as-root first to sign in.

If you click "Create token" for "Token never expires" multiple times you currently get exactly the same token each time, since it's just a signed token containing a copy of your actor dictionary.

I'm not sure if I like that. I could give each token a random ID (maybe using secrets.token_hex()) such that different tokens have different identities, which would be useful for logging and auditing and maybe even revocation at some point in the future.

Demo:

A /-/debug-token page that can take a token and decode it to show you how long until it expires, what actor it represents and the permissions it has will be useful as well.

I'm going to build an initial /-/create-token interface which just bakes a token with the current actor in it and an optional expiry timestamp. I'll try the limited permissions thing later.

In which case the token would need to duplicate the current actor and then add extra constraints. So maybe the token design looks like this: json { "a": { "copy_of": "actor_creating_token"}, "p": { "t": "... the thing designed earlier, with those permissions in it" }, "e": "integer timestamp when token expires" }

... so maybe there's a way to create a token that inherits the exact permissions of the actor that created the token? That could even be a default mode for tokens, with an option to then further restrict permissions if desired.

Interesting challenge: what permissions should users be allowed to grant to tokens?

Clearly a user should not be able to create a token with a permission that the user themselves does not have.

And should there be a permission that allows people to create tokens? I think so.

Datasette currently defaults to having everything public-readable by default, unless a permission plugin changes that default.

In thinking more about this API mechanism, I realized that it might be good to have a mode where Datasette doesn't default to public everything. Maybe datasette --private to start it like that?

Might even be an opportunity to get rid of the current slightly confusing mechanism where permission checks can announce that they should default to true:

https://github.com/simonw/datasette/blob/c7dd76c26257ded5bcdfd0570e12412531b8b88f/datasette/views/database.py#L152-L154

I'm going to implement the first version of this token mechanism using permissions that exist already. Right now that's:

https://docs.datasette.io/en/0.62/authentication.html#built-in-permissions

Here are the shortcuts I'll use for them:

• view-instance - vi
• view-database - vd
• view-database-download - vdd
• view-table - vt
• view-query - vq
• execute-sql - es

Might be neat for API tokens to be signed with an additional secret than can be rotated independently of DATASETTE_SECRET itself, in order to invalidate all tokens without needing to invalidate logged in users too.

But again, I don't want to implement something like that until I see an actual need for it.

... also, maybe there should be a UI (perhaps on that page) for resetting the Datasette secret? Useful for emergency invalidation of all tokens.

No, I'm not going to build that unless someone asks for it. Restarting the server with a fresh secret should be easy enough.

If you start Datasette without providing a DATASETTE_SECRET environment variable of --secret option it creates a random signing secret that only lasts for the lifetime of the server.

This means any signed API tokens you create will stop working if the server restarts.

I think the /-/create-token UI should know when this happens and show a warning message about it, to avoid confusion.

Here's what that example looks like signed: python from datasette.app import Datasette ds = Datasette() ds.sign('{"t":{"a":["ir","ur","dr"],"d":{"fixtures":["ir","ur","dr"]},"t":{"fixtures":{"searchable":["ir"]}}}}') .eJxTqo5RKolRsgJSiUAqOkYpsyhGSSdGqRRCpQCpWBANUZOWWVFSWpRajFNprQ7cPCS1QF5xamJRckZiUk4qQm9sLRAoAQCC8yph.O0Gaej6-VOLbbtPq7xU6T77jEO0 That's 129 characters.

Note that Datasette doesn't have its own mechanism for signing things for a specific duration yet: https://docs.datasette.io/en/stable/internals.html#sign-value-namespace-default

So I'll need to add a "e": 1666739744 field with the UTC timestamp at which the token should expire.

Token design concept: json { "t": { "a": ["ir", "ur", "dr"], "d": { "fixtures": ["ir", "ur", "dr"] }, "t": { "fixtures": { "searchable": ["ir"] } } } } That JSON would be minified and signed.

Minified version of the above looks like this (101 characters):

{"t":{"a":["ir","ur","dr"],"d":{"fixtures":["ir","ur","dr"]},"t":{"fixtures":{"searchable":["ir"]}}}}

The "t" key shows this is a token that as a default API key.

"a" means "all" - these are permissions that have been granted on all tables and databases.

"d" means "databases" - this is a way to set permissions for all tables in a specific database.

"t" means "tables" - this lets you set permissions at a finely grained table level.

Then the permissions themselves are two character codes which are shortened versions - so:

• ir = insert-row
• ur = update-row
• dr = delete-row

Maybe these tokens can be restricted to specific databases and tables when they are first created?

Since they're signed tokens, I could bundle a bunch of extra stuff in them - this token is allowed to do these permissions against these tables/rows for example.

General wisdom seems to be that 8KB is a sensible maximum length for this kind of token, which is easily long enough to fit in a bunch of database / table / permissions.