issue_comments

Here's what I have today - it's an undocumented datasette.metadata() method that returns a full JSON dictionary of values OR a single value if the optional key= argument is provided: https://github.com/simonw/datasette/blob/f2e2bfcdd9ad4891f3f66c9104c09943d943ffe4/datasette/app.py#L357-L388

In #942 I want to add support for per-column metadata - which means this new lookup mechanism will need to be able to answer the question "what description is available for this column".

So what should the .metadata() method look like? A couple of options:

• datasette.metadata("description", table=x, database=y) - can take optional column= too.
• datasette.table_metadata("description", table=x, database=y) and datasette.database_metadata("description", database=y) and so on - multiple methods for the different types of metadata.

I'll also allow any key to be looked up - so if users want to invent their own metadata keys other than the default license_url etc they can do so.

I'm going to go with a plugin hook (and Datasette method) that returns individual values - so you ask it for e.g. the license_url for a specific table and it returns a string or None.

The default plugin hook implementation that ships with Datasette will then implement cascading lookups against metadata.json - but other plugins will be able to provide their own implementations, which should make it easy to build a plugin that lets you keep metadata in a database file and edit it interactively.

Documentation: https://docs.datasette.io/en/latest/internals.html#setting-key

I'm going to make this a documented method in https://docs.datasette.io/en/latest/internals.html#datasette-class

This rebranding is complete - #1107 is a follow-up internal refactor.

I used an "exact redirect" instead and it worked:

https://docs.datasette.io/en/latest/config.html isn't redirecting though, even after I tried running a rebuild of the latest version.

https://docs.datasette.io/en/latest/settings.html is now live - need to redirect https://docs.datasette.io/en/latest/config.html to it using the ReadTheDocs redirects interface.

Most of these use plugin_config which is unaffected. It looks like the only code I need to worry about is this trick in datasette-graphl: https://github.com/simonw/datasette-graphql/blob/483c9a9e203bb90365def3df8b8f01dda1e75865/datasette_graphql/utils.py#L456-L460

python class DatasetteSpecialConfig(wrapt.ObjectProxy): def config(self, key): if key == "suggest_facets": return False return self.__wrapped__.config(key)

I ran rg '.config\(' datasette-*/ in my top-level directory: datasette-sentry/test_datasette_sentry.py: def plugin_config(self, name): datasette-sentry/datasette_sentry.py: config = datasette.plugin_config("datasette-sentry") or {} datasette-render-markdown/datasette_render_markdown/__init__.py: datasette.plugin_config( datasette-render-images/datasette_render_images.py: plugin_config = datasette.plugin_config("datasette-render-images") or {} datasette-render-html/datasette_render_html.py: config = datasette.plugin_config( datasette-render-timestamps/datasette_render_timestamps/__init__.py: datasette.plugin_config( datasette-permissions-sql/datasette_permissions_sql/__init__.py: for rule in datasette.plugin_config("datasette-permissions-sql") or []: datasette-mask-columns/datasette_mask_columns/__init__.py: datasette.plugin_config("datasette-mask-columns", database=database) or {} datasette-mask-columns/datasette_mask_columns/__init__.py: masks = datasette.plugin_config("datasette-mask-columns", database=database) or {} datasette-media/datasette_media/__init__.py: plugin_config = datasette.plugin_config("datasette-media") or {} datasette-insert/datasette_insert/__init__.py: plugin_config = datasette.plugin_config("datasette-insert") or {} datasette-indieauth/datasette_indieauth/__init__.py: plugin_config = datasette.plugin_config("datasette-indieauth") or {} datasette-graphql/datasette_graphql/__init__.py: config = datasette.plugin_config("datasette-graphql") or {} datasette-graphql/datasette_graphql/__init__.py: config = datasette.plugin_config("datasette-graphql") or {} datasette-graphql/datasette_graphql/utils.py: auto_camelcase=(datasette.plugin_config("datasette-graphql") or {}).get( datasette-graphql/datasette_graphql/utils.py: table_plugin_config = datasette.plugin_config( datasette-graphql/datasette_graphql/utils.py: def config(self, key): datasette-graphql/datasette_graphql/utils.py: return self.__wrapped__.config(key) datasette-init/datasette_init/__init__.py: config = datasette.plugin_config("datasette-init") datasette-edit-templates/datasette_edit_templates/__init__.py: plugin_config = datasette.plugin_config("datasette-edit-templates") or {} datasette-cors/datasette_cors.py: config = datasette.plugin_config("datasette-cors") or {} datasette-cluster-map-old/build/lib/datasette_cluster_map/__init__.py: datasette.plugin_config("datasette-cluster-map", database=database, table=table) datasette-cluster-map/datasette_cluster_map/__init__.py: datasette.plugin_config("datasette-cluster-map", database=database, table=table) datasette-cluster-map/datasette_cluster_map/__init__.py: datasette.plugin_config("datasette-cluster-map", database=database, table=table) datasette-cluster-map/tests/test_cluster_map.py:async def test_plugin_config(db_path, config, table, expected_fragments): datasette-configure-asgi/datasette_configure_asgi.py: configs = datasette.plugin_config("datasette-configure-asgi") or [] datasette-configure-asgi/test_datasette_configure_asgi.py: def plugin_config(self, name): datasette-cluster-map-old/datasette_cluster_map/__init__.py: datasette.plugin_config("datasette-cluster-map", database=database, table=table) datasette-auth-passwords/datasette_auth_passwords/__init__.py: config = datasette.plugin_config("datasette-auth-passwords") or {} datasette-auth-github/datasette_auth_github/views.py:def verify_config(config): datasette-auth-github/datasette_auth_github/views.py: config = datasette.plugin_config("datasette-auth-github") datasette-auth-github/datasette_auth_github/views.py: verify_config(config) datasette-auth-github/datasette_auth_github/views.py: config = datasette.plugin_config("datasette-auth-github") datasette-auth-github/datasette_auth_github/views.py: verify_config(config) datasette-atom/datasette_atom/__init__.py: plugin_config = datasette.plugin_config("datasette-atom") datasette-auth-google/datasette_auth_google/__init__.py: config = datasette.plugin_config("datasette-auth-github") or {} datasette-auth-existing-cookies/test_datasette_auth_existing_cookies.py: def plugin_config(self, name): datasette-auth-passwords/build/lib/datasette_auth_passwords/__init__.py: config = datasette.plugin_config("datasette-auth-passwords") or {} datasette-annotate/datasette_annotate/utils.py: plugin_config = datasette.plugin_config("datasette-annotate") or {} datasette-auth-existing-cookies/datasette_auth_existing_cookies/__init__.py: config = datasette.plugin_config("datasette-auth-existing-cookies") or {} datasette-auth-simple/datasette_auth_simple/__init__.py:datasette.plugin_config("datasette-auth-simple") datasette-auth-tokens/datasette_auth_tokens/__init__.py: config = datasette.plugin_config("datasette-auth-tokens") or {} datasette-block-robots/datasette_block_robots/__init__.py: config = datasette.plugin_config("datasette-block-robots") or {} datasette-block-robots/datasette_block_robots/__init__.py: config = datasette.plugin_config("datasette-block-robots") or {}

Are there any plugins that use this API even though it isn't documented?

https://readthedocs.org/dashboard/datasette/redirects/

I'll throw an error if a config.json file is detected.

This is well handled now: https://github.com/simonw/datasette/tree/0.51.1/tests/plugins

I'd like to redirect https://docs.datasette.io/en/stable/config.html to a new https://docs.datasette.io/en/stable/settings.html page too. I can use https://docs.readthedocs.io/en/stable/user-defined-redirects.html for that.

Part of #1105

Part of #1105

Part of #1105

I'll set up a redirect from /-/config to /-/settings.

Need to figure out the --setting foo bar alternative for this --config foo:bar logic: https://github.com/simonw/datasette/blob/4bac9f18f9d04e5ed10f072502bcc508e365438e/datasette/cli.py#L31-L63

I'm going to keep --config for the moment but show a deprecation warning that it will be gone in Datasette 1.0.

This is blocking progress on other metadata tickets like #860 because I want to split the concept of concrete metadata (source, license, etc) from configuration that currently lives in metadata (default sort order, default facets).

I'm going to solve this next to unblock that stuff.

I see two ways this plugin hook could work. It could be asked about a specific instance, database or table and return the full metadata for that object. OR it could ask for a specific metadata field - e.g. source_url for table X, and return that.

The more finely grained one would allow plugins to implement their own cascading rules pretty easily. Is there a reason it would be better for the hook to return an entire block of JSON for a specific table or database?

I also need to decide if this hook is just going to be about source/license/about displayed metadata, or if it will include the functionality that has been sneaking into metadata.json over time - stuff like page size, default sort order or default facets.

Perhaps I should split those out into a "configuration" concept first, after renaming --config to --setting in #992.

The trick I'm using here is to follow the next_url in order to paginate through all of the matching results. The loop calls the data() method multiple times, once for each page of results: https://github.com/simonw/datasette/blob/4bac9f18f9d04e5ed10f072502bcc508e365438e/datasette/views/base.py#L304-L307

Current design: https://docs.datasette.io/en/stable/plugin_hooks.html#register-output-renderer-datasette

python @hookimpl def register_output_renderer(datasette): return { "extension": "test", "render": render_demo, "can_render": can_render_demo, # Optional } Where render_demo looks something like this: python async def render_demo(datasette, columns, rows): db = datasette.get_database() result = await db.execute("select sqlite_version()") first_row = " | ".join(columns) lines = [first_row] lines.append("=" * len(first_row)) for row in rows: lines.append(" | ".join(row)) return Response( "\n".join(lines), content_type="text/plain; charset=utf-8", headers={"x-sqlite-version": result.first()[0]} ) Meanwhile here's where the CSV streaming mode is implemented: https://github.com/simonw/datasette/blob/4bac9f18f9d04e5ed10f072502bcc508e365438e/datasette/views/base.py#L297-L380

I'd like to implement this by first extending the register_output_renderer() hook to support streaming huge responses, then switching CSV to use the plugin hook in addition to TSV using it.