github
html_url | issue_url | id | node_id | user | created_at | updated_at | author_association | body | reactions | issue | performed_via_github_app |
---|---|---|---|---|---|---|---|---|---|---|---|
https://github.com/simonw/datasette/issues/1384#issuecomment-869075395 | https://api.github.com/repos/simonw/datasette/issues/1384 | 869075395 | MDEyOklzc3VlQ29tbWVudDg2OTA3NTM5NQ== | 9599 | 2021-06-26T23:54:21Z | 2021-06-26T23:59:21Z | OWNER | (It may well be that implementing #1168 involves a switch to async metadata) | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
930807135 | |
https://github.com/simonw/datasette/issues/1384#issuecomment-869075368 | https://api.github.com/repos/simonw/datasette/issues/1384 | 869075368 | MDEyOklzc3VlQ29tbWVudDg2OTA3NTM2OA== | 9599 | 2021-06-26T23:53:55Z | 2021-06-26T23:53:55Z | OWNER | Great, let's drop fallback then. My instinct at the moment is to ship this plugin hook as-is but with a warning that it may change before Datasette 1.0 - then before 1.0 either figure out an async variant or finish the database-backed metadata concept from #1168 and recommend that as an alternative. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
930807135 | |
https://github.com/simonw/datasette/issues/1384#issuecomment-869071790 | https://api.github.com/repos/simonw/datasette/issues/1384 | 869071790 | MDEyOklzc3VlQ29tbWVudDg2OTA3MTc5MA== | 9599 | 2021-06-26T23:04:12Z | 2021-06-26T23:04:12Z | OWNER | > Hmmm... that's tricky, since one of the most obvious ways to use this hook is to load metadata from database tables using SQL queries. > > @brandonrobertz do you have a working example of using this hook to populate metadata from database tables I can try? Answering my own question: here's how Brandon implements it in his `datasette-live-config` plugin: https://github.com/next-LI/datasette-live-config/blob/72e335e887f1c69c54c6c2441e07148955b0fc9f/datasette_live_config/__init__.py#L50-L160 That's using a completely separate SQLite connection (actually wrapped in `sqlite-utils`) and making blocking synchronous calls to it. This is a pragmatic solution, which works - and likely performs just fine, because SQL queries like this against a small database are so fast that not running them asynchronously isn't actually a problem. But... it's weird. Everywhere else in Datasette land uses `await db.execute(...)` - but here's an example where users are encouraged to use blocking calls instead. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
930807135 | |
https://github.com/simonw/datasette/issues/1384#issuecomment-869071435 | https://api.github.com/repos/simonw/datasette/issues/1384 | 869071435 | MDEyOklzc3VlQ29tbWVudDg2OTA3MTQzNQ== | 9599 | 2021-06-26T22:59:26Z | 2021-06-26T22:59:26Z | OWNER | The other alternative is to finish the work to build a `_metadata` internal table, see #1168. The idea there was that if we want to support efficient pagination and search across the metadata for thousands of attached tables powering it with a plugin hook doesn't work well - we don't want to call the hook once for every one of 1,000+ tables just to implement the homepage. So instead, all metadata for all attached databases would be loaded into an in-memory database called `_metadata`. Plugins that want to modify stored metadata could then do so by directly writing to that table. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
930807135 | |
https://github.com/simonw/datasette/issues/1384#issuecomment-869071167 | https://api.github.com/repos/simonw/datasette/issues/1384 | 869071167 | MDEyOklzc3VlQ29tbWVudDg2OTA3MTE2Nw== | 9599 | 2021-06-26T22:55:36Z | 2021-06-26T22:55:36Z | OWNER | Just realized I already have an issue open for this, at #860. I'm going to close that and continue work on this in this issue. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
930807135 | |
https://github.com/simonw/datasette/issues/1384#issuecomment-869070941 | https://api.github.com/repos/simonw/datasette/issues/1384 | 869070941 | MDEyOklzc3VlQ29tbWVudDg2OTA3MDk0MQ== | 9599 | 2021-06-26T22:53:34Z | 2021-06-26T22:53:34Z | OWNER | The `await` thing is worrying me a lot - it feels like this plugin hook is massively less useful if it can't make it's own DB queries and generally do asynchronous stuff - but I'd also like not to break every existing plugin that calls `datasette.metadata(...)`. One solution that could work: introduce a new method, maybe `await datasette.get_metadata(...)`, which uses this plugin hook - and keep the existing `datasette.metadata()` method (which doesn't call the hook) around. This would ensure existing plugins keep on working. Then, upgrade those plugins separately - with the goal of deprecating and removing `.metadata()` entirely in Datasette 1.0 - having upgraded the plugins in the meantime. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
930807135 | |
https://github.com/simonw/datasette/issues/1384#issuecomment-869070348 | https://api.github.com/repos/simonw/datasette/issues/1384 | 869070348 | MDEyOklzc3VlQ29tbWVudDg2OTA3MDM0OA== | 9599 | 2021-06-26T22:46:18Z | 2021-06-26T22:46:18Z | OWNER | Here's where the plugin hook is called, demonstrating the `fallback=` argument: https://github.com/simonw/datasette/blob/05a312caf3debb51aa1069939923a49e21cd2bd1/datasette/app.py#L426-L472 I'm not convinced of the use-case for passing `fallback=` to the hook here - is there a reason a plugin might care whether fallback is `True` or `False`, seeing as the `metadata()` method already respects that fallback logic on line 459? | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
930807135 | |
https://github.com/simonw/datasette/issues/1384#issuecomment-869070076 | https://api.github.com/repos/simonw/datasette/issues/1384 | 869070076 | MDEyOklzc3VlQ29tbWVudDg2OTA3MDA3Ng== | 9599 | 2021-06-26T22:42:21Z | 2021-06-26T22:42:21Z | OWNER | Hmmm... that's tricky, since one of the most obvious ways to use this hook is to load metadata from database tables using SQL queries. @brandonrobertz do you have a working example of using this hook to populate metadata from database tables I can try? | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
930807135 | |
https://github.com/simonw/datasette/issues/1384#issuecomment-869069926 | https://api.github.com/repos/simonw/datasette/issues/1384 | 869069926 | MDEyOklzc3VlQ29tbWVudDg2OTA2OTkyNg== | 9599 | 2021-06-26T22:40:15Z | 2021-06-26T22:40:53Z | OWNER | The documentation says: > **datasette**: You can use this to access plugin configuration options via `datasette.plugin_config(your_plugin_name)`, or to execute SQL queries. That's not accurate: since the plugin hook is a regular function, not an awaitable, you can't use it to run `await db.execute(...)` so you can't execute SQL queries. I can fix this with the await-me-maybe pattern, used for other plugin hooks: https://simonwillison.net/2020/Sep/2/await-me-maybe/ BUT... that requires changing the `ds.metadata()` function to be awaitable, which will affect every existing plugn that uses that documented internal method! | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
930807135 | |
https://github.com/simonw/datasette/issues/1384#issuecomment-869069768 | https://api.github.com/repos/simonw/datasette/issues/1384 | 869069768 | MDEyOklzc3VlQ29tbWVudDg2OTA2OTc2OA== | 9599 | 2021-06-26T22:37:53Z | 2021-06-26T22:37:53Z | OWNER | The documentation doesn't describe the ``fallback`` argument at the moment. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
930807135 | |
https://github.com/simonw/datasette/issues/1384#issuecomment-869069655 | https://api.github.com/repos/simonw/datasette/issues/1384 | 869069655 | MDEyOklzc3VlQ29tbWVudDg2OTA2OTY1NQ== | 9599 | 2021-06-26T22:36:14Z | 2021-06-26T22:37:37Z | OWNER | Documentation for the new hook is now live at https://docs.datasette.io/en/latest/plugin_hooks.html#get-metadata-datasette-key-database-table-fallback Link to the current snapshot of that documentation: https://github.com/simonw/datasette/blob/05a312caf3debb51aa1069939923a49e21cd2bd1/docs/plugin_hooks.rst#get-metadata-datasette-key-database-table-fallback | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
930807135 |