home / github / issue_comments

Menu
  • Search all tables
  • GraphQL API

issue_comments: 1708777964

This data as json

html_url issue_url id node_id user created_at updated_at author_association body reactions issue performed_via_github_app
https://github.com/simonw/datasette/issues/2177#issuecomment-1708777964 https://api.github.com/repos/simonw/datasette/issues/2177 1708777964 IC_kwDOBm6k_c5l2eHs 9599 2023-09-06T17:04:30Z 2023-09-06T17:04:30Z OWNER

Here's the thinking: setting theDATASETTE_INTERNAL environment variable for your entire machine will cause any time you run datasette x.db to result in a process that shares the same internal.db database as other instances.

You might run more than one instance at once (I often have 4 or 5 going). This would currently break, because they would over-write each other's catalog tables:

https://github.com/simonw/datasette/blob/e4abae3fd7a828625d00c35c316852ffbaa5ef2f/datasette/utils/internal_db.py#L5-L62

The breaking wouldn't be obvious because the catalog tables aren't used by any features yet, but it's still bad.

This convinced us that actually we should move those catalog_ tables OUT of internal.db. The _internal database will be reserved for plugins that want to use it for caching, storing progress, etc.

I think we move them to an in-memory _catalog database which is excluded from ds.databases (like _internal is ) but can be accessed using datasette.get_catalog_database() - similar to datasette.get_internal_database().

So each instance of Datasette gets its own truly private _catalog, which is in-memory and so gets cleared at the end of each process.

An interesting thing that came up about a shared _internal database is that it provides opportunities for things like a notes plugin which allows you to attach notes to any row in any table in a database... where those notes become available to multiple Datasette instances that you might launch on the same laptop.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
1884408624  
Powered by Datasette · Queries took 0.789ms · About: github-to-sqlite