{"id": 1174306154, "node_id": "I_kwDOBm6k_c5F_n1q", "number": 1668, "title": "Introduce concept of a database `route`, separate from its name", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 3268330, "label": "Datasette 1.0"}, "comments": 20, "created_at": "2022-03-19T16:48:28Z", "updated_at": "2022-03-20T16:43:16Z", "closed_at": "2022-03-20T16:43:16Z", "author_association": "OWNER", "pull_request": null, "body": "Some issues came up in the new `datasette-hashed-urls` plugin relating to the way it renames databases on startup to achieve unique URLs that depend on the database SHA-256 content:\r\n\r\n- https://github.com/simonw/datasette-hashed-urls/issues/10\r\n- https://github.com/simonw/datasette-hashed-urls/issues/9\r\n- https://github.com/simonw/datasette-hashed-urls/issues/8\r\n\r\nAll three of these could be addressed by making the \"path\" concept for a database (the `/foo` bit where it is served) work independently of the database's name, which would be used for default display and also as the alias when configuring cross-database aliases.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1668/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1485757511, "node_id": "I_kwDOBm6k_c5YjtxH", "number": 1939, "title": "register_permissions(datasette) plugin hook", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 8711695, "label": " Datasette 1.0a2"}, "comments": 20, "created_at": "2022-12-09T01:33:25Z", "updated_at": "2022-12-13T02:07:50Z", "closed_at": "2022-12-13T02:05:56Z", "author_association": "OWNER", "pull_request": null, "body": "A plugin hook that adds more named permissions to the list which is initially populated here:\r\n\r\nhttps://github.com/simonw/datasette/blob/e539c1c024bc62d88df91d9107cbe37e7f0fe55f/datasette/permissions.py#L1-L19\r\n\r\nOriginally imagined this hook in this comment:\r\n\r\n- https://github.com/simonw/datasette/issues/1881#issuecomment-1301639370\r\n\r\nI need this for a few reasons:\r\n\r\n- https://github.com/simonw/datasette/issues/1636\r\n  - Needs it in order to validate that permissions defined in `metadata.json` are set in the right place (don't set an instance permissions at table level for example)\r\n- https://github.com/simonw/datasette/issues/1855\r\n  - Needs it to be able to register additional abbreviations for use in signed cookies\r\n  - And for validation when you use `datasette create-token` and pass in extra permissions\r\n- The https://latest.datasette.io/-/permissions debug interface needs it to add extra debug options to the `<select>`", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1939/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1805076818, "node_id": "I_kwDOBm6k_c5rl0lS", "number": 2102, "title": "API tokens with view-table but not view-database/view-instance cannot access the table", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": {"value": 9599, "label": "simonw"}, "milestone": null, "comments": 20, "created_at": "2023-07-14T15:34:27Z", "updated_at": "2023-08-29T16:32:36Z", "closed_at": "2023-08-29T16:32:35Z", "author_association": "OWNER", "pull_request": null, "body": "> Spotted a problem while working on this: if you grant a token access to view table for a specific table but don't also grant view database and view instance permissions, that token is useless.\r\n>\r\n> This was a deliberate design decision in Datasette - it's documented on https://docs.datasette.io/en/1.0a2/authentication.html#access-permissions-in-metadata\r\n>\r\n>> If a user cannot access a specific database, they will not be able to access tables, views or queries within that database. If a user cannot access the instance they will not be able to access any of the databases, tables, views or queries.\r\n>\r\n> I'm now second-guessing if this was a good decision.\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/datasette-auth-tokens/issues/7#issuecomment-1636031702_\r\n            ", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/2102/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1250629388, "node_id": "I_kwDOCGYnMM5KixcM", "number": 440, "title": "CSV files with too many values in a row cause errors", "user": {"value": 4068, "label": "frafra"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 20, "created_at": "2022-05-27T10:54:44Z", "updated_at": "2022-06-14T22:23:01Z", "closed_at": "2022-06-14T20:12:46Z", "author_association": "NONE", "pull_request": null, "body": "*Original title: csv.DictReader can have None as key*\r\n\r\nIn some cases, `csv.DictReader` can have `None` as key for unnamed columns, and a list of values as value.\r\n`sqlite_utils.utils.rows_from_file` cannot handle that:\r\n\r\n```python\r\nurl=\"https://artsdatabanken.no/Fab2018/api/export/csv\"\r\ndb = sqlite_utils.Database(\":memory\")\r\n\r\nwith urlopen(url) as fab:\r\n    reader, _ = sqlite_utils.utils.rows_from_file(fab, encoding=\"utf-16le\")   \r\n    db[\"fab2018\"].insert_all(reader, pk=\"Id\")\r\n```\r\n\r\nResult:\r\n```\r\nTraceback (most recent call last):\r\n  File \"<stdin>\", line 3, in <module>\r\n  File \"/home/user/.local/pipx/venvs/sqlite-utils/lib/python3.8/site-packages/sqlite_utils/db.py\", line 2924, in insert_all\r\n    chunk = list(chunk)\r\n  File \"/home/user/.local/pipx/venvs/sqlite-utils/lib/python3.8/site-packages/sqlite_utils/db.py\", line 3454, in fix_square_braces\r\n    if any(\"[\" in key or \"]\" in key for key in record.keys()):\r\n  File \"/home/user/.local/pipx/venvs/sqlite-utils/lib/python3.8/site-packages/sqlite_utils/db.py\", line 3454, in <genexpr>\r\n    if any(\"[\" in key or \"]\" in key for key in record.keys()):\r\nTypeError: argument of type 'NoneType' is not iterable\r\n```\r\n\r\nCode:\r\nhttps://github.com/simonw/sqlite-utils/blob/59be60c471fd7a2c4be7f75e8911163e618ff5ca/sqlite_utils/db.py#L3454\r\n\r\n`sqlite-utils insert` from command line is not affected by this issue.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/440/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 628499086, "node_id": "MDU6SXNzdWU2Mjg0OTkwODY=", "number": 790, "title": "\"flash messages\" mechanism", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 5512395, "label": "Datasette 0.44"}, "comments": 20, "created_at": "2020-06-01T14:55:44Z", "updated_at": "2020-06-08T19:33:59Z", "closed_at": "2020-06-02T21:14:03Z", "author_association": "OWNER", "pull_request": null, "body": "> Passing `?_success` like this isn't necessarily the best approach. Potential improvements include:\r\n> \r\n> - Signing this message so it can't be tampered with (I could generate a signing secret on startup)\r\n> - Using a cookie with a temporary flash message in it instead\r\n> - Using HTML5 history API to remove the `?_success=` from the URL bar when the user lands on the page\r\n> \r\n> If I add an option to redirect the user to another page after success I may need a mechanism to show a flash message on that page as well, in which case I'll need a general flash message solution that works for any page.\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/datasette/pull/703_", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/790/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 679808124, "node_id": "MDU6SXNzdWU2Nzk4MDgxMjQ=", "number": 940, "title": "Move CI to GitHub Issues", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 5818042, "label": "Datasette 0.49"}, "comments": 20, "created_at": "2020-08-16T19:06:08Z", "updated_at": "2020-09-14T22:09:35Z", "closed_at": "2020-09-14T22:09:35Z", "author_association": "OWNER", "pull_request": null, "body": "It looks like the tests take 3m33s to run in GitHub Actions, but they're taking more than 8 minutes in Travis", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/940/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 733499930, "node_id": "MDU6SXNzdWU3MzM0OTk5MzA=", "number": 1072, "title": "load_template hook doesn't work for include/extends", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 6026070, "label": "0.51"}, "comments": 20, "created_at": "2020-10-30T20:33:44Z", "updated_at": "2020-10-31T20:48:18Z", "closed_at": "2020-10-30T22:50:57Z", "author_association": "OWNER", "pull_request": null, "body": "Includes like this one always go to disk, without hitting the `load_template` plugin hook:\r\n```html+jinja\r\n<footer class=\"ft\">{% block footer %}{% include \"_footer.html\" %}{% endblock %}</footer>\r\n```", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1072/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1651082214, "node_id": "PR_kwDOBm6k_c5NcFCf", "number": 2052, "title": "feat: Javascript Plugin API (Custom panels, column menu items with JS actions)", "user": {"value": 9020979, "label": "hydrosquall"}, "state": "closed", "locked": 0, "assignee": {"value": 9599, "label": "simonw"}, "milestone": null, "comments": 20, "created_at": "2023-04-02T20:23:44Z", "updated_at": "2023-10-14T17:49:03Z", "closed_at": "2023-10-13T00:00:27Z", "author_association": "CONTRIBUTOR", "pull_request": "simonw/datasette/pulls/2052", "body": "## Motivation\r\n\r\n- Allow plugins that add data visualizations [`datasette-vega`](https://github.com/simonw/datasette-vega), [`datasette-leaflet`](https://github.com/simonw/datasette-leaflet), and [`datasette-nteract-data-explorer`](https://github.com/hydrosquall/datasette-nteract-data-explorer) to co-exist safely\r\n- Standardize APIs / hooks to ease development for new JS plugin developers (better compat with datasette-lite) through standardized DOM selectors, methods for extending the existing Table UI. This has come up as a feature request several times (see research notes for examples)\r\n- Discussion w/ @simonw about a general-purpose Datasette JS API\r\n\r\n## Changes\r\n\r\nSummary: Provide 2 new surface areas for Datasette JS plugin developers. See alpha [documentation](https://github.com/simonw/datasette/pull/2052#issuecomment-1510423051)\r\n\r\n1. Custom column header items: <https://a.cl.ly/Kou97wJr>\r\n2. Basic \"panels\" controlled by buttons: <https://a.cl.ly/rRugWobd>\r\n\r\n### User Facing Changes\r\n\r\n- Allow creating menu items under table header that triggers JS (instead of opening hrefs per the existing [menu_link](https://docs.datasette.io/en/stable/plugin_hooks.html#menu-links-datasette-actor-request) hook). Items can respond to any column metadata provided by the column header (e.g. label). The proof of concept plugins log data to the console, or copy the column name to clipboard.\r\n- Allow plugins to register UI elements in a panel controller. The parent component handles switching the visibility of active plugins.\r\n  - Because native button elements are used, the panel is keyboard-accessible - use tab / shift-tab to cycle through tab options, and `enter` to select.\r\n  - There's room to improve the styling, but the focus of this PR is on the API rather than the UX.\r\n\r\n### (plugin) Developer Facing Changes\r\n\r\n- Dispatch a `datasette_init` [CustomEvent](https://developer.mozilla.org/en-US/docs/Web/API/CustomEvent/CustomEvent) when the `datasetteManager` is finished loading.\r\n- Provide `manager.registerPlugin` API for adding new functionality that coordinates with Datasette lifecycle events.\r\n- Provide a `manager.selectors` map of DOM elements that users may want to hook into.\r\n  - Updated `table.js` to use refer to these to motivating keeping things in sync\r\n- Allow plugins to register themselves with 2 hooks:\r\n  - `makeColumnActions`: Add items to menu in table column headers. Users can provide a `label`, and either `href` or `onClick` with full access to the metadata for the clicked column (name, type, misc. booleans)\r\n  - `makeAboveTablePanelConfigs`: Add items to the panel. Each panel has a unique ID (namespaced within that plugin), a render function, and a string label.\r\n\r\nSee [this file](https://github.com/simonw/datasette/blob/2d92b9328022d86505261bcdac419b6ed9cb2236/datasette/static/table-example-plugins.js) for example plugin usage.\r\n\r\n###  Core Developer Facing Changes\r\n\r\n- Modified `table.js` to make use of the `datasetteManager` API.\r\n- Added example plugins to the `demos/plugins` folder, and stored the test js in the `statics/` folder\r\n\r\n## Testing\r\n\r\nFor Datasette plugin developers, please see the [alpha-level documentation](https://github.com/simonw/datasette/pull/2052#issuecomment-1510423051) .\r\n\r\nTo run the examples:\r\n\r\n```bash\r\ndatasette serve fixtures.db  --plugins-dir=demos/plugins/\r\n```\r\n\r\nOpen local server: `http://127.0.0.1:8001/fixtures/facetable`\r\n\r\nOpen to all feedback on this PR, from API design to variable naming, to what additional hooks might be useful for the future.\r\n\r\nMy focus was more on the general shape of the API for developers, rather than on the UX of the test plugins.\r\n\r\n## Design notes\r\n\r\n- The manager tab panel could be a separate plugin if the implementation is too custom.\r\n- The `makeColumnHeaderItems` benefits from hooking into the logic of `table.js`\r\n- I wanted to offer this to the Datasette core, since the `datasette-manager` would be more powerful if it were connected to lifecycle and JS events that are part of the existing table.js.\r\n- Non-goals:\r\n  - Dependency management (for now) - there's no \"build\" step, we don't know when new plugins will be added. While there are some valid use cases (for example, allow multiple plugins to wait for a global leaflet object to be loaded), I don't see enough use-cases to justify doing this yet.\r\n  - Enabling single-page-app features - for now, most datasette actions lead to a new page being loaded. SPA development offers some benefits (no page jumping after clicking on a link), but also complexity that doesn't need to be in the core Datasette project.\r\n\r\n## Research Notes\r\n\r\n- Relocated to a [comment](https://github.com/simonw/datasette/pull/2052#issuecomment-1510423215), as this isn't required to review when evaluating the plugin. Including it just for those who are curious.\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/2052/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}