![\"fixtures__facetable__15_rows\"](\"https://user-images.githubusercontent.com/9599/113523748-a7d2b300-955e-11eb-904d-8cf2639b0b5f.png\")
\r\n\r\nTo https://latest.datasette.io/fixtures?sql=select+pk%2C+created%2C+planet_int%2C+on_earth%2C+state%2C+city_id%2C+neighborhood%2C+tags%2C+complex_array%2C+distinct_some_null+from+facetable+order+by+pk+limit+101\r\n\r\n![\"fixtures__select_pk__created__planet_int__on_earth__state__city_id__neighborhood__tags__complex_array__distinct_some_null_from_facetable_order_by_pk_limit_101\"](\"https://user-images.githubusercontent.com/9599/113523761-be790a00-955e-11eb-8c82-36b9d0226bc8.png\")
\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1293/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "reopened"}
{"id": 1551694938, "node_id": "PR_kwDOBm6k_c5IQeKz", "number": 1999, "title": "?_extra= support (draft)", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 49, "created_at": "2023-01-21T04:55:18Z", "updated_at": "2023-03-22T22:49:41Z", "closed_at": "2023-03-22T22:49:40Z", "author_association": "OWNER", "pull_request": "simonw/datasette/pulls/1999", "body": "Refs:\r\n- #262\r\n\r\n\r\n----\n:books: Documentation preview :books:: https://datasette--1999.org.readthedocs.build/en/1999/\n\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1999/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 973139047, "node_id": "MDU6SXNzdWU5NzMxMzkwNDc=", "number": 1439, "title": "Rethink how .ext formats (v.s. ?_format=) works before 1.0", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 3268330, "label": "Datasette 1.0"}, "comments": 48, "created_at": "2021-08-17T23:32:51Z", "updated_at": "2022-03-15T20:51:26Z", "closed_at": "2022-03-15T20:51:26Z", "author_association": "OWNER", "pull_request": null, "body": "Datasette currently has surprising special behaviour for if a table name ends in `.csv` - which can happen when a tool like `csvs-to-sqlite` creates tables that match the filename that they were imported from.\r\n\r\nhttps://latest.datasette.io/fixtures/table%2Fwith%2Fslashes.csv illustrates this behaviour: it links to `.csv` and `.json` that look like this:\r\n\r\n- https://latest.datasette.io/fixtures/table%2Fwith%2Fslashes.csv?_format=json\r\n- https://latest.datasette.io/fixtures/table%2Fwith%2Fslashes.csv?_format=csv&_size=max\r\n\r\nWhere normally Datasette would add the `.csv` or `.json` extension to the path component of the URL (as seen on other pages such as https://latest.datasette.io/fixtures/facet_cities) here the [path_with_format() function](https://github.com/simonw/datasette/blob/adb5b70de5cec3c3dd37184defe606a082c232cf/datasette/utils/__init__.py#L710) notices that there is already a `.` in the path and instead adds `?_format=csv` to the query string instead.\r\n\r\nThe problem with this mechanism is that it's pretty surprising. Anyone writing external code to Datasette who wants to get back the `.csv` or `.json` version giving the URL to a table page will need to know about and implement this behaviour themselves. That's likely to cause all kinds of bugs in the future.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1439/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1450363982, "node_id": "PR_kwDOBm6k_c5C9ZuP", "number": 1893, "title": "Upgrade to CodeMirror 6, add SQL autocomplete", "user": {"value": 95570, "label": "bgrins"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 48, "created_at": "2022-11-15T20:52:35Z", "updated_at": "2022-11-16T23:54:02Z", "closed_at": "2022-11-16T23:49:06Z", "author_association": "CONTRIBUTOR", "pull_request": "simonw/datasette/pulls/1893", "body": "In an effort to get closer to table / column autocomplete I took a shot at https://github.com/simonw/datasette/issues/1796. I haven't done a lot of testing but would be curious if this fixes some of the concerns raised in https://github.com/simonw/datasette/issues/1796#issue-1355148385 for example.\r\n\r\nDone:\r\n* Changed to bundling using rollup as per https://codemirror.net/examples/bundle/\r\n* Restored a fromTextArea-like function from https://codemirror.net/docs/migration/\r\n* Removed old JS and CSS files (no external CSS needed anymore as per https://codemirror.net/examples/styling/)\r\n* Updated instructions for building the bundle\r\n\r\nNot done:\r\n* cmResize had an error, so commented out the resize handle\r\n* Add extraKeys option for shift+enter and tab\r\n\r\n\r\n----\n:books: Documentation preview :books:: https://datasette--1893.org.readthedocs.build/en/1893/\n\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1893/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 712260429, "node_id": "MDU6SXNzdWU3MTIyNjA0Mjk=", "number": 983, "title": "JavaScript plugin hooks mechanism similar to pluggy", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 47, "created_at": "2020-09-30T20:32:43Z", "updated_at": "2021-01-25T04:43:58Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "> It would be neat to provide a JavaScript plugin hook that plugins can use to add their own options to this menu. No idea what that would look like though.\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/datasette/issues/981#issuecomment-701616922_", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/983/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 824064069, "node_id": "MDU6SXNzdWU4MjQwNjQwNjk=", "number": 1249, "title": "Updated Dockerfile with SpatiaLite version 5.0", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 45, "created_at": "2021-03-08T00:17:36Z", "updated_at": "2022-01-20T21:29:43Z", "closed_at": "2021-03-29T00:57:13Z", "author_association": "OWNER", "pull_request": null, "body": "The version bundled in Datasette's Docker image right now is 4.4.0-RC0\r\n\r\nhttps://github.com/simonw/datasette/blob/d0fd833b8cdd97e1b91d0f97a69b494895d82bee/Dockerfile#L16-L17\r\n\r\n5 has been out for a couple of months and has a bunch of big improvements, most notable stable KNN support.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1249/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1058072543, "node_id": "I_kwDOBm6k_c4_EOff", "number": 1518, "title": "Complete refactor of TableView and table.html template", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": {"value": 3268330, "label": "Datasette 1.0"}, "comments": 45, "created_at": "2021-11-19T02:55:16Z", "updated_at": "2022-03-15T18:35:49Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "Split from #878. The current `TableView` class is by far the most complex part of Datasette, and the most difficult to work on: https://github.com/simonw/datasette/blob/0.59.2/datasette/views/table.py\r\n\r\nIn #878 I started exploring a new pattern for building views. In doing so it became clear that `TableView` is the first beast that I need to slay - if I can refactor that into something neat the pattern for building other views will emerge as a natural consequence.\r\n\r\nI've been trying to build this as a `register_routes()` plugin, as originally suggested in #870 - though unfortunately it looks like those plugins can't replace existing Datasette default views at the moment, see #1517. [UPDATE: I was wrong about this, plugins can over-ride default views just fine]\r\n\r\nI also know that I want to have a fully documented template context for `table.html` as a major step on the way to Datasette 1.0, see #1510.\r\n\r\nAll of this adds up to the `TableView` factor being a major project that will unblock a whole flurry of other things - so I'm going to work on that in this separate issue.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1518/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 324188953, "node_id": "MDU6SXNzdWUzMjQxODg5NTM=", "number": 272, "title": "Port Datasette to ASGI", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": {"value": 9599, "label": "simonw"}, "milestone": {"value": 3268330, "label": "Datasette 1.0"}, "comments": 42, "created_at": "2018-05-17T21:16:32Z", "updated_at": "2019-06-24T04:54:15Z", "closed_at": "2019-06-24T03:33:06Z", "author_association": "OWNER", "pull_request": null, "body": "Datasette doesn't take much advantage of Sanic, and I'm increasingly having to work around parts of it because of idiosyncrasies that are specific to Datasette - caring about the exact order of querystring arguments for example.\r\n\r\nSince Datasette is GET-only our needs from a web framework are actually pretty slim.\r\n\r\nThis becomes more important as I expand the plugins #14 framework. Am I sure I want the plugin ecosystem to depend on a Sanic if I might move away from it in the future?\r\n\r\nIf Datasette wasn't all about async/await I would use WSGI, but today it makes more sense to use ASGI. I'd like to be confident that switching to ASGI would still give me the excellent performance that Sanic provides.\r\n\r\nhttps://github.com/django/asgiref/blob/master/specs/asgi.rst", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/272/reactions\", \"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 582526961, "node_id": "MDU6SXNzdWU1ODI1MjY5NjE=", "number": 699, "title": "Authentication (and permissions) as a core concept", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 5512395, "label": "Datasette 0.44"}, "comments": 40, "created_at": "2020-03-16T18:48:00Z", "updated_at": "2020-06-06T19:42:11Z", "closed_at": "2020-06-06T19:42:11Z", "author_association": "OWNER", "pull_request": null, "body": "Right now Datasette authentication is provided exclusively by plugins:\r\n\r\n* https://github.com/simonw/datasette-auth-github\r\n* https://github.com/simonw/datasette-auth-existing-cookies\r\n\r\nThis is an all-or-nothing approach: either your Datasette instance requires authentication at the top level or it does not.\r\n\r\nBut... as I build new plugins like https://github.com/simonw/datasette-configure-fts and https://github.com/simonw/datasette-edit-tables I increasingly have individual features which should be reserved for logged-in users while still wanting other parts of Datasette to be open to all.\r\n\r\nThis is too much for plugins to own independently of Datasette core. Datasette needs to ship a single \"user is authenticated\" concept (independent of how users actually sign in) so that different plugins can integrate with it.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/699/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1496652622, "node_id": "I_kwDOBm6k_c5ZNRtO", "number": 1955, "title": "invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things", "user": {"value": 32839123, "label": "Rik-de-Kort"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 36, "created_at": "2022-12-14T13:39:56Z", "updated_at": "2022-12-19T04:34:16Z", "closed_at": "2022-12-18T02:45:18Z", "author_association": "NONE", "pull_request": null, "body": "In the past (pre-september 14, #1809) I had a running deployment of Datasette on Azure WebApps by emulating the call in cli.py to Gunicorn: `gunicorn -w 2 -k uvicorn.workers.UvicornWorker app:app`.\r\n\r\nMy most recent deployment, however, fails loudly by shouting that `Datasette.invoke_startup()` was not called. It does not seem to be possible to call `invoke_startup` when running using a uvicorn command directly like this (I've reproduced this locally using `uvicorn`). Two candidates that I have tried:\r\n* Uvicorn has a `--factory` option, but the app factory has to be synchronous, so no `await invoke_startup` there\r\n* `asyncio.get_event_loop().run_until_complete` is also not an option because `uvicorn` already has the event loop running.\r\n\r\nOne additional option is:\r\n* Use Gunicorn's [server hooks](https://docs.gunicorn.org/en/stable/settings.html#server-hooks) to call `invoke_startup`. These are also synchronous, but I might be able to get ahead of the event loop starting here.\r\n\r\nIn my current deployment setup, it does not appear to be possible to use `datasette serve` directly, so I'm stuck either\r\n* Trying to rework my complete deployment setup, for instance, using Azure functions as described [here](https://github.com/simonw/azure-functions-datasette))\r\n* Or dig into the ASGI spec and write a wrapper for the sole purpose of launching Datasette using a direct Uvicorn invocation.\r\n\r\nQuestions for the maintainers:\r\n* Is this intended behaviour/will not support/etc.? If so, I'd be happy to add a PR with a couple lines in the documentation.\r\n* if this is not intended behaviour, what is a good way to fix it? I could have a go at the ASGI spec thing (I think the Azure Functions thing is related) and provide a PR with the wrapper here, but I'm all ears!\r\n\r\nAlmost forgot, minimal reproducer:\r\n```python\r\nfrom datasette import Datasette\r\n\r\nds = Datasette(files=['./global-power-plants.db'])]\r\napp = ds.app()\r\n```\r\n\r\nSave as app.py in the same folder as global-power-plants.db, and then try running\r\n`uvicorn app:app`.\r\n\r\nOpening the resulting Datasette instance in the browser will show the error message.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1955/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1058896236, "node_id": "I_kwDOBm6k_c4_HXls", "number": 1522, "title": "Deploy a live instance of demos/apache-proxy", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 34, "created_at": "2021-11-19T20:32:55Z", "updated_at": "2021-11-23T03:00:34Z", "closed_at": "2021-11-20T18:51:56Z", "author_association": "OWNER", "pull_request": null, "body": "> I'll get this working on my laptop first, but then I want to get it up and running on Cloud Run - maybe with a GitHub Actions workflow in this repo that re-deploys it on manual execution.\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1521#issuecomment-974322178_\r\n\r\nI started by following https://ahmet.im/blog/cloud-run-multiple-processes-easy-way/ - see example in https://github.com/ahmetb/multi-process-container-lazy-solution", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1522/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 681375466, "node_id": "MDU6SXNzdWU2ODEzNzU0NjY=", "number": 943, "title": "await datasette.client.get(path) mechanism for executing internal requests", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 5971510, "label": "Datasette 0.50"}, "comments": 33, "created_at": "2020-08-18T22:17:42Z", "updated_at": "2020-10-09T17:22:55Z", "closed_at": "2020-10-09T16:11:26Z", "author_association": "OWNER", "pull_request": null, "body": "`datasette-graphql` works by making internal requests to the TableView class (in order to take advantage of existing pagination logic, plus options like `?_search=` and `?_where=`) - see #915 \r\n\r\nI want to support a `mod_rewrite` style mechanism for putting nicer URLs on top of Datasette pages - I botched that together for a project here using an internal ASGI proxying trick: https://github.com/natbat/tidepools_near_me/commit/ec102c6da5a5d86f17628740d90b6365b671b5e1\r\n\r\nIf the `datasette` object provided a documented method for executing internal requests (in a way that makes sense with logging etc - i.e. doesn't get logged as a separate request) both of these use-cases would be much neater.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/943/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 770436876, "node_id": "MDU6SXNzdWU3NzA0MzY4NzY=", "number": 1150, "title": "Maintain an in-memory SQLite table of connected databases and their tables", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 3268330, "label": "Datasette 1.0"}, "comments": 32, "created_at": "2020-12-17T23:02:13Z", "updated_at": "2020-12-27T14:51:39Z", "closed_at": "2020-12-18T22:34:12Z", "author_association": "OWNER", "pull_request": null, "body": "I want Datasette to have its own internal metadata about connected tables, to power features like a paginated searchable homepage in #461. I want this to be a SQLite table.\r\n\r\nThis could also be part of the directory scanning mechanism prototyped in #672 - where Datasette can be set to continually scan a directory for new database files that it can serve.\r\n\r\nAlso relevant to the Datasette Library concept in #417.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1150/reactions\", \"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1217759117, "node_id": "I_kwDOBm6k_c5IlYeN", "number": 1727, "title": "Research: demonstrate if parallel SQL queries are worthwhile", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 32, "created_at": "2022-04-27T18:54:21Z", "updated_at": "2022-09-26T14:48:31Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "I added parallel SQL query execution here:\r\n- https://github.com/simonw/datasette/issues/1723\r\n\r\nMy hunch is that this will take advantage of multiple cores, since Python's `sqlite3` module releases the GIL once a query is passed to SQLite.\r\n\r\nI'd really like to prove this is the case though. Just not sure how to do it!\r\n\r\nLarger question: is this performance optimization actually improving performance at all? Under what circumstances is it worthwhile?", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1727/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 309471814, "node_id": "MDU6SXNzdWUzMDk0NzE4MTQ=", "number": 189, "title": "Ability to sort (and paginate) by column", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": {"value": 9599, "label": "simonw"}, "milestone": null, "comments": 31, "created_at": "2018-03-28T18:04:51Z", "updated_at": "2018-04-15T18:54:22Z", "closed_at": "2018-04-09T05:16:02Z", "author_association": "OWNER", "pull_request": null, "body": "As requested in https://github.com/simonw/datasette/issues/185#issuecomment-376614973\r\n\r\nI've previously avoided this for performance reasons: sort-by-column on a column without an index is likely to perform badly for hundreds of thousands of rows.\r\n\r\nThat's not a good enough reason to avoid the feature entirely though. A few options:\r\n\r\n* Allow sort-by-column by default, give users the option to disable it for specific tables/columns\r\n* Disallow sort-by-column by default, give users option (probably in `metadata.json`) to enable it for specific tables/columns\r\n* Automatically detect if a column either has an index on it OR a table has less than X rows in it\r\n\r\nWe already have the mechanism in place to cut off SQL queries that take more than X seconds, so if someone DOES try to sort by a column that's too expensive it won't actually hurt anything - but it would be nice to not show people a \"sort\" option which is guaranteed to throw a timeout error.\r\n\r\nThe vast majority of datasette usage that I've seen so far is on smaller datasets where the performance penalties of sort-by-column are extremely unlikely to show up.\r\n\r\n----\r\n\r\nStill left to do:\r\n\r\n- [x] UI that shows which sort order is currently being applied (in HTML and in JSON)\r\n- [x] UI for applying a sort order (with rel=nofollow to avoid Google crawling it)\r\n- [x] Sort column names should be escaped correctly in generated SQL\r\n- [x] Validation that the selected sort order is a valid column\r\n- [x] Throw error if user attempts to apply _sort AND _sort_desc at the same time\r\n- [x] Ability to disable sorting (or sort only for specific columns) in metadata.json\r\n- [x] Fix \"201 rows where sorted by sortable_with_nulls \" bug\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/189/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1901416155, "node_id": "I_kwDOBm6k_c5xVU7b", "number": 2189, "title": "Server hang on parallel execution of queries to named in-memory databases", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 31, "created_at": "2023-09-18T17:23:18Z", "updated_at": "2023-09-21T22:26:21Z", "closed_at": "2023-09-21T22:26:21Z", "author_association": "OWNER", "pull_request": null, "body": "I've started to encounter a bug where queries to tables inside named in-memory databases sometimes trigger server hangs.\r\n\r\nI'm still trying to figure out what's going on here - on one occasion I managed to Ctrl+C the server and saw an exception that mentioned a thread lock, but usually hitting Ctrl+C does nothing and I have to `kill -9` the PID instead.\r\n\r\nThis is all running on my M2 Mac.\r\n\r\nI've seen the bug in the Datasette 1.0 alphas and in Datasette 0.64.3 - but reverting to 0.61 appeared to fix it.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/2189/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1421552095, "node_id": "I_kwDOBm6k_c5Uuynf", "number": 1852, "title": "Default API token authentication mechanism", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 8658075, "label": "Datasette 1.0a0"}, "comments": 30, "created_at": "2022-10-24T22:31:07Z", "updated_at": "2022-11-15T19:57:00Z", "closed_at": "2022-10-26T02:19:54Z", "author_association": "OWNER", "pull_request": null, "body": "API authentication will be via `Authorization: Bearer XXX` request headers.\r\n\r\nI'm inclined to add a default token mechanism to Datasette based on tokens that are signed with the `DATASETTE_SECRET`. Maybe the root user can access `/-/create-token` which provides a UI for generating a time-limited signed token? Could also have a `datasette token` command for creating such tokens at the command-line.\r\n\r\nPlugins can then define alternative ways of creating tokens, such as the existing https://datasette.io/plugins/datasette-auth-tokens plugin.\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1850#issuecomment-1289706439_\r\n ", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1852/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1499150951, "node_id": "PR_kwDOBm6k_c5FlZmG", "number": 1960, "title": "Port as many tests as possible to async def tests against ds_client", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 29, "created_at": "2022-12-15T21:45:53Z", "updated_at": "2022-12-17T21:47:56Z", "closed_at": "2022-12-17T21:47:55Z", "author_association": "OWNER", "pull_request": "simonw/datasette/pulls/1960", "body": "Refs:\r\n- #1959\r\n\r\n\r\n----\n:books: Documentation preview :books:: https://datasette--1960.org.readthedocs.build/en/1960/\n\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1960/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 608058890, "node_id": "MDU6SXNzdWU2MDgwNTg4OTA=", "number": 744, "title": "link_or_copy_directory() error - Invalid cross-device link", "user": {"value": 30607, "label": "aborruso"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 28, "created_at": "2020-04-28T06:26:45Z", "updated_at": "2020-05-28T14:32:53Z", "closed_at": "2020-05-27T06:01:28Z", "author_association": "NONE", "pull_request": null, "body": "Hi,\r\nwhen I run\r\n\r\n```\r\ndatasette publish heroku -n myapp --template-dir ./template mydb.db\r\n```\r\n\r\nI have this error\r\n\r\n```\r\nTraceback (most recent call last):\r\n File \"/home/aborruso/.local/lib/python3.7/site-packages/datasette/utils/__init__.py\", line 607, in link_or_copy_directory\r\n shutil.copytree(src, dst, copy_function=os.link)\r\n File \"/usr/lib/python3.7/shutil.py\", line 365, in copytree\r\n raise Error(errors)\r\nshutil.Error: [('/myfolder/youtubeComunePalermo/processing/./template/base.html', '/tmp/tmps9_4mzc4/templates/base.html', \"[Errno 18] Invalid cross-device link: '/myfolder/youtubeComunePalermo/processing/./template/base.html' -> '/tmp/tmps9_4mzc4/templates/base.html'\"), ('/myfolder/youtubeComunePalermo/processing/./template/index.html', '/tmp/tmps9_4mzc4/templates/index.html', \"[Errno 18] Invalid cross-device link: '/myfolder/youtubeComunePalermo/processing/./template/index.html' -> '/tmp/tmps9_4mzc4/templates/index.html'\")]\r\n\r\nDuring handling of the above exception, another exception occurred:\r\n\r\nTraceback (most recent call last):\r\n File \"/home/aborruso/.local/bin/datasette\", line 8, in ![\"image\"](\"https://user-images.githubusercontent.com/9599/145907011-e4abc93c-8dbf-468d-afef-13d5e7463f92.png\")
\r\n\r\nI imagine this could be sped up a lot using tricks like this one from the SQLite documentation: https://sqlite.org/pragma.html#pragfunc\r\n\r\n```sql\r\nSELECT DISTINCT m.name || '.' || ii.name AS 'indexed-columns'\r\n FROM sqlite_schema AS m,\r\n pragma_index_list(m.name) AS il,\r\n pragma_index_info(il.name) AS ii\r\n WHERE m.type='table'\r\n ORDER BY 1;\r\n```\r\nhttps://latest-with-plugins.datasette.io/fixtures?sql=SELECT+DISTINCT+m.name+%7C%7C+%27.%27+%7C%7C+ii.name+AS+%27indexed-columns%27%0D%0A++FROM+sqlite_schema+AS+m%2C%0D%0A+++++++pragma_index_list%28m.name%29+AS+il%2C%0D%0A+++++++pragma_index_info%28il.name%29+AS+ii%0D%0A+WHERE+m.type%3D%27table%27%0D%0A+ORDER+BY+1%3B", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1555/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 325958506, "node_id": "MDU6SXNzdWUzMjU5NTg1MDY=", "number": 283, "title": "Support cross-database joins", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 26, "created_at": "2018-05-24T04:18:39Z", "updated_at": "2021-06-06T09:40:18Z", "closed_at": "2021-02-18T22:16:46Z", "author_association": "OWNER", "pull_request": null, "body": "SQLite has the ability to attach multiple databases to a single connection and then run joins across multiple databases.\r\n\r\nSince Datasette supports more than one database, this would make a pretty neat feature.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/283/reactions\", \"total_count\": 2, \"+1\": 2, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 582517965, "node_id": "MDU6SXNzdWU1ODI1MTc5NjU=", "number": 698, "title": "Ability for a canned query to write to the database", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 5512395, "label": "Datasette 0.44"}, "comments": 26, "created_at": "2020-03-16T18:31:59Z", "updated_at": "2020-06-06T19:43:49Z", "closed_at": "2020-06-06T19:43:48Z", "author_association": "OWNER", "pull_request": null, "body": "Canned queries are currently read-only: https://datasette.readthedocs.io/en/0.38/sql_queries.html#canned-queries\r\n\r\nAdd a `\"write\": true` option to their definition in `metadata.json` which turns them into queries that are submitted via POST and send their queries to the write queue.\r\n\r\nThen they can be used as a really quick way to define a writable interface and JSON API!", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/698/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 621989740, "node_id": "MDU6SXNzdWU2MjE5ODk3NDA=", "number": 114, "title": "table.transform() method for advanced alter table", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 5897911, "label": "2.20"}, "comments": 26, "created_at": "2020-05-20T18:20:46Z", "updated_at": "2020-09-22T07:51:37Z", "closed_at": "2020-09-22T04:20:02Z", "author_association": "OWNER", "pull_request": null, "body": "SQLite's `ALTER TABLE` can only do the following:\r\n\r\n* Rename a table\r\n* Rename a column\r\n* Add a column\r\n\r\nNotably, it cannot drop columns - so tricks like \"add a float version of this text column, populate it, then drop the old one and rename\" won't work.\r\n\r\nThe docs here https://www.sqlite.org/lang_altertable.html#making_other_kinds_of_table_schema_changes describe a way of implementing full alters safely within a transaction, but it's fiddly.\r\n\r\n1. Create new table\r\n2. Copy data\r\n3. Drop old table\r\n4. Rename new into old \r\n\r\nIt would be great if `sqlite-utils` provided an abstraction to help make these kinds of changes safely.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/114/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 648435885, "node_id": "MDU6SXNzdWU2NDg0MzU4ODU=", "number": 878, "title": "New pattern for views that return either JSON or HTML, available for plugins", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": {"value": 3268330, "label": "Datasette 1.0"}, "comments": 26, "created_at": "2020-06-30T19:26:13Z", "updated_at": "2022-03-19T16:19:30Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "Can be part of #870 - refactoring existing views to use `register_routes()`.\r\n\r\n> I'm going to put the new `check_permissions()` method on `BaseView` as well. If I want that method to be available to plugins I can do so by turning that `BaseView` class into a documented API that plugins are encouraged to use themselves.\r\n_Originally posted by @simonw in https://github.com/simonw/datasette/issues/832#issuecomment-651995453_", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/878/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 639993467, "node_id": "MDU6SXNzdWU2Mzk5OTM0Njc=", "number": 850, "title": "Proof of concept for Datasette on AWS Lambda with EFS", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 25, "created_at": "2020-06-16T21:48:31Z", "updated_at": "2020-06-16T23:52:16Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "https://aws.amazon.com/about-aws/whats-new/2020/06/aws-lambda-support-for-amazon-elastic-file-system-now-generally-/\r\n\r\nIf Datasette can run on Lambda with access to EFS it could both read AND write large databases there.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/850/reactions\", \"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 813880401, "node_id": "MDExOlB1bGxSZXF1ZXN0NTc3OTUzNzI3", "number": 5, "title": "WIP: Add Gmail takeout mbox import", "user": {"value": 306240, "label": "UtahDave"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 25, "created_at": "2021-02-22T21:30:40Z", "updated_at": "2021-07-28T07:18:56Z", "closed_at": null, "author_association": "FIRST_TIME_CONTRIBUTOR", "pull_request": "dogsheep/google-takeout-to-sqlite/pulls/5", "body": "WIP\r\n\r\nThis PR adds the ability to import emails from a Gmail mbox export from Google Takeout.\r\n\r\nThis is my first PR to a datasette/dogsheep repo. I've tested this on my personal Google Takeout mbox with ~520,000 emails going back to 2004. This took around ~20 minutes to process.\r\n\r\nTo provide some feedback on the progress of the import I added the \"rich\" python module. I'm happy to remove that if adding a dependency is discouraged. However, I think it makes a nice addition to give feedback on the progress of a long import.\r\n\r\nDo we want to log emails that have errors when trying to import them?\r\n\r\nDealing with encodings with emails is a bit tricky. I'm very open to feedback on how to deal with those better. As well as any other feedback for improvements.", "repo": {"value": 206649770, "label": "google-takeout-to-sqlite"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/5/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 1124731464, "node_id": "I_kwDOCGYnMM5DCgpI", "number": 399, "title": "Make it easier to insert geometries, with documentation and maybe code", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 25, "created_at": "2022-02-05T00:11:26Z", "updated_at": "2023-05-16T03:11:52Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "In playing with the new SpatiaLite helpers from #385 I noticed that actually populating geometry columns is still a little bit tricky. Here's what I ended up doing:\r\n\r\n```python\r\nimport httpx, sqlite_utils\r\ndb = sqlite_utils.Database(\"/tmp/spatial.db\")\r\nattractions = httpx.get(\"https://latest.datasette.io/fixtures/roadside_attractions.json?_shape=array\").json()\r\ndb[\"attractions\"].insert_all(attractions, pk=\"pk\")\r\n\r\n# Schema of that table is now:\r\n# CREATE TABLE [attractions] (\r\n# [pk] INTEGER PRIMARY KEY,\r\n# [name] TEXT,\r\n# [address] TEXT,\r\n# [latitude] FLOAT,\r\n# [longitude] FLOAT\r\n# )\r\n\r\ndb.init_spatialite()\r\ndb[\"attractions\"].add_geometry_column(\"point\", \"POINT\")\r\n\r\ndb.execute(\"\"\"\r\n update attractions set point = GeomFromText(\r\n 'POINT(' || longitude || ' ' || latitude || ')', 4326\r\n )\r\n\"\"\")\r\n```\r\nThat last line took some figuring out - especially the need for the SRID of `4326`, without which I got this error:\r\n\r\n> `IntegrityError: attractions.point violates Geometry constraint [geom-type or SRID not allowed]`\r\n\r\nIt would be good to both document this in more detail, but ideally also to come up with a more obvious pattern for inserting common types of spatial data.\r\n\r\nAlso related:\r\n- #398\r\n- #79", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/399/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 1662951875, "node_id": "I_kwDOBm6k_c5jHqHD", "number": 2057, "title": "DeprecationWarning: pkg_resources is deprecated as an API", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 25, "created_at": "2023-04-11T17:41:20Z", "updated_at": "2023-09-21T22:09:10Z", "closed_at": "2023-09-21T22:09:10Z", "author_association": "OWNER", "pull_request": null, "body": "Got this running tests against Python 3.11.\r\n\r\n```\r\n../../../.local/share/virtualenvs/datasette-big-local-6Yn-280V/lib/python3.11/site-packages/datasette/app.py:14: in \r\n
", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/859/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 657572753, "node_id": "MDU6SXNzdWU2NTc1NzI3NTM=", "number": 894, "title": "?sort=colname~numeric to sort by by column cast to real", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 21, "created_at": "2020-07-15T18:47:48Z", "updated_at": "2021-08-20T02:07:53Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "If a text column actually contains numbers, being able to \"sort by column, treated as numeric\" would be really useful.\r\n\r\nProbably depends on column actions enabled by #690", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/894/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 777333388, "node_id": "MDU6SXNzdWU3NzczMzMzODg=", "number": 1168, "title": "Mechanism for storing metadata in _metadata tables", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 21, "created_at": "2021-01-01T18:47:27Z", "updated_at": "2023-09-28T18:29:05Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "_Original title: Perhaps metadata should all live in a `_metadata` in-memory database_\r\n\r\nInspired by #1150 - metadata should be exposed as an API, and for large Datasette instances that API may need to be paginated. So why not expose it through an in-memory database table?\r\n\r\nOne catch to this: plugins. #860 aims to add a plugin hook for metadata. But if the metadata comes from an in-memory table, how do the plugins interact with it?\r\n\r\nThe need to paginate over metadata does make a plugin hook that returns metadata for an individual table seem less wise, since we don't want to have to do 10,000 plugin hook invocations to show a list of all metadata.\r\n\r\nIf those plugins write directly to the in-memory table how can their contributions survive the server restarting?", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1168/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 1243498298, "node_id": "I_kwDOBm6k_c5KHkc6", "number": 1746, "title": "Switch documentation theme to Furo", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 21, "created_at": "2022-05-20T18:42:17Z", "updated_at": "2022-05-20T21:28:29Z", "closed_at": "2022-05-20T21:28:29Z", "author_association": "OWNER", "pull_request": null, "body": "https://github.com/pradyunsg/furo\r\n\r\nI just did this for `shot-scraper` and I really like it: https://shot-scraper.datasette.io/en/latest/\r\n\r\n- https://github.com/simonw/shot-scraper/issues/77", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1746/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 628499086, "node_id": "MDU6SXNzdWU2Mjg0OTkwODY=", "number": 790, "title": "\"flash messages\" mechanism", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 5512395, "label": "Datasette 0.44"}, "comments": 20, "created_at": "2020-06-01T14:55:44Z", "updated_at": "2020-06-08T19:33:59Z", "closed_at": "2020-06-02T21:14:03Z", "author_association": "OWNER", "pull_request": null, "body": "> Passing `?_success` like this isn't necessarily the best approach. Potential improvements include:\r\n> \r\n> - Signing this message so it can't be tampered with (I could generate a signing secret on startup)\r\n> - Using a cookie with a temporary flash message in it instead\r\n> - Using HTML5 history API to remove the `?_success=` from the URL bar when the user lands on the page\r\n> \r\n> If I add an option to redirect the user to another page after success I may need a mechanism to show a flash message on that page as well, in which case I'll need a general flash message solution that works for any page.\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/datasette/pull/703_", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/790/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 679808124, "node_id": "MDU6SXNzdWU2Nzk4MDgxMjQ=", "number": 940, "title": "Move CI to GitHub Issues", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 5818042, "label": "Datasette 0.49"}, "comments": 20, "created_at": "2020-08-16T19:06:08Z", "updated_at": "2020-09-14T22:09:35Z", "closed_at": "2020-09-14T22:09:35Z", "author_association": "OWNER", "pull_request": null, "body": "It looks like the tests take 3m33s to run in GitHub Actions, but they're taking more than 8 minutes in Travis", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/940/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 733499930, "node_id": "MDU6SXNzdWU3MzM0OTk5MzA=", "number": 1072, "title": "load_template hook doesn't work for include/extends", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 6026070, "label": "0.51"}, "comments": 20, "created_at": "2020-10-30T20:33:44Z", "updated_at": "2020-10-31T20:48:18Z", "closed_at": "2020-10-30T22:50:57Z", "author_association": "OWNER", "pull_request": null, "body": "Includes like this one always go to disk, without hitting the `load_template` plugin hook:\r\n```html+jinja\r\n\r\n```", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1072/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1174306154, "node_id": "I_kwDOBm6k_c5F_n1q", "number": 1668, "title": "Introduce concept of a database `route`, separate from its name", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 3268330, "label": "Datasette 1.0"}, "comments": 20, "created_at": "2022-03-19T16:48:28Z", "updated_at": "2022-03-20T16:43:16Z", "closed_at": "2022-03-20T16:43:16Z", "author_association": "OWNER", "pull_request": null, "body": "Some issues came up in the new `datasette-hashed-urls` plugin relating to the way it renames databases on startup to achieve unique URLs that depend on the database SHA-256 content:\r\n\r\n- https://github.com/simonw/datasette-hashed-urls/issues/10\r\n- https://github.com/simonw/datasette-hashed-urls/issues/9\r\n- https://github.com/simonw/datasette-hashed-urls/issues/8\r\n\r\nAll three of these could be addressed by making the \"path\" concept for a database (the `/foo` bit where it is served) work independently of the database's name, which would be used for default display and also as the alias when configuring cross-database aliases.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1668/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1250629388, "node_id": "I_kwDOCGYnMM5KixcM", "number": 440, "title": "CSV files with too many values in a row cause errors", "user": {"value": 4068, "label": "frafra"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 20, "created_at": "2022-05-27T10:54:44Z", "updated_at": "2022-06-14T22:23:01Z", "closed_at": "2022-06-14T20:12:46Z", "author_association": "NONE", "pull_request": null, "body": "*Original title: csv.DictReader can have None as key*\r\n\r\nIn some cases, `csv.DictReader` can have `None` as key for unnamed columns, and a list of values as value.\r\n`sqlite_utils.utils.rows_from_file` cannot handle that:\r\n\r\n```python\r\nurl=\"https://artsdatabanken.no/Fab2018/api/export/csv\"\r\ndb = sqlite_utils.Database(\":memory\")\r\n\r\nwith urlopen(url) as fab:\r\n reader, _ = sqlite_utils.utils.rows_from_file(fab, encoding=\"utf-16le\") \r\n db[\"fab2018\"].insert_all(reader, pk=\"Id\")\r\n```\r\n\r\nResult:\r\n```\r\nTraceback (most recent call last):\r\n File \"\r\n/-/versions output\r\n
\r\n\r\n{\r\n \"python\": {\r\n \"version\": \"3.8.0\",\r\n \"full\": \"3.8.0 (default, Oct 28 2019, 16:14:01) \\n[GCC 8.3.0]\"\r\n },\r\n \"datasette\": {\r\n \"version\": \"0.44\"\r\n },\r\n \"asgi\": \"3.0\",\r\n \"uvicorn\": \"0.11.5\",\r\n \"sqlite\": {\r\n \"version\": \"3.22.0\",\r\n \"fts_versions\": [\r\n \"FTS5\",\r\n \"FTS4\",\r\n \"FTS3\"\r\n ],\r\n \"extensions\": {\r\n \"json1\": null\r\n },\r\n \"compile_options\": [\r\n \"COMPILER=gcc-7.4.0\",\r\n \"ENABLE_COLUMN_METADATA\",\r\n \"ENABLE_DBSTAT_VTAB\",\r\n \"ENABLE_FTS3\",\r\n \"ENABLE_FTS3_PARENTHESIS\",\r\n \"ENABLE_FTS3_TOKENIZER\",\r\n \"ENABLE_FTS4\",\r\n \"ENABLE_FTS5\",\r\n \"ENABLE_JSON1\",\r\n \"ENABLE_LOAD_EXTENSION\",\r\n \"ENABLE_PREUPDATE_HOOK\",\r\n \"ENABLE_RTREE\",\r\n \"ENABLE_SESSION\",\r\n \"ENABLE_STMTVTAB\",\r\n \"ENABLE_UNLOCK_NOTIFY\",\r\n \"ENABLE_UPDATE_DELETE_LIMIT\",\r\n \"HAVE_ISNAN\",\r\n \"LIKE_DOESNT_MATCH_BLOBS\",\r\n \"MAX_SCHEMA_RETRY=25\",\r\n \"MAX_VARIABLE_NUMBER=250000\",\r\n \"OMIT_LOOKASIDE\",\r\n \"SECURE_DELETE\",\r\n \"SOUNDEX\",\r\n \"TEMP_STORE=1\",\r\n \"THREADSAFE=1\"\r\n ]\r\n }\r\n}\r\n\r\n