{"id": 870125126, "node_id": "MDU6SXNzdWU4NzAxMjUxMjY=", "number": 1310, "title": "I'm creating a plugin to export a spreadsheet file (.ods or .xlsx)", "user": {"value": 3747136, "label": "ColinMaudry"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2021-04-28T16:20:11Z", "updated_at": "2021-04-30T07:26:11Z", "closed_at": "2021-04-30T06:58:46Z", "author_association": "NONE", "pull_request": null, "body": "Hi,\r\n\r\nI have started developing a plugin to export records as a spreadsheet file. It could be ods or xlsx, whatever is easier.\r\n\r\nI have spotted the following packages:\r\n\r\n- ods files: https://pypi.org/project/odswriter/\r\n- xlsx files: https://openpyxl.readthedocs.io/en/stable/index.html (quite powerful) or https://xlsxwriter.readthedocs.io/ (faster)\r\n\r\nThis is the code I have so far, I test it with the `--plugins-dir` option:\r\n\r\n```python\r\nfrom datasette import hookimpl\r\nfrom datasette.utils.asgi import Response\r\nimport odswriter as ods\r\n\r\ndef render_spreadsheet(rows):\r\n    with ods.writer(open(\"test.ods\",\"wb\")) as odsfile:\r\n        for row in rows:\r\n            odsfile.writerow([\"String\", \"ABCDEF123456\", \"123456\"])\r\n        return Response(odsfile, content_type=\"application/vnd.oasis.opendocument.spreadsheet\", status=200)\r\n\r\n\r\n@hookimpl\r\ndef register_output_renderer():\r\n    return {\"extension\": \"ods\", \"render\": render_spreadsheet}\r\n\r\n``` \r\n\r\nI get the following error:\r\n\r\n```\r\nTraceback (most recent call last):\r\n  File \"/home/colin/.local/lib/python3.8/site-packages/datasette/app.py\", line 1128, in route_path\r\n    await response.asgi_send(send)\r\n  File \"/home/colin/.local/lib/python3.8/site-packages/datasette/utils/asgi.py\", line 339, in asgi_send\r\n    body = body.encode(\"utf-8\")\r\nAttributeError: 'ODSWriter' object has no attribute 'encode'\r\nERROR:    Exception in ASGI application\r\nTraceback (most recent call last):\r\n  File \"/home/colin/.local/lib/python3.8/site-packages/datasette/app.py\", line 1128, in route_path\r\n    await response.asgi_send(send)\r\n  File \"/home/colin/.local/lib/python3.8/site-packages/datasette/utils/asgi.py\", line 339, in asgi_send\r\n    body = body.encode(\"utf-8\")\r\nAttributeError: 'ODSWriter' object has no attribute 'encode'\r\n\r\nDuring handling of the above exception, another exception occurred:\r\n\r\nTraceback (most recent call last):\r\n  File \"/home/colin/.local/lib/python3.8/site-packages/uvicorn/protocols/http/h11_impl.py\", line 396, in run_asgi\r\n    result = await app(self.scope, self.receive, self.send)\r\n  File \"/home/colin/.local/lib/python3.8/site-packages/uvicorn/middleware/proxy_headers.py\", line 45, in __call__\r\n    return await self.app(scope, receive, send)\r\n  File \"/home/colin/.local/lib/python3.8/site-packages/datasette/utils/asgi.py\", line 161, in __call__\r\n    await self.app(scope, receive, send)\r\n  File \"/home/colin/.local/lib/python3.8/site-packages/datasette/tracer.py\", line 75, in __call__\r\n    await self.app(scope, receive, send)\r\n  File \"/home/colin/.local/lib/python3.8/site-packages/asgi_csrf.py\", line 107, in app_wrapped_with_csrf\r\n    await app(scope, receive, wrapped_send)\r\n  File \"/home/colin/.local/lib/python3.8/site-packages/datasette/app.py\", line 1086, in __call__\r\n    return await self.route_path(scope, receive, send, path)\r\n  File \"/home/colin/.local/lib/python3.8/site-packages/datasette/app.py\", line 1133, in route_path\r\n    return await self.handle_500(request, send, exception)\r\n  File \"/home/colin/.local/lib/python3.8/site-packages/datasette/app.py\", line 1267, in handle_500\r\n    await asgi_send_html(\r\n  File \"/home/colin/.local/lib/python3.8/site-packages/datasette/utils/asgi.py\", line 217, in asgi_send_html\r\n    await asgi_send(\r\n  File \"/home/colin/.local/lib/python3.8/site-packages/datasette/utils/asgi.py\", line 237, in asgi_send\r\n    await asgi_start(send, status, headers, content_type)\r\n  File \"/home/colin/.local/lib/python3.8/site-packages/datasette/utils/asgi.py\", line 246, in asgi_start\r\n    await send(\r\n  File \"/home/colin/.local/lib/python3.8/site-packages/asgi_csrf.py\", line 103, in wrapped_send\r\n    await send(event)\r\n  File \"/home/colin/.local/lib/python3.8/site-packages/uvicorn/protocols/http/h11_impl.py\", line 482, in send\r\n    raise RuntimeError(msg % message_type)\r\nRuntimeError: Expected ASGI message 'http.response.body', but got 'http.response.start'.\r\n```\r\n\r\nI tried with `AsgiFileDownload` like in [DatabaseDownload](https://github.com/simonw/datasette/blob/main/datasette/views/database.py#L150) to deal with the binary nature of the ods file, but the renderer expects a Response:\r\n\r\n> <datasette.utils.asgi.AsgiFileDownload object at 0x7f5b9bd503d0> should be dict or Response\r\n\r\nHowever, the `Response` class only supports the following methods, not binary:\r\n\r\n- html\r\n- text\r\n- json\r\n- redirect\r\n\r\nHow would you suggest me to proceed to have my ods file downloaded?\r\n\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1310/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 871304967, "node_id": "MDU6SXNzdWU4NzEzMDQ5Njc=", "number": 1315, "title": "settings.json should be picked up by \"datasette publish cloudrun\"", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2021-04-29T18:16:41Z", "updated_at": "2021-04-29T18:16:41Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1315/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 870946764, "node_id": "MDU6SXNzdWU4NzA5NDY3NjQ=", "number": 1312, "title": "how to query many-to-many relationship via json API?", "user": {"value": 5268174, "label": "bram2000"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2021-04-29T12:09:49Z", "updated_at": "2021-04-29T12:09:49Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "Hi,\r\n\r\nFirstly thanks for Datasette, it's great!\r\n\r\nI'm trying to use the JSON API to query data from a Datasette instance. I have a simple 3 table many-to-many relationship, like so:\r\n\r\n`category` - list of categories\r\n`document` - list of documents\r\n`document_category` - join table (a category contains many documents, and a document can be a member of multiple categories)\r\n\r\nthe `document_category` table foreign keys to the other two using their respective row_ids.\r\n\r\nNow I want to return \"all documents within category X\" but I cannot see a way to do this without executing two queries; the first to lookup the row_id of category X, and the second to join `document` with `document_category` where category ID is <id>.\r\n\r\nI could easily write this in SQL, but this makes programmatic handling of pagination much more difficult (we'd have to dynamically modify the SQL to select the row_id and include the correct where and limit clauses).\r\n\r\nIs there a way to achieve this using the JSON API?\r\n\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1312/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 281110295, "node_id": "MDU6SXNzdWUyODExMTAyOTU=", "number": 173, "title": "I18n and L10n support", "user": {"value": 50138, "label": "janimo"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2017-12-11T17:49:58Z", "updated_at": "2021-04-26T12:10:01Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "It would be less geeky and more user friendly if the display strings in the filter menu and possibly other parts could be localized.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/173/reactions\", \"total_count\": 2, \"+1\": 2, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 866668415, "node_id": "MDU6SXNzdWU4NjY2Njg0MTU=", "number": 1308, "title": "Columns named \"link\" display in bold", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2021-04-24T05:58:11Z", "updated_at": "2021-04-24T06:07:49Z", "closed_at": "2021-04-24T06:07:49Z", "author_association": "OWNER", "pull_request": null, "body": "Reported in office hours today.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1308/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 860734722, "node_id": "MDU6SXNzdWU4NjA3MzQ3MjI=", "number": 1302, "title": "Fix disappearing facets", "user": {"value": 192568, "label": "mroswell"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2021-04-18T18:42:33Z", "updated_at": "2021-04-20T07:40:15Z", "closed_at": null, "author_association": "CONTRIBUTOR", "pull_request": null, "body": "1. Clone https://github.com/mroswell/list-N\r\n2. Run `datasette disinfectants.db -o`\r\n3. Select the `Safer_or_Toxic` facet.\r\n4. Select `Toxic`.\r\n5. Close out the `Safer_or_Toxic` facet.\r\n6. Examine `Suggested facets` list. `Safer_or_Toxic` is GONE.\r\n7. Try some other facets. When you select an element, and then close the list, in some cases, the facet properly returns to the `Suggested facet` list... Arrays and dates properly return to the list, but fields with strings don't return to the list. \r\n\r\nSince my site is devoted to whether disinfectants are Safer or Toxic, having the suggested facet disappear from the suggested facet list is very confusing* to end-users. This, along with a few other issues, unfortunately proved beyond my own programming ability to address. So I hired a Senior-level developer to address a number of issues, including this disappearing act.\r\n\r\n8. Open a new terminal. Run `datasette disinfectants.db -m metadata.json --static static:static/  --template-dir templates/ --plugins-dir plugins/ -p 8001 -o`\r\n9. Repeat steps 3-6, but this time, the Safer_or_Toxic facet returns to the list (and the related URL parameters are removed).\r\n\r\nI'm not sure how to do a pull request for this, because the plugin contains other functionality that goes beyond this bug. I wanted the facets sorted in a certain order (both in the suggested facet list, and the detail lists) (... the detail lists were hopping around all over the place before...) I wanted the duplicate facets removed (leaving only the one where you can facet by individual item in an array.) I wanted the arrays to be presented in a prettier fashion (I did that in the template... That could be moved over to the plugin at some point)\r\n\r\nI'm thinking it'll be very helpful if applicable parts of my project's plugin (sort_suggested_facets_plugin.py) will be able to be incorporated back into datasette, but I leave that to you to consider.\r\n\r\n(* The disappearing facet bug was especially confusing because I'm removing the filters and sql from the table page, at the request of the organization. The filters and sql detail created a lot of confusion for end users who try to find disinfectants used by Hospitals, for instance, as an '=' won't find them, since they are part of the Use_site array.) My disappearing-facet confusion was documented in my own issue: https://github.com/mroswell/list-N/issues/57 (addressed by the plugin). Other facet-related issues here: https://github.com/mroswell/list-N/issues/54 (addressed by the plugin); https://github.com/mroswell/list-N/issues/15 (addressed by template); https://github.com/mroswell/list-N/issues/53 (not yet addressed). \r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1302/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 791237799, "node_id": "MDU6SXNzdWU3OTEyMzc3OTk=", "number": 1196, "title": "Access Denied Error in Windows", "user": {"value": 2826376, "label": "QAInsights"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2021-01-21T15:40:40Z", "updated_at": "2021-04-14T19:28:38Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "I am trying to publish a db to vercel. But while issuing the below command throwing `Access Denied` error which is leading to `RecursionError: maximum recursion depth exceeded while calling a Python object`.\r\n\r\nI am using PyCharm and Python 3.9. I have reinstalled both and launched PyCharm as Admin in Windows 10. But still the issue persists.\r\n\r\nIssued command `datasette publish vercel jmeter.db --project jmeter --install datasette-vega`\r\n\r\nPS: localhost is working fine.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1196/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 856895291, "node_id": "MDU6SXNzdWU4NTY4OTUyOTE=", "number": 1299, "title": "Design better empty states", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2021-04-13T12:06:12Z", "updated_at": "2021-04-13T12:06:12Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "Inspiration here: https://emptystat.es/", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1299/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 855451460, "node_id": "MDU6SXNzdWU4NTU0NTE0NjA=", "number": 1297, "title": "Documentation: json1, and introspection endpoints", "user": {"value": 192568, "label": "mroswell"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2021-04-12T00:38:00Z", "updated_at": "2021-04-12T01:29:33Z", "closed_at": null, "author_association": "CONTRIBUTOR", "pull_request": null, "body": "https://docs.datasette.io/en/stable/facets.html notes that:\r\n> If your SQLite installation provides the json1 extension (you can check using /-/versions) Datasette will automatically detect columns that contain JSON arrays...\r\n\r\nWhen I check -/versions I see two sections relevant to json1:\r\n```\r\n        \"extensions\": {\r\n            \"json1\": null\r\n        },\r\n        \"compile_options\": [\r\n            ...\r\n            \"ENABLE_JSON1\",\r\n```\r\n        \r\n   The ENABLE_JSON1 makes me think json1 is likely available. But the  `\"json1\": null` made me think it wasn't available (because of the `null`). It would help if the documentation provided clarity about how to know if json1 is installed. It would also be helpful if the `/-/versions` information signalled somehow that that is to be appended to the hostname or domain name (or whatever you want to call it, or simply show it, using `example.com/-/versions` instead of `/-/versions`. Likewise on that last point, for https://docs.datasette.io/en/stable/introspection.html#introspection , at least at some point on that page detailing where those introspection endpoints go. (Sometimes documentation can be so abbreviated that it's hard for new users to figure out what's going on.)\r\n   ", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1297/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 853672224, "node_id": "MDU6SXNzdWU4NTM2NzIyMjQ=", "number": 1294, "title": "\"You can check out any time you like. But you can never leave!\"", "user": {"value": 192568, "label": "mroswell"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2021-04-08T17:02:15Z", "updated_at": "2021-04-08T18:35:50Z", "closed_at": null, "author_association": "CONTRIBUTOR", "pull_request": null, "body": "(Feel free to rename this one.)\r\n\r\n- The column gear lets you \"Show not-blank rows.\" Then it places a parameter in the URL, which a web developer would notice, but a lot of users won't notice, or know to delete it. Would be good to toggle \"Show not-blank rows\" with \"Show all rows.\" (Also would be quite helpful to have a \"Show blank rows | Show all rows\" option)\r\n- The column gear lets you \"Sort ascending\" and \"Sort descending\" but then you're stuck with some sort of sorted version thereafter, unless you know to sort the ID column, or to remove the full _sort parameter and its value in the URL. Would be good to offer a \"Remove sort\" option in the gear.\r\n- These requests are in the same camp as: https://github.com/simonw/datasette-vega/issues/36\r\n- I suspect there are other url parameter instances where similar analysis would be helpful, but the three above are the use cases I've run across. \r\n\r\nUPDATE:\r\n- It would be helpful to have a \"Previous page\" available for all but the first table page.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1294/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 520667773, "node_id": "MDU6SXNzdWU1MjA2Njc3NzM=", "number": 620, "title": "Mechanism for indicating foreign key relationships in the table and query page URLs", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2019-11-10T22:26:27Z", "updated_at": "2021-04-05T03:57:22Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "Datasette currently only inflates foreign keys (into names hyperlinks) if it detects them as foreign key constraints in the underlying database.\r\n\r\nIt would be useful if you could specify additional \"foreign keys\" using both `metadata.json` and the querystring - similar time how you can pass `?_fts_table=x` https://datasette.readthedocs.io/en/stable/full_text_search.html#configuring-full-text-search-for-a-table-or-view", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/620/reactions\", \"total_count\": 1, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 1}", "draft": null, "state_reason": null}
{"id": 672421411, "node_id": "MDU6SXNzdWU2NzI0MjE0MTE=", "number": 916, "title": "Support reverse pagination (previous page, has-previous-items)", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 7, "created_at": "2020-08-04T00:32:06Z", "updated_at": "2021-04-03T23:43:11Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "I need this for `datasette-graphql` for full compatibility with the way Relay likes to paginate - using cursors for paginating backwards as well as for paginating forwards.\r\n\r\n> This may be the kick I need to get Datasette pagination to work in reverse too.\r\n_Originally posted by @simonw in https://github.com/simonw/datasette-graphql/issues/2#issuecomment-668305853_", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/916/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 849396758, "node_id": "MDU6SXNzdWU4NDkzOTY3NTg=", "number": 1287, "title": "Upgrade to Python 3.9.4", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 5, "created_at": "2021-04-02T18:43:15Z", "updated_at": "2021-04-03T22:38:39Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "Has some security fixes https://pythoninsider.blogspot.com/2021/04/python-393-and-389-are-now-available.html", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1287/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 453131917, "node_id": "MDU6SXNzdWU0NTMxMzE5MTc=", "number": 502, "title": "Exporting sqlite database(s)?", "user": {"value": 7936571, "label": "chrismp"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2019-06-06T16:39:53Z", "updated_at": "2021-04-03T05:16:54Z", "closed_at": "2019-06-11T18:50:42Z", "author_association": "NONE", "pull_request": null, "body": "I'm working on datasette from one computer. But if I want to work on it from another computer and want to copy the SQLite database(s) already on the Heroku datasette instance, how to I copy the database(s) to the second computer so that I can then update it and push to online via datasette's command line code that pushes code to Heroku?", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/502/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 849543502, "node_id": "MDU6SXNzdWU4NDk1NDM1MDI=", "number": 1289, "title": "Speed up tests with pytest-xdist", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2021-04-03T00:47:39Z", "updated_at": "2021-04-03T03:42:28Z", "closed_at": "2021-04-03T03:42:28Z", "author_association": "OWNER", "pull_request": null, "body": "I think I can get this working for almost every test, then use the pattern in https://github.com/pytest-dev/pytest-xdist/issues/385#issuecomment-444545641 to opt specific tests out of being run in parallel.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1289/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 849512840, "node_id": "MDU6SXNzdWU4NDk1MTI4NDA=", "number": 1288, "title": "Facets: show counts for null", "user": {"value": 1111743, "label": "jungle-boogie"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2021-04-02T22:33:44Z", "updated_at": "2021-04-02T22:33:44Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "Hi,\r\n\r\nThank you for Datasette and being a fan of SQLite!\r\n\r\nNot all rows in a record will always contain data.\r\nSo when using a facet on a column where some records have data and others don't, you don't get an accurate count of the results.\r\n\r\nPlease consider also counting and showing null records with facets.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1288/reactions\", \"total_count\": 2, \"+1\": 2, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 817544251, "node_id": "MDU6SXNzdWU4MTc1NDQyNTE=", "number": 1245, "title": "Sticky table column headers would be useful, especially on the query page", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2021-02-26T17:42:51Z", "updated_at": "2021-04-02T20:53:35Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "Suggestion from office hours.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1245/reactions\", \"total_count\": 2, \"+1\": 2, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 826700095, "node_id": "MDU6SXNzdWU4MjY3MDAwOTU=", "number": 1255, "title": "Facets timing out but work when filtering", "user": {"value": 1219001, "label": "robroc"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2021-03-09T22:01:39Z", "updated_at": "2021-04-02T20:50:08Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "System info:\r\n\r\nWindows 10\r\nDatasette 0.55 installed via pip\r\nPython 3.8.5 in a conda environment\r\n\r\nI'm getting the message `These facets timed out` on any faceting operation. However, when I apply a filter, the facets appear in the filtered view. The error returns when the filter is removed. My data only has 38,450 rows.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1255/reactions\", \"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 847700726, "node_id": "MDU6SXNzdWU4NDc3MDA3MjY=", "number": 1285, "title": "Feature Request or Plugin Request: Numeric Range Facets", "user": {"value": 192568, "label": "mroswell"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2021-04-01T01:50:20Z", "updated_at": "2021-04-01T02:28:19Z", "closed_at": null, "author_association": "CONTRIBUTOR", "pull_request": null, "body": "It would be great to offer facets for numeric data ranges. \r\n\r\nThe ranges could pull from typical GIS methods of creating choropleth maps. \r\nhttps://gisgeography.com/choropleth-maps-data-classification/\r\nOf the following, for mapping, I've always preferred a Jenks Natural Breaks, or a cross between Jenks and Pretty breaks.\r\n\r\n- Equal Intervals \r\n- Quantile (equal count) \r\n- Standard Deviation\r\n- Natural Breaks (Jenks) Classification \r\n- Pretty Breaks\r\n- Some sort of Aggregate Jenks Classification (this isn't standard, but it would be nice to be able to set classification ranges that work across tables.)\r\n\r\nHere are some links for Natural Breaks, in case this method is unfamiliar.\r\n\r\n- https://en.wikipedia.org/wiki/Jenks_natural_breaks_optimization\r\n- http://wiki.gis.com/wiki/index.php/Jenks_Natural_Breaks_Classification\r\n- https://medium.com/analytics-vidhya/jenks-natural-breaks-best-range-finder-algorithm-8d1907192051\r\n\r\nPer that last link, there is a Jenks Python module...  They also describe it as data-intensive for larger datasets. Maybe this is a good plugin idea.\r\n\r\nAn example of equal Intervals would be \r\n0 \u2013 < 10\r\n10 \u2013 < 20\r\n20 \u2013 < 30\r\n30 \u2013 < 40\r\n\r\nIt's kind of confusing to have that less-than sign in there. it could also be displayed as:\r\n0 \u2013 10\r\n10 \u2013 20\r\n20 \u2013 30\r\n30 \u2013 40\r\n\r\nBut then it's not completely clear which category 10 is in, for instance.\r\n\r\n(Best to right-justify.. and use an \"en dash\" between numbers.)\r\n\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1285/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 841456306, "node_id": "MDU6SXNzdWU4NDE0NTYzMDY=", "number": 1276, "title": "Invalid SQL: \"no such table: pragma_database_list\" on database page", "user": {"value": 1314318, "label": "justinallen"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 7, "created_at": "2021-03-26T00:03:53Z", "updated_at": "2021-03-31T16:27:27Z", "closed_at": "2021-03-28T23:52:31Z", "author_association": "NONE", "pull_request": null, "body": "Don't think this has been covered here yet. I'm a little stumped with this one and can't tell if it's a bug or I have something misconfigured. \r\n\r\nOddly, when running locally the usual list of tables populates (i.e. at /charts a list of tables in charts.db). But when on the web server it throws an Invalid SQL error and \"no such table: pragma_database_list\" below. \r\n\r\nAll the url endpoints seem to work fine aside from this - individual tables (/charts/chart_one), as well as stored queries (/charts/query_one). \r\n\r\nNot sure if this has anything to do with upgrading to Datasette 0.55, or something to do with our setup, which uses a metadata build script similar to [the one for the 538 server](https://github.com/simonw/fivethirtyeight-datasette/blob/main/make_metadata.py), or something else. \r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1276/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 843884745, "node_id": "MDU6SXNzdWU4NDM4ODQ3NDU=", "number": 1283, "title": "advanced #export causes unexpected scrolling", "user": {"value": 192568, "label": "mroswell"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2021-03-29T22:46:57Z", "updated_at": "2021-03-29T22:46:57Z", "closed_at": null, "author_association": "CONTRIBUTOR", "pull_request": null, "body": "1. Visit a datasette table page\r\n2. Click on the \"(advanced)\" link. This adds a fragment identifier \"#export\" to the URL, and scrolls down to the \"Advanced export\" div with the \"export\" id.\r\n3. Manually scroll back up, and click on a suggested facet. The fragment identifier is still present, and the app scrolls back down to the \"Advanced export\" div. I think this is unwanted behavior.\r\n\r\nThe user remedy seems to be to manually remove the \"#export\" from the URL.\r\n\r\nThis behavior happens in my project, and in:\r\nhttps://covid-19.datasettes.com/covid/economist_excess_deaths  (for instance) \r\nbut not in this table: \r\nhttps://global-power-plants.datasettes.com/global-power-plants/global-power-plants", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1283/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 576722115, "node_id": "MDU6SXNzdWU1NzY3MjIxMTU=", "number": 696, "title": "Single failing unit test when run inside the Docker image", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 3268330, "label": "Datasette 1.0"}, "comments": 2, "created_at": "2020-03-06T06:16:36Z", "updated_at": "2021-03-29T17:04:19Z", "closed_at": "2021-03-07T07:41:18Z", "author_association": "OWNER", "pull_request": null, "body": "```\r\ndocker run -it -v `pwd`:/mnt datasetteproject/datasette:latest /bin/bash\r\nroot@0e1928cfdf79:/# cd /mnt\r\nroot@0e1928cfdf79:/mnt# pip install -e .[test]\r\nroot@0e1928cfdf79:/mnt# pytest\r\n```\r\nI get one failure!\r\n\r\nIt was for `test_searchable[/fixtures/searchable.json?_search=te*+AND+do*&_searchmode=raw-expected_rows3]`\r\n```\r\n    def test_searchable(app_client, path, expected_rows):\r\n        response = app_client.get(path)\r\n>       assert expected_rows == response.json[\"rows\"]\r\nE       AssertionError: assert [[1, 'barry c...sel', 'puma']] == []\r\nE         Left contains 2 more items, first extra item: [1, 'barry cat', 'terry dog', 'panther']\r\nE         Full diff:\r\nE         + []\r\nE         - [[1, 'barry cat', 'terry dog', 'panther'],\r\nE         -  [2, 'terry dog', 'sara weasel', 'puma']]\r\n```\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/datasette/issues/695#issuecomment-595614469_", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/696/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 842881221, "node_id": "MDU6SXNzdWU4NDI4ODEyMjE=", "number": 1281, "title": "Latest Datasette tags missing from Docker Hub", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 7, "created_at": "2021-03-29T00:58:30Z", "updated_at": "2021-03-29T01:41:48Z", "closed_at": "2021-03-29T01:41:48Z", "author_association": "OWNER", "pull_request": null, "body": "Spotted this while testing https://github.com/simonw/datasette/issues/1249#issuecomment-808998719_\r\n\r\nhttps://hub.docker.com/r/datasetteproject/datasette/tags?page=1&ordering=last_updated isn't showing the tags for any version more recent than 0.54.1 - we are up to 0.56 now.\r\n\r\nBut the `:latest` tag is for the new 0.56 release.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1281/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 807433181, "node_id": "MDU6SXNzdWU4MDc0MzMxODE=", "number": 1224, "title": "can't start immutable databases from configuration dir mode", "user": {"value": 295329, "label": "camallen"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2021-02-12T17:50:13Z", "updated_at": "2021-03-29T00:17:31Z", "closed_at": "2021-03-29T00:17:31Z", "author_association": "CONTRIBUTOR", "pull_request": null, "body": "Say I have a `/databases/` directory with multiple sqlite db files in that dir (`1.db` & `2.db`) and an `inspect-data.json` file.\r\n\r\nIf I start datasette via `datasette -h 0.0.0.0 /databases/` then the resulting databases are set to `is_mutable: true` as inspected via http://127.0.0.1:8001/-/databases.json\r\n\r\nI don't want to have to list out the databases by name, e.g. `datasette -i /databases/1.db -i /databases/2.db` as i want the system to autodetect the sqlite dbs i have in the configuration directory \r\n\r\nAccording to the docs outlined in https://docs.datasette.io/en/latest/settings.html?highlight=immutable#configuration-directory-mode this should be possible\r\n> `inspect-data.json` the result of running datasette inspect - any database files listed here will be treated as immutable, so they should not be changed while Datasette is running\r\n \r\nI believe that if the `inspect-json.json` file present, then in theory the databases will be automatically set to immutable via this code https://github.com/simonw/datasette/blob/9603d893b9b72653895318c9104d754229fdb146/datasette/app.py#L211-L216\r\n\r\nHowever it appears the Click Multiple Options will return a tuple via https://github.com/simonw/datasette/blob/9603d893b9b72653895318c9104d754229fdb146/datasette/cli.py#L311-L317\r\n\r\nThe resulting tuple is passed to the Datasette app via `kwargs` and overrides the behaviour to set the databases to immutable via this arg https://github.com/simonw/datasette/blob/9603d893b9b72653895318c9104d754229fdb146/datasette/app.py#L182\r\n\r\nIf you think this is a bug and needs fixing, I am willing to make a PR to check for the empty `immutable` tuple before calling the Datasette class initializer as I think leaving that class interface alone is the best path here.\r\n\r\nThoughts?\r\n\r\nAlso - i'm loving Datasette, it truly is a wonderful tool, thank you :)", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1224/reactions\", \"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 763207948, "node_id": "MDU6SXNzdWU3NjMyMDc5NDg=", "number": 1141, "title": "Default styling for bullet point lists", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2020-12-12T02:49:33Z", "updated_at": "2021-03-29T00:14:05Z", "closed_at": "2021-03-29T00:14:05Z", "author_association": "OWNER", "pull_request": null, "body": "I just noticed that https://datasette.io/content/recent_releases (which uses `datasette-render-markdown`) is missing its bullet points:\r\n\r\n<img width=\"864\" alt=\"content__recent_releases__399_rows\" src=\"https://user-images.githubusercontent.com/9599/101970823-95b0bb80-3be1-11eb-9512-9274cef64cf3.png\">\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1141/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 741862364, "node_id": "MDU6SXNzdWU3NDE4NjIzNjQ=", "number": 1090, "title": "Custom widgets for canned query forms", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2020-11-12T19:21:07Z", "updated_at": "2021-03-27T16:25:25Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "This is an idea that was cut from the first version of writable canned queries:\r\n\r\n> I really want the option to use a `<textarea>` for a specific value.\r\n> \r\n> Idea: metadata syntax like this:\r\n\r\n> \r\n> ```json\r\n> {\r\n>     \"databases\": {\r\n>        \"my-database\": {\r\n>            \"queries\": {\r\n>                \"add_twitter_handle\": {\r\n>                    \"sql\": \"insert into twitter_handles (username) values (:username)\",\r\n>                    \"write\": true,\r\n>                    \"params\": {\r\n>                        \"username\": {\r\n>                             \"widget\": \"textarea\"\r\n>                        }\r\n>                    }\r\n>                }\r\n>            }\r\n>        }\r\n>     }\r\n> }\r\n> ```\r\n> \r\n> I can ship with some default widgets and provide a plugin hook for registering extra widgets.\r\n>\r\n> This opens up some really exciting possibilities for things like map widgets that let you draw polygons.\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/datasette/issues/698#issuecomment-608125928_", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1090/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 842212586, "node_id": "MDU6SXNzdWU4NDIyMTI1ODY=", "number": 1277, "title": "Facet by array breaks if table name contains a space", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2021-03-26T18:38:19Z", "updated_at": "2021-03-27T03:49:38Z", "closed_at": "2021-03-27T03:49:34Z", "author_association": "OWNER", "pull_request": null, "body": "It breaks when you try to select a filtered item.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1277/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 839367451, "node_id": "MDU6SXNzdWU4MzkzNjc0NTE=", "number": 1275, "title": "Idea: long-running query mode", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2021-03-24T05:23:20Z", "updated_at": "2021-03-24T05:23:20Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "It would be cool if you could run Datasette in a long-running query mode, for use with trusted users - something like this:\r\n\r\n    datasette --unlimited my.db\r\n\r\nThis would disable the query limit, but would also enable a feature where if a query takes longer than e.g. 1s to return Datasette returns an HTML page to the browser with a progress indicator and polls the server until the query is complete.... but also provides the user with a \"cancel\" button. This relates to the `.interrupt()` research in #1270.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1275/reactions\", \"total_count\": 2, \"+1\": 2, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 839008371, "node_id": "MDU6SXNzdWU4MzkwMDgzNzE=", "number": 1274, "title": "Might there be some way to comment metadata.json?", "user": {"value": 192568, "label": "mroswell"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2021-03-23T18:33:00Z", "updated_at": "2021-03-23T20:14:54Z", "closed_at": "2021-03-23T20:14:54Z", "author_association": "CONTRIBUTOR", "pull_request": null, "body": "I don't know what license to use... Would be nice to be able to add a comment regarding that uncertainty in my metadata.json file\r\n\r\nI like laktak's little video comment in favor of Human json (Hjson)\r\nhttps://stackoverflow.com/questions/244777/can-comments-be-used-in-json\r\n\r\nHmmm... one of the commenters there said comments are allowed in yaml... so that's a good argument for yaml.\r\n\r\nAnyhow, just came to mind, and thought I'd mention it here. Looks like https://hjson.github.io/ has the details.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1274/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 837350092, "node_id": "MDU6SXNzdWU4MzczNTAwOTI=", "number": 1270, "title": "Try implementing SQLite timeouts using .interrupt() instead of using .set_progress_handler()", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2021-03-22T06:00:17Z", "updated_at": "2021-03-23T16:45:39Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "> Maybe I could implement SQLite query timeouts using the `interrupt()` method instead of the progress handler hack I'm currently using?\r\n>\r\n> https://stackoverflow.com/questions/43240496/python-sqlite3-how-to-quickly-and-cleanly-interrupt-long-running-query-with-e has some tips.\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1268#issuecomment-803764919_", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1270/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 280013907, "node_id": "MDU6SXNzdWUyODAwMTM5MDc=", "number": 164, "title": "datasette skeleton command for kick-starting database and table metadata", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 2949431, "label": "Custom templates edition"}, "comments": 3, "created_at": "2017-12-07T06:13:28Z", "updated_at": "2021-03-23T02:45:12Z", "closed_at": "2017-12-07T06:20:45Z", "author_association": "OWNER", "pull_request": null, "body": "Generates an example `metadata.json` file populated with all of the databases and tables inspected from the specified databases.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/164/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 279547886, "node_id": "MDU6SXNzdWUyNzk1NDc4ODY=", "number": 163, "title": "Document the querystring argument for setting a different time limit", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2017-12-05T22:05:08Z", "updated_at": "2021-03-23T02:44:33Z", "closed_at": "2017-12-06T15:06:57Z", "author_association": "OWNER", "pull_request": null, "body": "http://datasette.readthedocs.io/en/latest/sql_queries.html#query-limits\r\n\r\nNeed to explain why this is useful too.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/163/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 273775212, "node_id": "MDU6SXNzdWUyNzM3NzUyMTI=", "number": 88, "title": "Add NHS England Hospitals example to wiki", "user": {"value": 15543, "label": "tomdyson"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2017-11-14T12:29:10Z", "updated_at": "2021-03-22T23:46:36Z", "closed_at": "2017-11-14T22:54:06Z", "author_association": "CONTRIBUTOR", "pull_request": null, "body": "https://nhs-england-hospitals.now.sh\r\n\r\nand an associated map visualisation:\r\n\r\nhttp://run.plnkr.co/preview/cj9zlf1qc0003414y90ajkwpk/\r\n\r\nDatasette is wonderful!\r\n\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/88/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 769520939, "node_id": "MDU6SXNzdWU3Njk1MjA5Mzk=", "number": 1149, "title": "Make it easier to theme Datasette with CSS", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": {"value": 3268330, "label": "Datasette 1.0"}, "comments": 3, "created_at": "2020-12-17T05:01:26Z", "updated_at": "2021-03-22T21:43:16Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "I want to theme https://datasette.io/ so that when you visit https://datasette.io/content (the Datasette UI part of it) the navigation from the parent site is used.\r\n\r\nI tried dropping in a `base.html` template like this:\r\n\r\n```html\r\n{% extends \"page_base.html\" %}\r\n\r\n{% block base_extra_head %}\r\n\r\n<meta name=\"viewport\" content=\"width=device-width, initial-scale=1, shrink-to-fit=no\">\r\n{% for url in extra_css_urls %}\r\n    <link rel=\"stylesheet\" href=\"{{ url.url }}\"{% if url.sri %} integrity=\"{{ url.sri }}\" crossorigin=\"anonymous\"{% endif %}>\r\n{% endfor %}\r\n{% for url in extra_js_urls %}\r\n    <script src=\"{{ url.url }}\"{% if url.sri %} integrity=\"{{ url.sri }}\" crossorigin=\"anonymous\"{% endif %}></script>\r\n{% endfor %}\r\n{% block extra_head %}{% endblock %}\r\n{% endblock %}\r\n\r\n{% block extra_body_end %}\r\n{% include \"_close_open_menus.html\" %}\r\n\r\n{% for body_script in body_scripts %}\r\n    <script>{{ body_script }}</script>\r\n{% endfor %}\r\n{% endblock %}\r\n```\r\nBut this resulted in pages looking like this:\r\n\r\n<img width=\"1067\" alt=\"content__categories__3_rows\" src=\"https://user-images.githubusercontent.com/9599/102446045-c168e280-3fe1-11eb-94d6-e7350798eb96.png\">\r\n\r\nNote that the cog menu is broken and the filter UI is unstyled. To get these working correctly I would need to copy over a whole lot of Datasette's default CSS - and that means that when Datasette changes in the future those pages could break in subtle ways.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1149/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 837348479, "node_id": "MDU6SXNzdWU4MzczNDg0Nzk=", "number": 1269, "title": "Don't attempt to run count(*) against virtual tables", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2021-03-22T05:57:43Z", "updated_at": "2021-03-22T17:40:42Z", "closed_at": "2021-03-22T17:40:41Z", "author_association": "OWNER", "pull_request": null, "body": "Counting the rows in a virtual table doesn't seem very interesting to me, and it's the cause of at least one crashing bug with SpatiaLite 5.0 on Linux, see https://github.com/simonw/datasette/issues/1268", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1269/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 837208901, "node_id": "MDU6SXNzdWU4MzcyMDg5MDE=", "number": 1267, "title": "Update Datasette alternativeto listening with details", "user": {"value": 921217, "label": "RayBB"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2021-03-21T23:20:20Z", "updated_at": "2021-03-22T04:37:26Z", "closed_at": "2021-03-22T04:37:26Z", "author_association": "NONE", "pull_request": null, "body": "Hello,\r\n\r\nI recently learned about Datasette from an old hackernews post. It seems like an awesome project and I actually have use case I might be trying out in the coming months.\r\n\r\nAlas, to get a better understanding of your project I looked it up on alternativeto to see what it is similar too. I promise it's not spam, it's reputable enough to have a [Wikipedia](https://en.wikipedia.org/wiki/AlternativeTo) page. There was no listing on the website so I went ahead and created a listing that is now approved.\r\n\r\nI encourage anyone who likes this project and hopes to spread the word to help update the listing by:\r\n1. Adding to the list of software it compares to\r\n2. Uploading screenshots\r\n3. Writing a review\r\n4. Adding \"features\"\r\n\r\nI know this may seem spammy but I promise I have no affiliation with alternativeto I'm just a happy user and know it's a popular site for discovering software.\r\n\r\nHere is the listing for datasette:\r\nhttps://alternativeto.net/software/datasette/about/\r\n\r\nCheers", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1267/reactions\", \"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 830567275, "node_id": "MDU6SXNzdWU4MzA1NjcyNzU=", "number": 1259, "title": "Research using CTEs for faster facet counts", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 5, "created_at": "2021-03-12T22:19:49Z", "updated_at": "2021-03-21T22:55:31Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "https://www.sqlite.org/changes.html#version_3_35_0\r\n\r\n> Add support for the [MATERIALIZED](https://www.sqlite.org/lang_with.html#mathint) and [NOT MATERIALIZED](https://www.sqlite.org/lang_with.html#mathint) hints when specifying [common table expressions](https://www.sqlite.org/lang_with.html). The default behavior was formerly NOT MATERIALIZED, but is now changed to MATERIALIZED for CTEs that are used more than once.\r\n\r\nIf a CTE creates a table that is used multiple time in that query, SQLite will now default to creating a materialized table for the duration of that query.\r\n\r\nThis could be a big performance boost when applying faceting multiple times against the same query. Consider this example query:\r\n```sql\r\nWITH data as (\r\n  select\r\n    *\r\n  from\r\n    [global-power-plants]\r\n),\r\ncountry_long as (select \r\n  'country_long' as col, country_long as value, count(*) as c from data group by country_long\r\n  order by c desc limit 10\r\n),\r\nprimary_fuel as (\r\nselect\r\n  'primary_fuel' as col, primary_fuel as value, count(*) as c from data group by primary_fuel\r\n  order by c desc limit 10\r\n)\r\nselect * from primary_fuel union select * from country_long order by col, c desc\r\n```\r\nhttps://global-power-plants.datasettes.com/global-power-plants?sql=WITH+data+as+%28%0D%0A++select%0D%0A++++*%0D%0A++from%0D%0A++++%5Bglobal-power-plants%5D%0D%0A%29%2C%0D%0Acountry_long+as+%28select+%0D%0A++%27country_long%27+as+col%2C+country_long+as+value%2C+count%28*%29+as+c+from+data+group+by+country_long%0D%0A++order+by+c+desc+limit+10%0D%0A%29%2C%0D%0Aprimary_fuel+as+%28%0D%0Aselect%0D%0A++%27primary_fuel%27+as+col%2C+primary_fuel+as+value%2C+count%28*%29+as+c+from+data+group+by+primary_fuel%0D%0A++order+by+c+desc+limit+10%0D%0A%29%0D%0Aselect+*+from+primary_fuel+union+select+*+from+country_long+order+by+col%2C+c+desc\r\n\r\nOutputs:\r\n\r\ncol | value | c\r\n-- | -- | --\r\ncountry_long | United States of America | 8688\r\ncountry_long | China | 4235\r\ncountry_long | United Kingdom | 2603\r\ncountry_long | Brazil | 2360\r\ncountry_long | France | 2155\r\ncountry_long | India | 1590\r\ncountry_long | Germany | 1309\r\ncountry_long | Canada | 1159\r\ncountry_long | Spain | 829\r\ncountry_long | Russia | 545\r\nprimary_fuel | Solar | 9662\r\nprimary_fuel | Hydro | 7155\r\nprimary_fuel | Wind | 5188\r\nprimary_fuel | Gas | 3922\r\nprimary_fuel | Coal | 2390\r\nprimary_fuel | Oil | 2290\r\nprimary_fuel | Biomass | 1396\r\nprimary_fuel | Waste | 1087\r\nprimary_fuel | Nuclear | 198\r\nprimary_fuel | Geothermal | 189\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1259/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 832092321, "node_id": "MDU6SXNzdWU4MzIwOTIzMjE=", "number": 1261, "title": "Some links aren't properly URL encoded.", "user": {"value": 812795, "label": "brimstone"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2021-03-15T18:43:59Z", "updated_at": "2021-03-21T02:06:44Z", "closed_at": "2021-03-20T21:36:06Z", "author_association": "NONE", "pull_request": null, "body": "It seems like a percent sign in the query causes some links to end invalid.\r\n\r\nThe json and CSV links on this page don't behave like expected: https://honeypot-brimston3.vercel.app/honeypot?sql=select+time%2C+count%28time%29+as+count+from+%28select+strftime%28%22%25Y-%25m-%25d%22%2C+_etime%29+as+time+from+ssh+%29+group+by+time+order+by+time%3B\r\n\r\nI can take a swing at trying to fix this, but my python isn't strong and I need a pointer at the right approach and files to change.\r\n\r\nThanks!", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1261/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 836273891, "node_id": "MDU6SXNzdWU4MzYyNzM4OTE=", "number": 1266, "title": "Documentation for Response.asgi_send(send) method", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2021-03-19T18:52:49Z", "updated_at": "2021-03-20T21:35:00Z", "closed_at": "2021-03-20T21:32:28Z", "author_association": "OWNER", "pull_request": null, "body": "I found myself wanting to use this method for https://github.com/simonw/datasette-auth-passwords/issues/15 - but it's not documented. It should be documented.\r\n\r\nhttps://github.com/simonw/datasette/blob/8e18c7943181f228ce5ebcea48deb59ce50bee1f/datasette/utils/asgi.py#L320-L340", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1266/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 836123030, "node_id": "MDU6SXNzdWU4MzYxMjMwMzA=", "number": 1265, "title": "Support for HTTP Basic Authentication", "user": {"value": 468612, "label": "yunzheng"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2021-03-19T15:31:09Z", "updated_at": "2021-03-19T22:05:12Z", "closed_at": "2021-03-19T21:03:09Z", "author_association": "NONE", "pull_request": null, "body": "It would be nice if datasette could support [HTTP Basic Authentication](https://en.wikipedia.org/wiki/Basic_access_authentication).\r\n\r\nFor now I could ofcourse leverage Nginx for basic authentication, but it would be nice to have support for this in datasette by default or via a plugin like datasette-auth-github.\r\n\r\nMy main usecase is to put the whole datasette instance behind a username/password prompt via Basic Auth and not specific urls.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1265/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 834602299, "node_id": "MDU6SXNzdWU4MzQ2MDIyOTk=", "number": 1262, "title": "Plugin hook that could support 'order by random()' for table view", "user": {"value": 19328961, "label": "henry501"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2021-03-18T10:02:01Z", "updated_at": "2021-03-18T17:55:01Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "I am frequently using Datasette to quickly get a visual impression for a table without reviewing it in its entirety. Because I have some groups of similar records, the default sorting options mean that each page is very similar and not representative of the full dataset. The current interface allows sorting by columns, but random sorting is only available via custom SQL.\r\n\r\nMaybe this could be a button or link.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1262/reactions\", \"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 787173276, "node_id": "MDU6SXNzdWU3ODcxNzMyNzY=", "number": 1193, "title": "Research plugin hook for alternative database backends", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2021-01-15T20:27:50Z", "updated_at": "2021-03-12T01:01:54Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "I started exploring what Datasette would like running against PostgreSQL in #670 and @dazzag24 did some work on Parquet described in #657.\r\n\r\nI had initially thought this was WAY too much additional complexity, but I'm beginning to think that the `Database` class may be small enough that having it abstract away the details of running queries against alternative database backends could be feasible.\r\n\r\nA bigger issue is SQL generation, but I realized that most of Datasette's SQL generation code exists just in the `TableView` class that runs the table page. If this was abstracted into some kind of SQL builder that could be then customized per-database it might be reasonable to get it working.\r\n\r\nVery unlikely for this to make it into Datasette 1.0, but maybe this would be the defining feature of Datasette 2.0?", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1193/reactions\", \"total_count\": 3, \"+1\": 3, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 824067604, "node_id": "MDU6SXNzdWU4MjQwNjc2MDQ=", "number": 1250, "title": "Research: Plugin hook for alternative database connections", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2021-03-08T00:28:15Z", "updated_at": "2021-03-12T01:01:25Z", "closed_at": "2021-03-12T01:01:17Z", "author_association": "OWNER", "pull_request": null, "body": "The `Database` class is a natural looking fit for a plugin hook to load custom database connections... potentially even databases other than SQLite. DuckDB (refs #968) could make for a great starting point, since it looks very compatible with the existing SQLite code.\r\n\r\nThe real win would be if this could lead to running Datasette against PostgreSQL. I made some initial explorations in that direction a while ago in #670.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1250/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 794554881, "node_id": "MDU6SXNzdWU3OTQ1NTQ4ODE=", "number": 1208, "title": "A lot of open(file) functions are used without a context manager thus producing ResourceWarning: unclosed file <_io.TextIOWrapper", "user": {"value": 4488943, "label": "kbaikov"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2021-01-26T20:56:28Z", "updated_at": "2021-03-11T16:15:49Z", "closed_at": "2021-03-11T16:15:49Z", "author_association": "CONTRIBUTOR", "pull_request": null, "body": "Your code is full of open files that are never closed, especially when you deal with reading/writing json/yaml files.\r\n\r\nIf you run python with warnings enabled this problem becomes evident.\r\nThis probably contributes to some memory leaks in long running datasettes if the GC will not 'collect' those resources properly.\r\n\r\nThis is easily fixed by using a context manager instead of just using open:\r\n```python\r\nwith open('some_file', 'w') as opened_file:\r\n    opened_file.write('string')\r\n```\r\n\r\nIn some newer parts of the code you use Path objects 'read_text' and 'write_text' functions which close the file properly and are prefered in some cases.\r\n\r\n\r\nIf you want I can create a PR for all places i found this pattern in.\r\n\r\n\r\nBellow is a fraction of places where i found a ResourceWarning:\r\n```python\r\n\r\nupdate-docs-help.py:\r\n  20          actual = actual.replace(\"Usage: cli \", \"Usage: datasette \")\r\n  21:         open(docs_path / filename, \"w\").write(actual)\r\n  22  \r\n\r\ndatasette\\app.py:\r\n  210          ):\r\n  211:             inspect_data = json.load((config_dir / \"inspect-data.json\").open())\r\n  212              if immutables is None:\r\n\r\n  266          if config_dir and (config_dir / \"settings.json\").exists() and not config:\r\n  267:             config = json.load((config_dir / \"settings.json\").open())\r\n  268          self._settings = dict(DEFAULT_SETTINGS, **(config or {}))\r\n\r\n  445              self._app_css_hash = hashlib.sha1(\r\n  446:                 open(os.path.join(str(app_root), \"datasette/static/app.css\"))\r\n  447                  .read()\r\n\r\ndatasette\\cli.py:\r\n  130      else:\r\n  131:         out = open(inspect_file, \"w\")\r\n  132      loop = asyncio.get_event_loop()\r\n\r\n  459      if inspect_file:\r\n  460:         inspect_data = json.load(open(inspect_file))\r\n  461  \r\n\r\n```\r\n\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1208/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 824750134, "node_id": "MDU6SXNzdWU4MjQ3NTAxMzQ=", "number": 1251, "title": "facet option not appearing when table is big", "user": {"value": 15836677, "label": "verajosemanuel"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2021-03-08T16:54:04Z", "updated_at": "2021-03-08T16:54:16Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "I have a big table with more than 500.000 rows.\r\nTrying to facet by one of my columns, the options are not available as for the other smaller tables.\r\n\r\nI have tried to set it in URL as:\r\n\r\n`&_facet=city_id`\r\n\r\nto no avail.\r\n\r\nis there any limit?  how can I force the option \"facet\" to appear for big tables?\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1251/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 823035080, "node_id": "MDU6SXNzdWU4MjMwMzUwODA=", "number": 1248, "title": "duckdb database (very low performance in SQLite)", "user": {"value": 15836677, "label": "verajosemanuel"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2021-03-05T12:20:29Z", "updated_at": "2021-03-08T00:25:27Z", "closed_at": "2021-03-08T00:25:27Z", "author_association": "NONE", "pull_request": null, "body": "My sqlite is getting too big to be processed by datasette (more than 10 minutes waiting to load) so I am working with duckdb and is waaaaay faster. I think the fastest embeddable database actually.\r\n\r\nhttps://duckdb.org/\r\n\r\nTaking into account DuckDb is SQLite based it would be GREAT to use it with datasette.\r\n\r\nis that possible?\r\n\r\nRegards and thanks for a superb job", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1248/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 617323873, "node_id": "MDU6SXNzdWU2MTczMjM4NzM=", "number": 766, "title": "Enable wildcard-searches by default", "user": {"value": 2181410, "label": "clausjuhl"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2020-05-13T10:14:48Z", "updated_at": "2021-03-05T16:35:21Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "Hi Simon.\r\n\r\nIt seems that datasette currently has wildcard-searches disabled by default (along with the boolean search-options, NEAR-queries and more, and despite the docs). If I try out the search-url provided in the [docs](https://datasette.readthedocs.io/en/stable/full_text_search.html#the-table-page-and-table-view-api) (https://fara.datasettes.com/fara/FARA_All_ShortForms?_search=manafort), it does not handle wildcard-searches, and I'm unable to make it work on my datasette-instance.\r\n\r\nI would argue that wildcard-searches is such a standard query, that it should be enabled by default. Requiring \"_searchmode=raw\" when using prefix-searches seems unnecessary. Plus: What happens to non-ascii searches when using \"_searchmode=raw\"? Is the \"escape_fts\"-function from datasette.utils ignored?\r\n\r\n\r\nThanks!\r\n\r\n/Claus", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/766/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 818430405, "node_id": "MDU6SXNzdWU4MTg0MzA0MDU=", "number": 1247, "title": "datasette.add_memory_database() method", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2021-03-01T03:48:38Z", "updated_at": "2021-03-01T04:02:26Z", "closed_at": "2021-03-01T04:02:26Z", "author_association": "OWNER", "pull_request": null, "body": "I just wrote this code:\r\n\r\nhttps://github.com/simonw/datasette/blob/47eb885cc2c3aafa03645c330c6f597bee9b3b25/tests/test_facets.py#L334-L335\r\n\r\nIt would be nice if you didn't have to separately instantiate a database object here.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1247/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 817597268, "node_id": "MDU6SXNzdWU4MTc1OTcyNjg=", "number": 1246, "title": "Suggest for ArrayFacet possibly confused by blank values", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2021-02-26T19:11:52Z", "updated_at": "2021-03-01T03:46:11Z", "closed_at": "2021-03-01T03:46:11Z", "author_association": "OWNER", "pull_request": null, "body": "I sometimes don't get the suggestion for facet-by-array for columns that contain arrays. I think it may be because they have empty spaces in them - or perhaps it's because the null detection doesn't actually work.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1246/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 718259202, "node_id": "MDU6SXNzdWU3MTgyNTkyMDI=", "number": 1005, "title": "Remove xfail tests when new httpx is released", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 3268330, "label": "Datasette 1.0"}, "comments": 3, "created_at": "2020-10-09T16:00:19Z", "updated_at": "2021-02-28T22:41:08Z", "closed_at": "2021-02-28T22:41:08Z", "author_association": "OWNER", "pull_request": null, "body": "> My `httpx` pull request adding `raw_path` support was just merged: https://github.com/encode/httpx/pull/1357 - but it's not in a release yet.\r\n>\r\n> I'm going to mark these tests as `xfail` so I can land this change - I'll remove that once an `httpx` release comes out that I can use to get the tests passing.\r\n>\r\n_Originally posted by @simonw in https://github.com/simonw/datasette/pull/1000#issuecomment-706263157_", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1005/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 814591962, "node_id": "MDU6SXNzdWU4MTQ1OTE5NjI=", "number": 1240, "title": "Allow facetting on custom queries", "user": {"value": 7107523, "label": "Kabouik"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2021-02-23T15:52:19Z", "updated_at": "2021-02-26T18:19:46Z", "closed_at": "2021-02-26T18:18:18Z", "author_association": "NONE", "pull_request": null, "body": "Facets are a tremendously useful feature, especially for people peeking at the database for the first time and still having little knowledge about the details of the data. It is of great assistance to discover interesting features to explore futher in advanced queries.\r\n\r\nYet, it seems it's impossible to use facets when running a custom SQL query, be it from the little gear icons in column names, the facet suggestions at the top (hidden when performing a custom query), or by appending a facet code to the URL. \r\n\r\nIs there a technical limitation, or is this something that could be unlocked easily?", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1240/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 817528452, "node_id": "MDU6SXNzdWU4MTc1Mjg0NTI=", "number": 1244, "title": "Plugin tip: look at the examples linked from the hooks page", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2021-02-26T17:18:27Z", "updated_at": "2021-02-26T17:30:38Z", "closed_at": "2021-02-26T17:27:15Z", "author_association": "OWNER", "pull_request": null, "body": "Someone asked \"what are good example plugins I can look at?\" and I realized that the answer is to look through the example links on https://docs.datasette.io/en/stable/plugin_hooks.html - but that tip should be written down somewhere on the https://docs.datasette.io/en/stable/writing_plugins.html page.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1244/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 803356942, "node_id": "MDU6SXNzdWU4MDMzNTY5NDI=", "number": 1218, "title": " /usr/local/opt/python3/bin/python3.6: bad interpreter: No such file or directory", "user": {"value": 11855322, "label": "robmarkcole"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2021-02-08T09:07:00Z", "updated_at": "2021-02-23T12:12:17Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "Error as above, however I do have python3.8 and the readme indicates this is supported.\r\n\r\n```\r\n(venv) (base) Robins-MacBook:datasette robin$ ls /usr/local/opt/python3/bin/\r\n\r\n.. pip3 python3 python3.8\r\n```", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1218/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 813978858, "node_id": "MDU6SXNzdWU4MTM5Nzg4NTg=", "number": 1239, "title": "JSON filter fails if column contains spaces", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2021-02-23T00:18:07Z", "updated_at": "2021-02-23T00:22:53Z", "closed_at": "2021-02-23T00:22:53Z", "author_association": "OWNER", "pull_request": null, "body": "Got this exception:\r\n\r\n`ERROR: conn=<sqlite3.Connection object at 0x10ea68e40>, sql = 'select Address, Affiliation, County, [Has Report], [Latest report notes], [Latest report yes], Latitude, [Location Type], Longitude, Name, id, [Appointment scheduling instructions], [Availability Info], [Latest report] from locations where rowid in (\\n            select locations.rowid from locations, json_each(locations.Availability Info) j\\n            where j.value = :p0\\n        ) and \"Latest report yes\" = :p1 order by id limit 101', params = {'p0': 'Yes: appointment required', 'p1': '1'}: near \"Info\": syntax error`", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1239/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 811505638, "node_id": "MDU6SXNzdWU4MTE1MDU2Mzg=", "number": 1234, "title": "Runtime support for ATTACHing multiple databases", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2021-02-18T22:06:47Z", "updated_at": "2021-02-22T21:06:28Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "> The implementation in #1232 is ready to land. It's the simplest-thing-that-could-possibly-work: you can run `datasette one.db two.db three.db --crossdb` and then use the `/_memory` page to run joins across tables from multiple databases.\r\n>\r\n> It only works on the first 10 databases that were passed to the command-line. This means that if you have a Datasette instance with hundreds of attached databases (see [Datasette Library](https://github.com/simonw/datasette/issues/417)) this won't be particularly useful for you.\r\n>\r\n> So... a better, future version of this feature would be one that lets you join across databases on command - maybe by hitting `/_memory?attach=db1&attach=db2` to get a special connection.\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/datasette/issues/283#issuecomment-781665560_", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1234/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 797651831, "node_id": "MDU6SXNzdWU3OTc2NTE4MzE=", "number": 1212, "title": "Tests are very slow. ", "user": {"value": 4488943, "label": "kbaikov"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2021-01-31T08:06:16Z", "updated_at": "2021-02-19T22:54:13Z", "closed_at": "2021-02-19T22:54:13Z", "author_association": "CONTRIBUTOR", "pull_request": null, "body": "Working on my PR i noticed that tests are very slow.\r\n\r\nThe plain pytest run took about 37 minutes for me.\r\nHowever i could shave of about 10 minutes from that if i used pytest-xdist to parallelize execution.\r\n`pytest -n 8` is run only in 28 minutes on my machine.\r\n\r\nI can create a PR to mention that in your documentation.\r\nThis will be a simple change to add pytest-xdist to requirements and change a command to run pytest in documentation.\r\n\r\nDoes that make sense to you?\r\n\r\nAfter a bit more investigation it looks like python-xdist is not an answer. It creates a race condition for tests that try to clead temp dir before run.\r\n\r\nProfiling shows that most time is spent on conn.executescript(TABLES) in make_app_client function. Which makes sense.\r\n\r\nPerhaps the better approach would be look at the app_client fixture which is already session scoped, but not used by all test cases.\r\nAnd/or use conn = sqlite3.connect(\":memory:\") which is much faster.\r\nAnd/or truncate tables after each TC instead of deleting the file and re-creating them.\r\n\r\nI can take a look which is the best approach if you give the go-ahead. ", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1212/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 811589344, "node_id": "MDU6SXNzdWU4MTE1ODkzNDQ=", "number": 1235, "title": "Upgrade Python version used by official Datasette Docker image", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2021-02-19T00:47:40Z", "updated_at": "2021-02-19T01:48:31Z", "closed_at": "2021-02-19T01:48:30Z", "author_association": "OWNER", "pull_request": null, "body": "Currently uses 3.7.2:\r\n\r\nhttps://github.com/simonw/datasette/blob/73bed175631a79e13a521eee82f8451dd0477eb3/Dockerfile#L1\r\n\r\nThere's a security fix for Python which it would be good to ship in this image (even though I'm reasonably confident it doesn't affect Datasette): https://bugs.python.org/issue42938", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1235/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 811458446, "node_id": "MDU6SXNzdWU4MTE0NTg0NDY=", "number": 1233, "title": "\"datasette publish cloudrun\" cannot publish files with spaces in their name", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2021-02-18T21:08:31Z", "updated_at": "2021-02-18T21:10:08Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "Got this error:\r\n```\r\nStep 6/9 : RUN datasette inspect fixtures.db extra database.db --inspect-file inspect-data.json\r\n ---> Running in db9da0068592\r\nUsage: datasette inspect [OPTIONS] [FILES]...\r\nTry 'datasette inspect --help' for help.\r\n\r\nError: Invalid value for '[FILES]...': Path 'extra' does not exist.\r\nThe command '/bin/sh -c datasette inspect fixtures.db extra database.db --inspect-file inspect-data.json' returned a non-zero code: 2\r\nERROR\r\nERROR: build step 0 \"gcr.io/cloud-builders/docker\" failed: step exited with non-zero status: 2\r\n```\r\nWhile working on the demo for #1232, using this deploy command:\r\n```\r\nGITHUB_SHA=crossdb datasette publish cloudrun fixtures.db 'extra database.db' \\\r\n            -m fixtures.json \\\r\n            --plugins-dir=plugins \\\r\n            --branch=$GITHUB_SHA \\\r\n            --version-note=$GITHUB_SHA \\\r\n            --extra-options=\"--setting template_debug 1 --crossdb\" \\\r\n            --install=pysqlite3-binary \\\r\n            --service=datasette-latest-crossdb\r\n```", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1233/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 808843401, "node_id": "MDU6SXNzdWU4MDg4NDM0MDE=", "number": 1226, "title": "--port option should validate port is between 0 and 65535", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2021-02-15T22:01:33Z", "updated_at": "2021-02-18T18:41:27Z", "closed_at": "2021-02-18T18:41:27Z", "author_association": "OWNER", "pull_request": null, "body": "Currently throws an ugly error message:\r\n```\r\n(datasette-graphql) datasette-graphql % datasette fivethirtyeight.db -p 80094\r\nINFO:     Started server process [45497]\r\nINFO:     Waiting for application startup.\r\nINFO:     Application startup complete.\r\nTraceback (most recent call last):\r\n  File \"/Users/simon/.local/share/virtualenvs/datasette-graphql-n1OSJCS8/bin/datasette\", line 8, in <module>\r\n    sys.exit(cli())\r\n...\r\n    server = await loop.create_server(\r\n  File \"/Users/simon/.pyenv/versions/3.8.2/lib/python3.8/asyncio/base_events.py\", line 1461, in create_server\r\n    sock.bind(sa)\r\nOverflowError: bind(): port must be 0-65535.\r\n```", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1226/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 811054000, "node_id": "MDU6SXNzdWU4MTEwNTQwMDA=", "number": 1230, "title": "Vega charts are plotted only for rows on the visible page, cluster maps only for rows in the remaining pages", "user": {"value": 7107523, "label": "Kabouik"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2021-02-18T12:27:02Z", "updated_at": "2021-02-18T15:22:15Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "I filtered a data set on some criteria and obtain 265 results, split over three pages (100, 100, 65), and reazlized that Vega plots are only applied to the results displayed on the current page, instead of the whole filtered data, _e.g._, 100 on page 1, 100 on page 2, 65 on page 3. Is there a way to force the graphs to consider all results instead of just the page, considering that pages rarely represent sensible information?\r\n\r\nLikewise, while the cluster map does show all results on the first page, if you go to next pages, it will show all remaining results except the previous page(s), _e.g._, 265 on page 1, 165 on page 2, 65 on page 3.\r\n\r\nIn both cases, I don't see many situations where one would like to represent the data this way, and it might even lead to interpretation errors when viewing the data. Am I missing some cases where this would be best? Perhaps a clickable option to subset visual representations according visible pages _vs._ display all search results would do?\r\n\r\n[Edit] Oh, I just saw the \"Load all\" button under the cluster map as well as the [setting to alter the max number or results](https://docs.datasette.io/en/stable/settings.html#max-returned-rows). So I guess this issue only is about the Vega charts.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": null, "draft": null, "state_reason": null}
{"id": 808771690, "node_id": "MDU6SXNzdWU4MDg3NzE2OTA=", "number": 1225, "title": "More flexible formatting of records with CSS grid", "user": {"value": 649467, "label": "mhalle"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2021-02-15T19:28:17Z", "updated_at": "2021-02-15T19:28:35Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "In several applications I've been experimenting with alternate formatting of datasette query results. Lately I've found that CSS grids work very well and seem quite general for formatting rows. In CSS I use grid templates to define the layout of each record and the regions for each field, hiding the fields I don't want. It's pretty flexible and looks good. It's also a great basis for highly responsive layout.\r\n\r\nI initially thought I'd only use this feature for record detail views, but now I use it for index views as well. \r\n\r\nHowever, there are some limitations:\r\n* With the existing table templates, it seems that you can change the `display` property on the enclosing `table`, `tbody`, and `tr` to make them be grid-like, but that seems hacky (convert `table` and `tbody` to be `display: block` and `tr` to be `display: grid`).\r\n* More significantly, it's very nice to have the column name available when rendering each record to display headers/field labels. The existing templates don't do that, so a custom `_table` template is necessary.\r\n* I don't know if any plugins are sensitive to whether data is rendered as a table or not since I'm not completely clear how plugins get their data. \r\n* Regardless, you need custom CSS to take full advantage of grids. I don't have a proposal on how to integrate them more deeply.\r\n\r\nIt would be helpful to at least have an official example or test that used a grid layout for records to make sure nothing in datasette breaks with it. ", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1225/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 806743116, "node_id": "MDU6SXNzdWU4MDY3NDMxMTY=", "number": 1220, "title": "Installing datasette via docker: Path 'fixtures.db' does not exist", "user": {"value": 30607, "label": "aborruso"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2021-02-11T21:09:14Z", "updated_at": "2021-02-12T21:35:17Z", "closed_at": "2021-02-12T21:35:17Z", "author_association": "NONE", "pull_request": null, "body": "Hi,\r\nIf I run\r\n\r\n```\r\ndocker run -p 8001:8001 -v `pwd`:/mnt \\                                                                         1 \u21b5\r\n    datasetteproject/datasette \\\r\n    datasette -p 8001 -h 0.0.0.0 fixtures.db\r\n```\r\n\r\nI have \r\n\r\n```\r\nError: Invalid value for '[FILES]...': Path 'fixtures.db' does not exist.\r\n```\r\n\r\nIf I run `test -f fixtures.db && echo \"it exists.\"` I have `it exists.`.\r\n\r\nWhat's my error?\r\n\r\nThank you", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1220/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 792890765, "node_id": "MDU6SXNzdWU3OTI4OTA3NjU=", "number": 1200, "title": "?_size=10 option for the arbitrary query page would be useful", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2021-01-24T20:55:35Z", "updated_at": "2021-02-11T03:13:59Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "https://latest.datasette.io/fixtures?sql=select+*+from+compound_three_primary_keys&_size=10 - `_size=10` does not do anything at the moment. It would be useful if it did.\r\n\r\nWould also be good if it persisted in a hidden form field.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1200/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 803929694, "node_id": "MDU6SXNzdWU4MDM5Mjk2OTQ=", "number": 1219, "title": "Try profiling Datasette using scalene", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2021-02-08T20:37:06Z", "updated_at": "2021-02-08T22:13:00Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "https://github.com/emeryberger/scalene looks like an interesting profiling tool.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1219/reactions\", \"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 796234313, "node_id": "MDU6SXNzdWU3OTYyMzQzMTM=", "number": 1210, "title": "Immutable Database w/ Canned Queries", "user": {"value": 525780, "label": "heyarne"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2021-01-28T18:08:29Z", "updated_at": "2021-02-05T11:30:34Z", "closed_at": "2021-02-05T11:30:34Z", "author_association": "NONE", "pull_request": null, "body": "I have a database that I only want to read from; when instructing datasette to treat the database as immutable my defined canned queries disappear. Are these two features incompatible or have I hit an unintended bug? Thanks for datasette in any way, it's a joy to use!", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1210/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 799693777, "node_id": "MDU6SXNzdWU3OTk2OTM3Nzc=", "number": 1214, "title": "Re-submitting filter form duplicates _x querystring arguments", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2021-02-02T21:13:35Z", "updated_at": "2021-02-02T21:28:53Z", "closed_at": "2021-02-02T21:21:13Z", "author_association": "OWNER", "pull_request": null, "body": "Really nasty bug, caused by #1194 fix in 07e163561592c743e4117f72102fcd350a600909\r\n\r\nNavigate to this page: https://github-to-sqlite.dogsheep.net/github/labels?_search=help&_sort=id\r\n\r\nClick \"Apply\" to submit the form and the resulting URL is https://github-to-sqlite.dogsheep.net/github/labels?_search=help&_sort=id&_search=help&_sort=id\r\n\r\nThat's because the (truncated) HTML for the form looks like this:\r\n\r\n```html\r\n    ... <input id=\"_search\" type=\"search\" name=\"_search\" value=\"help\">\r\n    ...\r\n            <div class=\"select-wrapper small-screen-only\">\r\n                <select name=\"_sort\" id=\"sort_by\">\r\n                    <option value=\"\">Sort...</option>\r\n                            <option value=\"id\" selected>Sort by id</option>\r\n                            <option value=\"node_id\">Sort by node_id</option>\r\n                            ...\r\n                </select>\r\n            </div>\r\n            ...\r\n            <input type=\"hidden\" name=\"_search\" value=\"help\">\r\n            <input type=\"hidden\" name=\"_sort\" value=\"id\">\r\n        <input type=\"submit\" value=\"Apply\">\r\n```", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1214/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 799663959, "node_id": "MDU6SXNzdWU3OTk2NjM5NTk=", "number": 1213, "title": "gzip support for HTML (and JSON) responses", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2021-02-02T20:36:28Z", "updated_at": "2021-02-02T20:41:55Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "This page https://datasette-tiles-demo.datasette.io/San_Francisco/tiles is 2MB because of all of the base64 images. Gzipped it's 1.5MB.\r\n\r\nSince Datasette is usually deployed without a frontend gzipping proxy, Datasette itself needs to solve for this.\r\n\r\nGzipping everything won't work because some endpoints - the all-rows CSV endpoint and the download-database endpoint - are streaming and hence can't be buffered-and-gzipped.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1213/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 793881756, "node_id": "MDU6SXNzdWU3OTM4ODE3NTY=", "number": 1207, "title": "Document the Datasette(..., pdb=True) testing pattern", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2021-01-26T02:48:10Z", "updated_at": "2021-01-29T02:37:19Z", "closed_at": "2021-01-29T02:12:34Z", "author_association": "OWNER", "pull_request": null, "body": "If you're writing tests for a Datasette plugin and you get a 500 error from inside Datasette, you can cause Datasette to open a PDB session within the application server code by doing this:\r\n\r\n```python\r\nds = Datasette([db_path], pdb=True)\r\nresponse = await ds.client.get(\"/\")\r\n```\r\n\r\nYou'll need to run `pytest -s` to interact with the debugger, otherwise you'll get an error.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1207/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 795367402, "node_id": "MDU6SXNzdWU3OTUzNjc0MDI=", "number": 1209, "title": "v0.54 500 error from sql query in custom template; code worked in v0.53; found a workaround", "user": {"value": 11788561, "label": "jrdmb"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2021-01-27T19:08:13Z", "updated_at": "2021-01-28T23:00:27Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "v0.54 500 error in sql query template; code worked in v0.53; found a workaround\r\n\r\n**schema:**   \r\nCREATE TABLE \"talks\" (\"talk\" TEXT,\"series\" INTEGER, \"talkdate\" TEXT)  \r\nCREATE TABLE \"series\" (\"id\" INTEGER PRIMARY KEY, \"series\" TEXT, talks_list TEXT default '', website TEXT default '');\r\n\r\n**Live example of correctly rendered template in v.053:** https://cosmotalks-cy6xkkbezq-uw.a.run.app/cosmotalks/talks/1\r\n\r\n**Description of problem:** I needed 'sql select' code in a custom row-mydatabase-mytable.html template to lookup the series name for a foreign key integer value in the talks table.  So `metadata.json` specifies the `datasette-template-sql` plugin.\r\n\r\nThe code below worked perfectly in v0.53 (just the relevant sql statement part is shown; full code is [here](https://github.com/jrdmb/cosmotalks-datasette/blob/main/templates/row-cosmotalks-talks.html)):\r\n\r\n```\r\n{# custom addition #}  \r\n{% for row in display_rows %}  \r\n    ...  \r\n    {% set sname = sql(\"select series from series where id = ?\", [row.series]) %}  \r\n    <strong>Series name: {{ sname[0].series }}  \r\n    ...  \r\n{% endfor %}  \r\n{# End of custom addition #}  \r\n```\r\n\r\n**In v0.54, that code resulted in a 500 error with a 'no such table series' message.**  A second query in that template also did not work but the above is fully illustrative of the problem.\r\n\r\nAll templates were up-to-date along with datasette v0.54.\r\n\r\n**Workaround:** After fiddling around with trying different things, what worked was the syntax from [Querying a different database from the datasette-template-sql github repo](https://github.com/simonw/datasette-template-sql#querying-a-different-database) to add the database name to the sql statement:\r\n\r\n`{% set sname = sql(\"select series from series where id = ?\", [row.series], database=\"cosmotalks\") %}`\r\n\r\nThough this was found to work, it should not be necessary to add `database=\"cosmotalks\"` since per the `datasette-template-sql` README, it's only needed when querying a different database, but here it's a table within the same database.\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": null, "draft": null, "state_reason": null}
{"id": 793027837, "node_id": "MDU6SXNzdWU3OTMwMjc4Mzc=", "number": 1205, "title": "Rename /:memory: to /_memory", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 3268330, "label": "Datasette 1.0"}, "comments": 3, "created_at": "2021-01-25T05:04:56Z", "updated_at": "2021-01-28T22:55:02Z", "closed_at": "2021-01-28T22:51:42Z", "author_association": "OWNER", "pull_request": null, "body": "For consistency with `/_internal` - and because then we don't need to escape the `:` characters.\r\n\r\nThis change would need to be in before Datasette 1.0. I could land it earlier and set up redirects from the old URLs though.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1205/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 770448622, "node_id": "MDU6SXNzdWU3NzA0NDg2MjI=", "number": 1151, "title": "Database class mechanism for cross-connection in-memory databases", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 6346396, "label": "Datasette 0.54"}, "comments": 11, "created_at": "2020-12-17T23:25:43Z", "updated_at": "2021-01-26T19:07:44Z", "closed_at": "2020-12-18T01:01:26Z", "author_association": "OWNER", "pull_request": null, "body": "> Next challenge: figure out how to use the `Database` class from https://github.com/simonw/datasette/blob/0.53/datasette/database.py for an in-memory database which persists data for the duration of the lifetime of the server, and allows access to that in-memory database from multiple threads in a way that lets them see each other's changes.\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1150#issuecomment-747768112_", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1151/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 714377268, "node_id": "MDU6SXNzdWU3MTQzNzcyNjg=", "number": 991, "title": "Redesign application homepage", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 7, "created_at": "2020-10-04T18:48:45Z", "updated_at": "2021-01-26T19:06:36Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "Most Datasette instances only host a single database, but the current homepage design assumes that it should leave plenty of space for multiple databases:\r\n\r\n<img width=\"878\" alt=\"Datasette_Fixtures__fixtures\" src=\"https://user-images.githubusercontent.com/9599/95024344-5b51fd80-0637-11eb-8a11-40bad16f6907.png\">\r\n\r\nReconsider this design - should the default show more information?\r\n\r\nThe Covid-19 Datasette homepage looks particularly sparse I think: https://covid-19.datasettes.com/\r\n\r\n<img width=\"782\" alt=\"COVID-19_cases__using_data_from_Johns_Hopkins_CSSE__the_New_York_Times_and_the_LA_Times__covid\" src=\"https://user-images.githubusercontent.com/9599/95024391-876d7e80-0637-11eb-8f19-ef38e4c87d2a.png\">", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/991/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 792904595, "node_id": "MDU6SXNzdWU3OTI5MDQ1OTU=", "number": 1201, "title": "Release notes for Datasette 0.54", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 6346396, "label": "Datasette 0.54"}, "comments": 5, "created_at": "2021-01-24T21:22:28Z", "updated_at": "2021-01-25T17:42:21Z", "closed_at": "2021-01-25T17:42:21Z", "author_association": "OWNER", "pull_request": null, "body": "These will incorporate the release notes from the alpha, much expanded: https://github.com/simonw/datasette/releases/tag/0.54a0", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1201/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 780278550, "node_id": "MDU6SXNzdWU3ODAyNzg1NTA=", "number": 1179, "title": "Make original path available to render hooks", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 8, "created_at": "2021-01-06T08:31:45Z", "updated_at": "2021-01-25T04:44:33Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "https://github.com/simonw/datasette-export-notebook/blob/0.1/datasette_export_notebook/__init__.py\r\n\r\n```python\r\nasync def render_notebook(datasette, request):\r\n    return Response.html(\r\n        await datasette.render_template(\r\n            \"export_notebook.html\",\r\n            {\r\n                \"csv_stream_url\": datasette.absolute_url(\r\n                    request,\r\n                    path_with_format(\r\n                        request=request, format=\"csv\", extra_qs={\"_stream\": \"on\"}\r\n                    ),\r\n                ),\r\n                \"json_url\": datasette.absolute_url(\r\n                    request,\r\n                    path_with_format(\r\n                        request=request, format=\"json\", extra_qs={\"_shape\": \"array\"}\r\n                    ),\r\n                ),\r\n                \"json\": json,\r\n            },\r\n        )\r\n    )\r\n```\r\nThis results in https://latest-with-plugins.datasette.io/github/issue_comments.Notebook showing `http://latest-with-plugins.datasette.io/github/issue_comments.Notebook?_format=json&_shape=array`", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1179/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 712260429, "node_id": "MDU6SXNzdWU3MTIyNjA0Mjk=", "number": 983, "title": "JavaScript plugin hooks mechanism similar to pluggy", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 47, "created_at": "2020-09-30T20:32:43Z", "updated_at": "2021-01-25T04:43:58Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "> It would be neat to provide a JavaScript plugin hook that plugins can use to add their own options to this menu. No idea what that would look like though.\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/datasette/issues/981#issuecomment-701616922_", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/983/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 789336592, "node_id": "MDU6SXNzdWU3ODkzMzY1OTI=", "number": 1195, "title": "view_name = \"query\" for the query page", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2021-01-19T20:21:36Z", "updated_at": "2021-01-25T04:40:08Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "It uses `view_name` of `database` at the moment which isn't as useful.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1195/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 712984738, "node_id": "MDU6SXNzdWU3MTI5ODQ3Mzg=", "number": 987, "title": "Documented HTML hooks for JavaScript plugin authors", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 7, "created_at": "2020-10-01T16:10:14Z", "updated_at": "2021-01-25T04:00:03Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "In #981 I added `data-column=` attributes to the `<th>` on the table page. These should become part of Datasette's documented API so JavaScript plugin authors can use them to derive things about the tables shown on a page (`datasette-cluster-map uses them as-of https://github.com/simonw/datasette-cluster-map/issues/18).", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/987/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 788447787, "node_id": "MDU6SXNzdWU3ODg0NDc3ODc=", "number": 1194, "title": "?_size= argument is not persisted by hidden form fields in the table filters", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 6346396, "label": "Datasette 0.54"}, "comments": 3, "created_at": "2021-01-18T17:41:52Z", "updated_at": "2021-01-25T03:10:23Z", "closed_at": "2021-01-25T03:10:23Z", "author_association": "OWNER", "pull_request": null, "body": "Click \"Apply\" on https://covid-19.datasettes.com/covid/ny_times_us_counties?_size=1000&county__exact=San+Francisco&state__exact=California&_sort_desc=date#g.mark=line&g.x_column=date&g.x_type=temporal&g.y_column=cases&g.y_type=quantitative and the `?_size=1000` parameter from the URL will no longer apply on the reloaded page.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1194/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 777145954, "node_id": "MDU6SXNzdWU3NzcxNDU5NTQ=", "number": 1167, "title": "Add Prettier to contributing documentation", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 6346396, "label": "Datasette 0.54"}, "comments": 3, "created_at": "2020-12-31T22:00:55Z", "updated_at": "2021-01-25T02:01:19Z", "closed_at": "2021-01-25T01:58:28Z", "author_association": "OWNER", "pull_request": null, "body": "Following #1166 - the docs at https://docs.datasette.io/en/stable/contributing.html should include a section about JavaScript, and it should document how to run Prettier.\r\n\r\nI run it in VS Code but it can be run on the command-line too:\r\n\r\n    npx prettier 'datasette/static/*[!.min].js' --write\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1167/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 771208009, "node_id": "MDU6SXNzdWU3NzEyMDgwMDk=", "number": 1154, "title": "Documentation for new _internal database and tables", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 6346396, "label": "Datasette 0.54"}, "comments": 2, "created_at": "2020-12-18T22:34:52Z", "updated_at": "2021-01-25T00:09:22Z", "closed_at": "2021-01-25T00:08:41Z", "author_association": "OWNER", "pull_request": null, "body": "> Needs documentation, but I can wait to write that until I've tested out the feature a bit more.\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1150#issuecomment-748352106_", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1154/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 792931244, "node_id": "MDU6SXNzdWU3OTI5MzEyNDQ=", "number": 1202, "title": "Documentation convention for marking unstable APIs.", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 6346396, "label": "Datasette 0.54"}, "comments": 2, "created_at": "2021-01-24T23:47:18Z", "updated_at": "2021-01-25T00:01:02Z", "closed_at": "2021-01-25T00:01:02Z", "author_association": "OWNER", "pull_request": null, "body": "> I'm going to document this but mark it as unstable, using a new documentation convention for marking unstable APIs.\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1154#issuecomment-766462197_", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1202/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 785588942, "node_id": "MDU6SXNzdWU3ODU1ODg5NDI=", "number": 1187, "title": "extra_body_script() support for script type=\"module\"", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 6346396, "label": "Datasette 0.54"}, "comments": 1, "created_at": "2021-01-14T02:01:47Z", "updated_at": "2021-01-24T21:21:44Z", "closed_at": "2021-01-14T02:14:39Z", "author_association": "OWNER", "pull_request": null, "body": "Follows #1186. The `extra_body_script()` plugin hook should provide a mechanism for specifying that the script should use `<script type=\"module\">`.\r\n\r\nRelevant docs: https://docs.datasette.io/en/stable/plugin_hooks.html#extra-body-script-template-database-table-columns-view-name-request-datasette", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1187/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 785573793, "node_id": "MDU6SXNzdWU3ODU1NzM3OTM=", "number": 1186, "title": "script type=\"module\" support", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 6346396, "label": "Datasette 0.54"}, "comments": 1, "created_at": "2021-01-14T01:17:47Z", "updated_at": "2021-01-24T21:21:41Z", "closed_at": "2021-01-14T01:50:58Z", "author_association": "OWNER", "pull_request": null, "body": "Custom JavaScript can be loaded in `metadata.json` like this:\r\n```json\r\n{\r\n    \"extra_js_urls\": [\r\n        {\r\n            \"url\": \"https://code.jquery.com/jquery-3.2.1.slim.min.js\",\r\n            \"sri\": \"sha256-k2WSCIexGzOj3Euiig+TlR8gA0EmPjuc79OEeY5L45g=\"\r\n        }\r\n    ]\r\n}\r\n```\r\nAdd a `\"module\": true` option which causes the resulting script element to use `<script type=\"module\">`", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1186/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 784628163, "node_id": "MDU6SXNzdWU3ODQ2MjgxNjM=", "number": 1185, "title": "\"Statement may not contain PRAGMA\" error is not strictly true", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 6346396, "label": "Datasette 0.54"}, "comments": 3, "created_at": "2021-01-12T22:07:10Z", "updated_at": "2021-01-24T21:21:37Z", "closed_at": "2021-01-12T22:26:26Z", "author_association": "OWNER", "pull_request": null, "body": "Consider https://latest.datasette.io/fixtures?sql=select+%27select%0D%0A%27+%7C%7C+group_concat%28%27++++case+when+%5B%27+%7C%7C+name+%7C%7C+%27%5D+is+not+null+then+%27+%7C%7C+quote%28name+%7C%7C+%27%2C+%27%29+%7C%7C+%27+else+%27%27%27%27+end%27%2C+%27+%7C%7C%0D%0A%27%29+%7C%7C+%27%0D%0A++as+columns%2C%0D%0A++count%28*%29+as+num_rows%0D%0Afrom%0D%0A++%5B%27+%7C%7C+%3Atable+%7C%7C+%27%5D%0D%0Agroup+by%0D%0A++columns%0D%0Aorder+by%0D%0A++num_rows+desc%27+as+query+from+pragma_ytable_info%28%3Atable%29&table=facetable\r\n\r\nIt says \"Statement may not contain PRAGMA\" - but that's not actually true. Datasette has an allow-list of PRAGMA that are OK - in this case there was a typo in `pragma_ytable_info` which caused the error, but pragma_table_info` would have been OK.\r\n\r\nSo the error message is misleading.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1185/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 783714076, "node_id": "MDU6SXNzdWU3ODM3MTQwNzY=", "number": 1184, "title": "request.full_path property", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 6346396, "label": "Datasette 0.54"}, "comments": 0, "created_at": "2021-01-11T21:21:58Z", "updated_at": "2021-01-24T21:21:16Z", "closed_at": "2021-01-11T21:34:47Z", "author_association": "OWNER", "pull_request": null, "body": "> I'll also add `request.full_path` for consistency with these: https://github.com/simonw/datasette/blob/97fb10c17dd007a275ab743742e93e932335ad67/datasette/utils/asgi.py#L77-L90\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1179#issuecomment-755495387_", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1184/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 782692159, "node_id": "MDU6SXNzdWU3ODI2OTIxNTk=", "number": 1182, "title": "Retire \"Ecosystem\" page in favour of datasette.io/plugins and /tools", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 6346396, "label": "Datasette 0.54"}, "comments": 3, "created_at": "2021-01-09T21:54:47Z", "updated_at": "2021-01-24T21:21:09Z", "closed_at": "2021-01-09T22:17:28Z", "author_association": "OWNER", "pull_request": null, "body": "https://docs.datasette.io/en/stable/ecosystem.html is no longer needed.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1182/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 780267857, "node_id": "MDU6SXNzdWU3ODAyNjc4NTc=", "number": 1178, "title": "Use force_https_urls on when deploying with Cloud Run", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 6346396, "label": "Datasette 0.54"}, "comments": 9, "created_at": "2021-01-06T08:20:55Z", "updated_at": "2021-01-24T21:21:05Z", "closed_at": "2021-01-06T18:24:47Z", "author_association": "OWNER", "pull_request": null, "body": "_Original title: datasette.absolute_url() should return https:// not http:// on Cloud Run_\r\n\r\nhttps://latest-with-plugins.datasette.io/github/issue_comments.Notebook?_labels=on currently provides `http://` links, which break in Observable since it won't load `http://` content.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1178/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 772438273, "node_id": "MDU6SXNzdWU3NzI0MzgyNzM=", "number": 1157, "title": "Use time.perf_counter() instead of time.time() to measure performance", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 6346396, "label": "Datasette 0.54"}, "comments": 1, "created_at": "2020-12-21T20:21:41Z", "updated_at": "2021-01-24T21:20:42Z", "closed_at": "2020-12-21T21:49:20Z", "author_association": "OWNER", "pull_request": null, "body": "I do that in a bunch of places: https://ripgrep.datasette.io/-/ripgrep?pattern=time%28%29&literal=on&glob=datasette%2F%2A%2A\r\n\r\nhttps://docs.python.org/3/library/time.html#time.perf_counter\r\n\r\n> `time.``perf_counter`() \u2192 float\r\n> \r\n> Return the value (in fractional seconds) of a performance counter, i.e. a clock with the highest available resolution to measure a short duration. It does include time elapsed during sleep and is system-wide. The reference point of the returned value is undefined, so that only the difference between the results of consecutive calls is valid.\r\n> \r\n> _New in version 3.3._", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1157/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 772408750, "node_id": "MDU6SXNzdWU3NzI0MDg3NTA=", "number": 1156, "title": "Rename _schemas to _internal", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 6346396, "label": "Datasette 0.54"}, "comments": 1, "created_at": "2020-12-21T19:27:58Z", "updated_at": "2021-01-24T21:20:39Z", "closed_at": "2020-12-21T19:51:18Z", "author_association": "OWNER", "pull_request": null, "body": "I like `_internal` as the name for the in-memory Datasette schema database better.\r\n\r\nRefs #1154 #1150 #1155", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1156/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 742011049, "node_id": "MDU6SXNzdWU3NDIwMTEwNDk=", "number": 1091, "title": ".json and .csv exports fail to apply base_url", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 6346396, "label": "Datasette 0.54"}, "comments": 22, "created_at": "2020-11-12T23:45:16Z", "updated_at": "2021-01-24T21:20:24Z", "closed_at": "2021-01-09T22:19:29Z", "author_association": "OWNER", "pull_request": null, "body": "> Just tested with the latest Docker image, and it works pretty much everywhere!  THANK YOU!\r\n> \r\n> I did notice that if I try to export json or csv, the base is not applied.  Not sure if I should reopen this issue or open a new one.\r\n> \r\n> To see this, go here: https://corpora.tika.apache.org/datasette/corpora-metadata/REF_PARSE_EXCEPTION_TYPES\r\n> \r\n> Click/hover over json or CSV and you'll see that the 'datasette' base is not included.\r\n\r\n_Originally posted by @tballison in https://github.com/simonw/datasette/issues/865#issuecomment-726385422_", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1091/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 760605882, "node_id": "MDU6SXNzdWU3NjA2MDU4ODI=", "number": 1135, "title": "Feature: --create option to create database file if it does not yet exist", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2020-12-09T19:23:58Z", "updated_at": "2021-01-24T21:19:39Z", "closed_at": "2020-12-09T19:45:52Z", "author_association": "OWNER", "pull_request": null, "body": "I'd like to be able to tell people to run the following in the Datasette documentation to get started:\r\n\r\n    brew install datasette\r\n    datasette install datasette-upload-csvs\r\n    datasette data.db --create --root --open\r\n\r\nThis would give them a local Datasette instance with the ability to drag-and-drop CSV files directly into it.\r\n\r\nJust one catch: I don't want to have to talk them through creating an empty SQLite database file. So I want to add a new `--create` option which means \"If any of the database files passed on the command-line do not yet exist, create them\".", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1135/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 791381623, "node_id": "MDU6SXNzdWU3OTEzODE2MjM=", "number": 1197, "title": "DB size limit for publishing with Heroku", "user": {"value": 1186275, "label": "mtdukes"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2021-01-21T18:08:43Z", "updated_at": "2021-01-24T20:53:44Z", "closed_at": "2021-01-24T20:53:44Z", "author_association": "NONE", "pull_request": null, "body": "Hello,\r\nI tried searching for this, but can't seem to get a great answer: Does anybody know the size limit for databases deploying to Heroku? The files I'm working with are pretty large, but I might be able to pare them down if I have a limit in mind.\r\n\r\n I'm getting the following error when running `datasette heroku publish`:\r\n\r\n`RangeError [ERR_INVALID_OPT_VALUE]: The value \"14504095744\" is invalid for option \"size\"`", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1197/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 792625812, "node_id": "MDU6SXNzdWU3OTI2MjU4MTI=", "number": 1198, "title": "Plugin testing documentation on using pytest-httpx", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2021-01-23T18:46:16Z", "updated_at": "2021-01-24T20:40:38Z", "closed_at": "2021-01-24T20:38:43Z", "author_association": "OWNER", "pull_request": null, "body": "I keep on having to figure this out: if you use the https://pypi.org/project/pytest-httpx/ fixture to write tests against mocked external APIs, they will fail because that module will break Datasette's own `datasette.client` testing mechanism.\r\n\r\nYou can fix this using:\r\n\r\n```python\r\n@pytest.fixture\r\ndef non_mocked_hosts():\r\n    return [\"localhost\"]\r\n```\r\n\r\nSee https://github.com/simonw/datasette-indieauth/blob/1.2/tests/test_indieauth.py\r\n\r\nI can add this tip to the https://docs.datasette.io/en/stable/testing_plugins.html page.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1198/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 725996507, "node_id": "MDU6SXNzdWU3MjU5OTY1MDc=", "number": 1036, "title": "Make it possible to download BLOB data from the Datasette UI", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 6026070, "label": "0.51"}, "comments": 16, "created_at": "2020-10-20T22:47:56Z", "updated_at": "2021-01-18T17:45:00Z", "closed_at": "2020-10-25T00:14:52Z", "author_association": "OWNER", "pull_request": null, "body": "Currently you can only extract binary BLOB data as base64-encoded JSON, which is not user friendly at all. It should always be possible for end-users to get the binary data out.\r\n\r\nI'm worried about XSS vulnerabilities here, but hopefully sending `Content-Type: application/octet-stream` helps there? Need to research that.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1036/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 548591089, "node_id": "MDU6SXNzdWU1NDg1OTEwODk=", "number": 657, "title": "Allow creation of virtual tables at startup", "user": {"value": 1055831, "label": "dazzag24"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2020-01-12T16:10:55Z", "updated_at": "2021-01-15T20:24:35Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "Hi,\r\n \r\nI've been experimenting with SQLite reading from huge datasets using this excellent Parquet extension from @cldellow.\r\nhttps://cldellow.com/2018/06/22/sqlite-parquet-vtable.html\r\nhttps://github.com/cldellow/sqlite-parquet-vtable\r\n\r\nThis works really well, but I was keen to see if I could combine datasette with this. Having previously experimented with the spatialite extension I knew that datasette supports loading extensions in the underlying sqlite instance.  However I hit a blocker as the current design only allows SELECT statements to be executed and so I am unable to execute the crucial \r\n\r\nCREATE VIRTUAL TABLE .........\r\n\r\ncommand that is required to load the data from the parquet file into the table.\r\n\r\nIt seems like this would be a simple-ish change, but I don't know enough about the architecture of datasette to start implementing this myself?  Could this be done as a datasette plugin?  or would this require more fundamental changes at initialisation time?\r\n\r\nMy thoughts are that something at init time could detect that the user was loading a *.parquet file and then switch to a mode were it loads that via the \"CREATE VIRTUAL TABLE...\" rather than loading the *.db file in the default case??\r\n\r\nI'm happy to contribute code and testing, I just need some pointers on the best approach.\r\n\r\nThanks\r\nDarren", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/657/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 787104850, "node_id": "MDU6SXNzdWU3ODcxMDQ4NTA=", "number": 1192, "title": "Form Plugin for in-depth Datasette Querying", "user": {"value": 1024355, "label": "tomershvueli"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2021-01-15T18:24:50Z", "updated_at": "2021-01-15T18:24:50Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "I envision a sort of easy-to-build form plugin that would be able to map a user's inputs to different fields/columns in a Datasette database. ", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1192/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 782708469, "node_id": "MDU6SXNzdWU3ODI3MDg0Njk=", "number": 1183, "title": "Take advantage of sqlite-utils cached table counts, if available", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2021-01-09T23:51:48Z", "updated_at": "2021-01-12T02:42:08Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "sqlite-utils 3.2 now has a mechanism for creating a `_counts` table with triggers to maintain counts for individual tables. Datasette could look for this table and use it for counts, dramatically speeding up places that currently suffer from slow counts. Refs #859.\r\n\r\nhttps://sqlite-utils.datasette.io/en/stable/python-api.html#cached-table-counts-using-triggers", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1183/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 778450486, "node_id": "MDU6SXNzdWU3Nzg0NTA0ODY=", "number": 1171, "title": "GitHub Actions workflow to build and sign macOS binary executables", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 8, "created_at": "2021-01-04T23:36:59Z", "updated_at": "2021-01-07T19:36:00Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "Using PyInstaller, as explored in #93 and https://til.simonwillison.net/python/packaging-pyinstaller\r\n\r\nThe bigger challenge will be the code signing bit. I'll need a Apple Developer account ($99/year) and some extensive CI fiddling.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1171/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 780767542, "node_id": "MDU6SXNzdWU3ODA3Njc1NDI=", "number": 1180, "title": "Lazily evaluated arguments for call_with_supported_arguments", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2021-01-06T18:43:34Z", "updated_at": "2021-01-07T18:56:24Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "While building https://github.com/simonw/datasette-export-notebook I thought it would be nice to be able to show a count of exported records on the page \"This will stream 10,422 records to your notebook\".\r\n\r\nNone of the documented arguments on https://docs.datasette.io/en/0.53/plugin_hooks.html#register-output-renderer-datasette expose the count. The closest is `sql` which could be executed as `select count(*) from ({sql})` but that's a bit inelegant.\r\n\r\nSo, idea: if your defined render function takes a `total` argument, the caller runs a `count(*)` query and passes you that total.\r\n\r\nTo implement this I would need to teach the `call_with_supported_arguments` that some arguments are lazy - they should execute a function (or an async function) but only if they are needed.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1180/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 497170355, "node_id": "MDU6SXNzdWU0OTcxNzAzNTU=", "number": 576, "title": "Documented internals API for use in plugins", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 3268330, "label": "Datasette 1.0"}, "comments": 10, "created_at": "2019-09-23T15:28:50Z", "updated_at": "2021-01-05T23:12:51Z", "closed_at": "2021-01-05T23:12:37Z", "author_association": "OWNER", "pull_request": null, "body": "Quite a few of the plugin hooks make a `datasette\u201d`instance of the Datasette class available to the plugins, so that they can look up configuration settings and execute database queries.\r\n\r\nThis means it should provide a documented, stable API so that plugin authors can rely on it.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/576/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}