{"id": 274343647, "node_id": "MDExOlB1bGxSZXF1ZXN0MTUyOTE0NDgw", "number": 107, "title": "add support for ?field__isnull=1", "user": {"value": 3433657, "label": "raynae"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2017-11-15T23:36:36Z", "updated_at": "2017-11-17T15:12:29Z", "closed_at": "2017-11-17T13:29:22Z", "author_association": "CONTRIBUTOR", "pull_request": "simonw/datasette/pulls/107", "body": "Is this what you had in mind for [this issue](https://github.com/simonw/datasette/issues/64)?", "repo": {"value": 107914493, "label": "datasette"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/107/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null} {"id": 309047460, "node_id": "MDU6SXNzdWUzMDkwNDc0NjA=", "number": 188, "title": "Ability to bundle metadata and templates inside the SQLite file", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2018-03-27T16:42:07Z", "updated_at": "2020-12-04T17:18:34Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "One of the nicest qualities of SQLite as a data format is that you get a single file which you can then backup or share with other people. \r\n\r\nDatasette breaks this a little once you start including custom metadata.json or template files and CSS.\r\n\r\nIt would be cool if there was an optional mechanism for baking that extra configuration into the SQLite file itself. That way entire datasette mini-applications (including canned queries and custom HTML and CSS) could be constructed as single .db files.\r\n\r\nSince datasette configuration is all file-based, one way to achieve that would be to support a \"datasette_files\" table which, if present is used to search for file contents by path.\r\n\r\nThis is inline with the philosophy described by https://www.sqlite.org/appfileformat.html\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/188/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 325352370, "node_id": "MDExOlB1bGxSZXF1ZXN0MTg5NzA3Mzc0", "number": 279, "title": "Add version number support with Versioneer", "user": {"value": 198537, "label": "rgieseke"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2018-05-22T15:39:45Z", "updated_at": "2018-05-22T19:35:23Z", "closed_at": "2018-05-22T19:35:22Z", "author_association": "CONTRIBUTOR", "pull_request": "simonw/datasette/pulls/279", "body": "I think that's all for getting Versioneer support, I've been happily using it in a couple of projects ... \r\n\r\n```\r\nIn [2]: datasette.__version__\r\nOut[2]: '0.22+3.g6e12445'\r\n```\r\nRepo:\r\nhttps://github.com/warner/python-versioneer\r\n\r\nVersioneer Licence:\r\nPublic Domain (CC0-1.0)\r\n\r\nCloses #273\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/279/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null} {"id": 335200136, "node_id": "MDU6SXNzdWUzMzUyMDAxMzY=", "number": 327, "title": "Explore if SquashFS can be used to shrink size of packaged Docker containers", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2018-06-24T18:15:16Z", "updated_at": "2022-02-17T23:37:24Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "Inspired by this article: https://cldellow.com/2018/06/22/sqlite-parquet-vtable.html#sqlite-database-indexed--squashed\r\n\r\nhttps://en.wikipedia.org/wiki/SquashFS is \"a compressed read-only file system for Linux\" - which means it could be a really nice fit for Datasette and its read-only SQLite databases.\r\n\r\nIt would be interesting to explore a Dockerfile recipe that used SquashFS to compress the SQLite database file that was bundled up by `datasette package` and friends.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/327/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 377155320, "node_id": "MDU6SXNzdWUzNzcxNTUzMjA=", "number": 370, "title": "Integration with JupyterLab", "user": {"value": 82988, "label": "psychemedia"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2018-11-04T13:57:13Z", "updated_at": "2022-09-29T08:17:47Z", "closed_at": null, "author_association": "CONTRIBUTOR", "pull_request": null, "body": "I just watched a demo video for the [JupyterLab Chart Editor](https://www.crowdcast.io/e/introducing-JupyterLab-Chart-Editor/) which wraps the plotly chart editor app in a JupyterLab panel and lets you open a plotly chart JSON file in that editor. Essentially, it pops an HTML app into a panel in JupyterLab, and I think registers the app as a file viewer for a particular file type. (I'm not completely taken by it, tbh, because it means you can do irreproducible things to the chart definition file, but that's another issue).\r\n\r\nJupyterLab extensions can also open files from a dialogue as the iframe/html previewer shows: https://github.com/timkpaine/jupyterlab_iframe.\r\n\r\nThis made me wonder about what `datasette` integration with JupyterLab might do.\r\n\r\nFor example, by right-clicking on a CSV file (for which there is already a CSV table view) in the file browser, offer a *View / Run as datasette* file viewer option that will:\r\n\r\n- run the CSV file through `csvs-to-sqlite`;\r\n- launch the `datasette` server and display the `datasette` view in a JupyterLab panel.\r\n\r\n(? Create a new SQLite db for each CSV file and launch each datasette view on a new port? Or have a JupyterLab (session?) SQLite db that stores all `datasette` viewed CSVs and runs on a single port?) \r\n\r\nAs a freebie, the `datasette` API would allow you to run efficient SQL queries against the file eg using using `pandas.read_sql()` queries in a notebook in the same space.\r\n\r\nRelated:\r\n\r\n- [JupyterLab extensions docs](https://jupyterlab.readthedocs.io/en/stable/user/extensions.html)\r\n- a [cookiecutter for wrting JupyterLab extensions using Javascript](https://github.com/jupyterlab/extension-cookiecutter-js)\r\n- a [cookiecutter for writing JupyterLab extensions using Typescript](https://github.com/jupyterlab/extension-cookiecutter-ts)\r\n- tutorial: [Let\u2019s Make an xkcd JupyterLab Extension](https://jupyterlab.readthedocs.io/en/stable/developer/xkcd_extension_tutorial.html)", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/370/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 432893491, "node_id": "MDExOlB1bGxSZXF1ZXN0MjcwMjUxMDIx", "number": 432, "title": "Refactor facets to a class and new plugin, refs #427", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2019-04-13T20:04:45Z", "updated_at": "2019-05-03T00:04:24Z", "closed_at": "2019-05-03T00:04:24Z", "author_association": "OWNER", "pull_request": "simonw/datasette/pulls/432", "body": "WIP for #427", "repo": {"value": 107914493, "label": "datasette"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/432/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null} {"id": 443021509, "node_id": "MDU6SXNzdWU0NDMwMjE1MDk=", "number": 461, "title": "Paginate + search for databases/tables on the homepage", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": {"value": 3268330, "label": "Datasette 1.0"}, "comments": 4, "created_at": "2019-05-11T18:05:34Z", "updated_at": "2020-12-17T22:14:46Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "Split out from #460 - in order to support large numbers of connected databases the homepage needs to be paginated.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/461/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 455852801, "node_id": "MDU6SXNzdWU0NTU4NTI4MDE=", "number": 507, "title": "Every datasette plugin on the ecosystem page should have a screenshot", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2019-06-13T17:02:51Z", "updated_at": "2020-09-17T02:47:35Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "https://github.com/simonw/datasette/blob/master/docs/ecosystem.rst", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/507/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 465728430, "node_id": "MDExOlB1bGxSZXF1ZXN0Mjk1NzExNTA0", "number": 554, "title": "Fix static mounts using relative paths and prevent traversal exploits", "user": {"value": 3243482, "label": "abdusco"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2019-07-09T11:32:02Z", "updated_at": "2019-07-11T16:29:26Z", "closed_at": "2019-07-11T16:13:19Z", "author_association": "CONTRIBUTOR", "pull_request": "simonw/datasette/pulls/554", "body": "While debugging why my static mounts using a relative path (`--static mystatic:rel/path/to/dir`) not working, I noticed that the requests fail no matter what, returning 404 errors. \r\n\r\nThe reason is that datasette tries to prevent traversal exploits by checking if the path is relative to its registered directory. This check fails when the mount is a relative directory, because `/abs/dir/file` obviously not under `dir/file`. \r\n\r\nhttps://github.com/simonw/datasette/blob/81fa8b6cdc5457b42a224779e5291952314e8d20/datasette/utils/asgi.py#L303-L306\r\n\r\nThis also has the consequence of returning any requested file, because when `/abs/dir/../../evil.file` resolves `aiofiles` happily returns it to the client after it resolves the path itself. The solution is to make sure we're checking relativity of paths after they're fully resolved.\r\n\r\nI've implemented the mentioned changes and also updated the tests.", "repo": {"value": 107914493, "label": "datasette"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/554/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null} {"id": 466996584, "node_id": "MDExOlB1bGxSZXF1ZXN0Mjk2NzM1MzIw", "number": 557, "title": "Get tests running on Windows using Travis CI", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2019-07-11T16:36:57Z", "updated_at": "2021-07-10T23:39:48Z", "closed_at": "2021-07-10T23:39:48Z", "author_association": "OWNER", "pull_request": "simonw/datasette/pulls/557", "body": "Refs #511", "repo": {"value": 107914493, "label": "datasette"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/557/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null} {"id": 481885279, "node_id": "MDU6SXNzdWU0ODE4ODUyNzk=", "number": 569, "title": "More advanced connection pooling", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2019-08-17T13:20:41Z", "updated_at": "2019-10-02T22:44:37Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "We need a much smarter way of handling database connections.\r\n\r\nToday, connections are simple: Datasette runs a number of threads (defaults to 3) and each thread gets a threadlocal read-only (or immutable) connection to each attached database - opened on demand.\r\n\r\nFor Datasette Library (#417) I want to support potentially hundreds of attached databases. Datasette Edit (#567) is going to introduce a need for writable connections too.\r\n\r\nI'd also like to be able to run joins across multiple databases (#283) which further complicates things.\r\n\r\nSupporting thousands of open SQLite connections at once feels like it won't provide good enough performance (though I should benchmark that to be sure). Some kind of connection pooling is likely to be necessary.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/569/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 487987958, "node_id": "MDExOlB1bGxSZXF1ZXN0MzEzMTA1NjM0", "number": 57, "title": "Add triggers while enabling FTS", "user": {"value": 49260, "label": "amjith"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2019-09-02T04:23:40Z", "updated_at": "2019-09-03T01:03:59Z", "closed_at": "2019-09-02T23:42:29Z", "author_association": "CONTRIBUTOR", "pull_request": "simonw/sqlite-utils/pulls/57", "body": "This adds the option for a user to set up triggers in the database to keep their FTS table in sync with the parent table. \r\n\r\nRef: https://sqlite.org/fts5.html#external_content_and_contentless_tables\r\n\r\nI would prefer to make the creation of triggers the default behavior, but that will break existing usage where people have been calling `populate_fts` after inserting new rows.\r\n\r\nI am happy to make changes to the PR as you see fit. ", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/57/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null} {"id": 548591089, "node_id": "MDU6SXNzdWU1NDg1OTEwODk=", "number": 657, "title": "Allow creation of virtual tables at startup", "user": {"value": 1055831, "label": "dazzag24"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2020-01-12T16:10:55Z", "updated_at": "2021-01-15T20:24:35Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "Hi,\r\n \r\nI've been experimenting with SQLite reading from huge datasets using this excellent Parquet extension from @cldellow.\r\nhttps://cldellow.com/2018/06/22/sqlite-parquet-vtable.html\r\nhttps://github.com/cldellow/sqlite-parquet-vtable\r\n\r\nThis works really well, but I was keen to see if I could combine datasette with this. Having previously experimented with the spatialite extension I knew that datasette supports loading extensions in the underlying sqlite instance. However I hit a blocker as the current design only allows SELECT statements to be executed and so I am unable to execute the crucial \r\n\r\nCREATE VIRTUAL TABLE .........\r\n\r\ncommand that is required to load the data from the parquet file into the table.\r\n\r\nIt seems like this would be a simple-ish change, but I don't know enough about the architecture of datasette to start implementing this myself? Could this be done as a datasette plugin? or would this require more fundamental changes at initialisation time?\r\n\r\nMy thoughts are that something at init time could detect that the user was loading a *.parquet file and then switch to a mode were it loads that via the \"CREATE VIRTUAL TABLE...\" rather than loading the *.db file in the default case??\r\n\r\nI'm happy to contribute code and testing, I just need some pointers on the best approach.\r\n\r\nThanks\r\nDarren", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/657/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 607067303, "node_id": "MDExOlB1bGxSZXF1ZXN0NDA5MTIzODk3", "number": 737, "title": "Custom pages mechanism, refs #648", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2020-04-26T17:31:41Z", "updated_at": "2020-04-26T18:46:43Z", "closed_at": "2020-04-26T18:46:43Z", "author_association": "OWNER", "pull_request": "simonw/datasette/pulls/737", "body": "Refs #648. TODO:\r\n- [x] Pass a `view_name` to `render_template()`\r\n- [x] Mechanism for custom status code / headers / redirect\r\n- [x] Documentation", "repo": {"value": 107914493, "label": "datasette"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/737/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null} {"id": 615626118, "node_id": "MDU6SXNzdWU2MTU2MjYxMTg=", "number": 22, "title": "Try out ExifReader", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2020-05-11T06:32:13Z", "updated_at": "2020-05-14T05:59:53Z", "closed_at": null, "author_association": "MEMBER", "pull_request": null, "body": "https://pypi.org/project/ExifReader/\r\n\r\nNew fork that should be able to handle EXIF in HEIC files.\r\n\r\nForked here: https://github.com/ianare/exif-py/issues/102#issuecomment-626376522\r\n\r\nRefs #3 ", "repo": {"value": 256834907, "label": "dogsheep-photos"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/dogsheep-photos/issues/22/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 632919570, "node_id": "MDExOlB1bGxSZXF1ZXN0NDI5NjEzODkz", "number": 809, "title": "Publish secrets", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 5512395, "label": "Datasette 0.44"}, "comments": 4, "created_at": "2020-06-07T02:00:31Z", "updated_at": "2020-06-11T16:02:13Z", "closed_at": "2020-06-11T16:02:03Z", "author_association": "OWNER", "pull_request": "simonw/datasette/pulls/809", "body": "Refs #787. Will need quite a bit of manual testing since this involves code which runs against Heroku and Cloud Run.", "repo": {"value": 107914493, "label": "datasette"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/809/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null} {"id": 648749062, "node_id": "MDExOlB1bGxSZXF1ZXN0NDQyNTA1MDg4", "number": 883, "title": "Skip counting hidden tables", "user": {"value": 3243482, "label": "abdusco"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2020-07-01T07:38:08Z", "updated_at": "2020-07-02T00:25:44Z", "closed_at": null, "author_association": "CONTRIBUTOR", "pull_request": "simonw/datasette/pulls/883", "body": "Potential fix for https://github.com/simonw/datasette/issues/859.\r\n\r\nDisabling table counts for hidden tables speeds up database page quite a bit. In my setup it reduced load time by 2/3 (~300 -> ~90ms)", "repo": {"value": 107914493, "label": "datasette"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/883/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null} {"id": 703246031, "node_id": "MDU6SXNzdWU3MDMyNDYwMzE=", "number": 51, "title": "github-to-sqlite should handle rate limits better", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2020-09-17T04:01:50Z", "updated_at": "2022-10-14T16:34:07Z", "closed_at": null, "author_association": "MEMBER", "pull_request": null, "body": "From #50 - right now it will crash with an error of it hits the rate limit. Since the rate limit information (including reset time) is available in the headers it could automatically sleep and try again instead.", "repo": {"value": 207052882, "label": "github-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/github-to-sqlite/issues/51/reactions\", \"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 706001517, "node_id": "MDU6SXNzdWU3MDYwMDE1MTc=", "number": 163, "title": "Idea: conversions= could take Python functions", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2020-09-22T00:37:12Z", "updated_at": "2021-12-20T00:56:52Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "Right now you use `conversions=` like this:\r\n\r\n```python\r\ndb[\"example\"].insert({\r\n \"name\": \"The Bigfoot Discovery Museum\"\r\n}, conversions={\"name\": \"upper(?)\"})\r\n```\r\nHow about if you could optionally provide a Python function (or a lambda) like this?\r\n```python\r\ndb[\"example\"].insert({\r\n \"name\": \"The Bigfoot Discovery Museum\"\r\n}, conversions={\"name\": lambda s: s.upper()})\r\n```\r\nThis would work by creating a random name for that function, registering it (similar to #162), executing the SQL and then un-registering the custom function at the end.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/163/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 718238967, "node_id": "MDU6SXNzdWU3MTgyMzg5Njc=", "number": 1003, "title": "from_json jinja2 filter", "user": {"value": 649467, "label": "mhalle"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2020-10-09T15:30:58Z", "updated_at": "2020-10-09T17:17:07Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "When JSON fields are rendered in a jinja2 template, it is handy to be able to manipulate them as data (e.g., iterate over an array of values). \r\n\r\nAnsible has a \"from_json\" function, which just called json.loads. It's a trivial as a datasette plugin, but it seems generally useful. Does it makes sense to add it directly into the app?", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1003/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 727848625, "node_id": "MDU6SXNzdWU3Mjc4NDg2MjU=", "number": 12, "title": "Some workout columns should be float, not text", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2020-10-23T02:47:02Z", "updated_at": "2022-06-23T04:35:02Z", "closed_at": null, "author_association": "MEMBER", "pull_request": null, "body": "Columns `duration`, `totalDistance` and `totalEnergyBurned` should be converted to float.\r\n\r\nhttps://github.com/dogsheep/healthkit-to-sqlite/blob/71e36e1cf034b96de2a8e6652265d782d3fdf63b/healthkit_to_sqlite/utils.py#L50-L57", "repo": {"value": 197882382, "label": "healthkit-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/12/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 727915394, "node_id": "MDExOlB1bGxSZXF1ZXN0NTA4NzE5NTY3", "number": 1043, "title": "Include LICENSE in sdist", "user": {"value": 45380, "label": "bollwyvl"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2020-10-23T05:04:12Z", "updated_at": "2020-10-26T00:14:57Z", "closed_at": "2020-10-23T20:54:35Z", "author_association": "CONTRIBUTOR", "pull_request": "simonw/datasette/pulls/1043", "body": "Hi, thanks for `datasette`! \r\n\r\nThis PR adds the `LICENSE` to source distributions, which seems the norm for Apache-2.0 stuff.\r\n\r\nI noticed the [0.50.2 sdist](https://files.pythonhosted.org/packages/f2/ba/1b5f182c3f1769c0863bcaa77406bdcb81c92e31bb579959c01b1d8951c0/datasette-0.50.2.tar.gz) doesn't ship `LICENSE`, but the 0.5.2 `whl` does, so I'm assuming the intent _is_ to ship... and it's a one-liner!\r\n\r\nMotivation: \r\n\r\nIt might be a bit of a slog, but I'm looking to see about getting `datasette` (and friends!) available on conda-forge. There are a few missing upstreams (`asgi-csrf`, `python-basecov`, `mergedeep`) and some of the plugins don't even appear to _have_ tarballs (just `whl`!), but the little stuff like licenses are nice to get out handled upstream vs separately grabbing them.", "repo": {"value": 107914493, "label": "datasette"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1043/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null} {"id": 729017519, "node_id": "MDExOlB1bGxSZXF1ZXN0NTA5NTkwMjA1", "number": 1049, "title": "Add template block prior to extra URL loaders", "user": {"value": 82988, "label": "psychemedia"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2020-10-25T13:08:55Z", "updated_at": "2020-10-29T09:20:52Z", "closed_at": "2020-10-29T09:20:34Z", "author_association": "CONTRIBUTOR", "pull_request": "simonw/datasette/pulls/1049", "body": "To handle packages that require Javascript state setting prior to loading a package (eg [`thebelab`](https://thebelab.readthedocs.io/en/latest/examples/minimal_example.html), provide a template block before the URLs are loaded.", "repo": {"value": 107914493, "label": "datasette"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1049/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null} {"id": 732634375, "node_id": "MDExOlB1bGxSZXF1ZXN0NTEyNTQ1MzY0", "number": 1061, "title": ".blob output renderer", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 6026070, "label": "0.51"}, "comments": 4, "created_at": "2020-10-29T20:25:08Z", "updated_at": "2020-10-29T22:01:40Z", "closed_at": "2020-10-29T22:01:39Z", "author_association": "OWNER", "pull_request": "simonw/datasette/pulls/1061", "body": "- [x] Remove the `/-/...blob/...` route I added in #1040 in place of the new `.blob` renderer URLs\r\n- [x] Link to new `.blob` download links on the arbitrary query page (using `_blob_hash=...`) - plus tests for this\r\n\r\nCloses #1050, Closes #1051", "repo": {"value": 107914493, "label": "datasette"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1061/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null} {"id": 756876238, "node_id": "MDExOlB1bGxSZXF1ZXN0NTMyMzQ4OTE5", "number": 1130, "title": "Fix footer not sticking to bottom in short pages", "user": {"value": 3243482, "label": "abdusco"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2020-12-04T07:29:01Z", "updated_at": "2021-06-15T13:27:48Z", "closed_at": null, "author_association": "CONTRIBUTOR", "pull_request": "simonw/datasette/pulls/1130", "body": "Fixes https://github.com/simonw/datasette/issues/1129", "repo": {"value": 107914493, "label": "datasette"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1130/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null} {"id": 773913793, "node_id": "MDExOlB1bGxSZXF1ZXN0NTQ0OTIzNDM3", "number": 1158, "title": "Modernize code to Python 3.6+", "user": {"value": 6774676, "label": "eumiro"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 6346396, "label": "Datasette 0.54"}, "comments": 4, "created_at": "2020-12-23T16:21:38Z", "updated_at": "2021-01-24T21:20:50Z", "closed_at": "2020-12-23T17:04:32Z", "author_association": "CONTRIBUTOR", "pull_request": "simonw/datasette/pulls/1158", "body": "- compact dict and set building\r\n- remove redundant parentheses\r\n- simplify chained conditions\r\n- change method name to lowercase\r\n- use triple double quotes for docstrings\r\n\r\nplease feel free to accept/reject any of these independent commits", "repo": {"value": 107914493, "label": "datasette"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1158/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null} {"id": 779156520, "node_id": "MDU6SXNzdWU3NzkxNTY1MjA=", "number": 1175, "title": "Use structlog for logging", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2021-01-05T15:11:36Z", "updated_at": "2022-07-26T12:52:10Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "To solve #241 JSON logging.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1175/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 789336592, "node_id": "MDU6SXNzdWU3ODkzMzY1OTI=", "number": 1195, "title": "view_name = \"query\" for the query page", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2021-01-19T20:21:36Z", "updated_at": "2021-01-25T04:40:08Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "It uses `view_name` of `database` at the moment which isn't as useful.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1195/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 802513359, "node_id": "MDU6SXNzdWU4MDI1MTMzNTk=", "number": 1217, "title": "Possible to deploy as a python app (for Rstudio connect server)?", "user": {"value": 6165713, "label": "plpxsk"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2021-02-05T22:21:24Z", "updated_at": "2022-11-04T11:37:52Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "Is it possible to deploy a `datasette` application as a python\u00a0web app?\r\n\r\nIn my enterprise, I have option to deploy python apps via [Rstudio Connect](https://github.com/rstudio/rsconnect-python), and I would like to publish a `datasette` dashboard for sharing.\r\n\r\nI welcome any pointers to converting `datasette serve` into a python app that can be run as something like `python datasette.py --my_data.db`", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1217/reactions\", \"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 803338729, "node_id": "MDU6SXNzdWU4MDMzMzg3Mjk=", "number": 33, "title": "photo-to-sqlite: command not found", "user": {"value": 11855322, "label": "robmarkcole"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2021-02-08T08:42:57Z", "updated_at": "2021-02-12T15:00:44Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "Having installed in a venv I get:\r\n```\r\n(venv) (base) Robins-MacBook:datasette robin$ photo-to-sqlite apple-photos photos.db\r\n\r\n-bash: photo-to-sqlite: command not found\r\n```", "repo": {"value": 256834907, "label": "dogsheep-photos"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/dogsheep-photos/issues/33/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 838382890, "node_id": "MDU6SXNzdWU4MzgzODI4OTA=", "number": 1273, "title": "Refresh SpatiaLite documentation", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2021-03-23T06:05:55Z", "updated_at": "2022-01-20T21:28:50Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "https://docs.datasette.io/en/0.55/spatialite.html was written before I had tools like [geojson-to-sqlite](https://datasette.io/tools/geojson-to-sqlite) and [shapefile-to-sqlite](https://datasette.io/tools/shapefile-to-sqlite).", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1273/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 845794436, "node_id": "MDU6SXNzdWU4NDU3OTQ0MzY=", "number": 1284, "title": "Feature or Documentation Request: Individual table as home page template", "user": {"value": 192568, "label": "mroswell"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2021-03-31T03:56:17Z", "updated_at": "2021-11-04T03:15:01Z", "closed_at": null, "author_association": "CONTRIBUTOR", "pull_request": null, "body": "It would be great to have a sample showing how to move a single database that has a single table, to the index page. I'm trying it now, and find there is a real depth of Datasette and Python understanding that's required to be successful. \r\n\r\nI've got all the basic jinja concepts down... variables, template control structures, template inheritance, template overrides, css, html, the --template-dir and --static arguments, etc. \r\n\r\nBut copying the table.html file to index.html doesn't work. There are undocumented functions and filters... I can figure some of them out (yay, url_builder.py and utils/__init__.py!) but it's a slog better handled by a much stronger Python developer. \r\n\r\nOne sample would make a world of difference. The ideal form of this documentation would be a diff between the default table.html and how that would look if essentially moved to index.html. The use case is for everyone who wants to create a public-facing website to explore a single table at the root directory. (Maybe a second bit of documentation for people who have a single database with multiple tables.)\r\n\r\n(Hmm... might be cool to have a setting for that, where it happens automagically! If only one table, then home page is at the table level. if only one database, then home page is at the database level.... as an option.)\r\n\r\nI suppose I could ignore this, and somehow do this in the DNS settings once I hook up Vercel to a domain name, maybe.. and remove the breadcrumbs in table.html... but for now, a documentation request in the form of a diff... for viewing a single table (or a single database) at the root.\r\n\r\n(Actually, there's probably room for a whole expanded section on templates. Noticed some nice table metadata in one of the datasette examples, for instance... Hmm... maybe a whole library of solutions in one place... maybe a documentation hackathon! If that's of interest, of course it's a separate issue. )\r\n\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1284/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 855446829, "node_id": "MDExOlB1bGxSZXF1ZXN0NjEzMTc4OTY4", "number": 1296, "title": "Dockerfile: use Ubuntu 20.10 as base", "user": {"value": 82332573, "label": "tmcl-it"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2021-04-12T00:23:32Z", "updated_at": "2021-07-20T08:52:13Z", "closed_at": null, "author_association": "FIRST_TIME_CONTRIBUTOR", "pull_request": "simonw/datasette/pulls/1296", "body": "This PR changes the main Dockerfile to use ubuntu:20.10 as base image instead of python:3.9.2-slim-buster (itself based on debian:buster-slim).\r\n\r\nThe Dockerfile is essentially the one from https://github.com/simonw/datasette/issues/1249#issuecomment-803698983 with some additional cleanups to slim it down.\r\n\r\nThis fixes a couple of issues:\r\n1. The SQLite version in Debian Buster (2.6.0) doesn't support generated columns\r\n2. Installing SpatiaLite from the Debian sid repositories has the side effect of also installing updates to libc and libstdc++ from sid.\r\n\r\nAs a bonus, the Docker image becomes smaller:\r\n\r\n\r\n```\r\n$ docker image ls\r\nREPOSITORY TAG IMAGE ID CREATED SIZE\r\ndatasette 0.56-ubuntu f7aca255140a 5 hours ago 212MB\r\ndatasetteproject/datasette 0.56 efb3b282f390 13 days ago 258MB\r\n```\r\n\r\n### Reproduction of the first issue\r\n\r\n```\r\n$ curl -O https://latest.datasette.io/fixtures.db\r\n % Total % Received % Xferd Average Speed Time Time Time Current\r\n Dload Upload Total Spent Left Speed\r\n100 260k 0 260k 0 0 489k 0 --:--:-- --:--:-- --:--:-- 489k\r\n\r\n$ docker run -v `pwd`:/mnt datasetteproject/datasette:0.56 datasette /mnt/fixtures.db\r\nTraceback (most recent call last):\r\n File \"/usr/local/bin/datasette\", line 8, in \r\n sys.exit(cli())\r\n File \"/usr/local/lib/python3.9/site-packages/click/core.py\", line 829, in __call__\r\n return self.main(*args, **kwargs)\r\n File \"/usr/local/lib/python3.9/site-packages/click/core.py\", line 782, in main\r\n rv = self.invoke(ctx)\r\n File \"/usr/local/lib/python3.9/site-packages/click/core.py\", line 1259, in invoke\r\n return _process_result(sub_ctx.command.invoke(sub_ctx))\r\n File \"/usr/local/lib/python3.9/site-packages/click/core.py\", line 1066, in invoke\r\n return ctx.invoke(self.callback, **ctx.params)\r\n File \"/usr/local/lib/python3.9/site-packages/click/core.py\", line 610, in invoke\r\n return callback(*args, **kwargs)\r\n File \"/usr/local/lib/python3.9/site-packages/datasette/cli.py\", line 544, in serve\r\n asyncio.get_event_loop().run_until_complete(check_databases(ds))\r\n File \"/usr/local/lib/python3.9/asyncio/base_events.py\", line 642, in run_until_complete\r\n return future.result()\r\n File \"/usr/local/lib/python3.9/site-packages/datasette/cli.py\", line 584, in check_databases\r\n await database.execute_fn(check_connection)\r\n File \"/usr/local/lib/python3.9/site-packages/datasette/database.py\", line 155, in execute_fn\r\n return await asyncio.get_event_loop().run_in_executor(\r\n File \"/usr/local/lib/python3.9/concurrent/futures/thread.py\", line 52, in run\r\n result = self.fn(*self.args, **self.kwargs)\r\n File \"/usr/local/lib/python3.9/site-packages/datasette/database.py\", line 153, in in_thread\r\n return fn(conn)\r\n File \"/usr/local/lib/python3.9/site-packages/datasette/utils/__init__.py\", line 892, in check_connection\r\n for r in conn.execute(\r\nsqlite3.DatabaseError: malformed database schema (generated_columns) - near \"AS\": syntax error\r\n```\r\n\r\nHere is the SQLite version:\r\n\r\n```\r\n$ docker run -v `pwd`:/mnt -it datasetteproject/datasette:0.56 /bin/bash\r\nroot@d9220d3b95dd:/# python3\r\nPython 3.9.2 (default, Mar 27 2021, 02:50:26) \r\n[GCC 8.3.0] on linux\r\nType \"help\", \"copyright\", \"credits\" or \"license\" for more information.\r\n>>> import sqlite3\r\n>>> sqlite3.version\r\n'2.6.0'\r\n```\r\n\r\n### Reproduction of the second issue\r\n\r\n```\r\n$ docker build . -t datasette --build-arg VERSION=0.55\r\n[...snip...]\r\nThe following packages will be upgraded:\r\n libc-bin libc6 libstdc++6\r\n[...snip...]\r\nUnpacking libc6:amd64 (2.31-11) over (2.28-10) ...\r\n[...snip...]\r\nUnpacking libstdc++6:amd64 (10.2.1-6) over (8.3.0-6) ...\r\n[...snip...]\r\n```\r\n\r\nBoth libc and libstdc++ are backwards compatible, so the image still works, but it will result in a combination of libraries and Python versions that exists only in the Datasette image, so it's likely untested. In addition, since Debian sid is an always-changing rolling-release, the versions of libc, libstdc++, Spatialite, and their dependencies change frequently, so the library versions in the Datasette image will depend on the day when it was built.\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1296/reactions\", \"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null} {"id": 855476501, "node_id": "MDU6SXNzdWU4NTU0NzY1MDE=", "number": 1298, "title": "improve table horizontal scroll experience", "user": {"value": 192568, "label": "mroswell"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2021-04-12T01:55:16Z", "updated_at": "2022-08-30T21:11:49Z", "closed_at": null, "author_association": "CONTRIBUTOR", "pull_request": null, "body": "Wide tables aren't a huge problem if you know to click and drag right. But it's not at all obvious to do that. (it also tends to blue-select any content as it's dragging.) Depending on column widths, public users might entirely miss all the columns to the right. \r\n\r\nThere is a scrollbar at the bottom of the table, but I'm displaying ALL my records because it's the only way for datasette-vega to make accurate charts. So that bottom scrollbar is likely to be missed. I wonder if some sort of javascript-y mouseover to an arrow might help, similar to those seen in image carousels. Ah: here's a perfect example:\r\n\r\n1. Visit http://google.com\r\n2. Search for: animals endangered\r\n3. Note the 'g-right-button' (in the code) that looks like a right-facing caret in a circle. \r\n4. Click on that and the carousel scrolls right (and 'g-left-button' appears on the left).\r\n\r\nMight be tricky to do that on a table, rather than a one-row carousel, but it's worth experimenting with.\r\n\r\nAnother option is just to put the scrollbars at the top of the table, too. \r\n\r\nMeantime, I'm trying to build a button like the \"View/hide all columns on https://salaries.news.baltimoresun.com/salaries-be494cf/2019+Maryland+state+salaries\r\nMight be nice to have that available by default, with settings in the metadata showing which are on by default.\r\n\r\n(I saw some other closed issues related to horizontal scrolling, and admit I don't entirely understand them. For instance, the animated gif at https://github.com/simonw/datasette/issues/998#issuecomment-714117534 confuses me. )\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1298/reactions\", \"total_count\": 4, \"+1\": 4, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 863884805, "node_id": "MDU6SXNzdWU4NjM4ODQ4MDU=", "number": 1304, "title": "Document how to send multiple values for \"Named parameters\" ", "user": {"value": 9308268, "label": "rayvoelker"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2021-04-21T13:19:06Z", "updated_at": "2021-12-08T03:23:14Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "https://docs.datasette.io/en/stable/sql_queries.html#named-parameters\r\n\r\nI thought that I had seen an example of how to do this example below, but I can't seem to find it\r\n\r\n```sql\r\nselect\r\n *\r\nfrom\r\n bib\r\nwhere\r\n bib.bib_record_num in (1008088,1008092)\r\n```\r\n\r\n```sql\r\nselect\r\n *\r\nfrom\r\n bib\r\nwhere\r\n bib.bib_record_num in (:bib_record_numbers)\r\n```\r\n![image](https://user-images.githubusercontent.com/9308268/115558839-2333a480-a281-11eb-85e6-ce3bada79140.png)\r\n\r\nhttps://ilsweb.cincinnatilibrary.org/collection-analysis/current_collection-204d100?sql=select%0D%0A++*%0D%0Afrom%0D%0A++bib%0D%0Awhere%0D%0A++bib.bib_record_num+in+%28%3Abib_record_numbers%29&bib_record_numbers=1008088%2C1008092\r\n\r\nOr, maybe this isn't a fully supported feature.\r\n\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1304/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 864979486, "node_id": "MDExOlB1bGxSZXF1ZXN0NjIxMTE3OTc4", "number": 1306, "title": "Avoid error sorting by relationships if related tables are not allowed", "user": {"value": 416374, "label": "gfrmin"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2021-04-22T13:53:17Z", "updated_at": "2021-06-02T04:27:00Z", "closed_at": "2021-06-02T04:25:28Z", "author_association": "CONTRIBUTOR", "pull_request": "simonw/datasette/pulls/1306", "body": "Refs #1305", "repo": {"value": 107914493, "label": "datasette"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1306/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null} {"id": 915421499, "node_id": "MDU6SXNzdWU5MTU0MjE0OTk=", "number": 267, "title": "row.update() or row.pk", "user": {"value": 12721157, "label": "Gravitar64"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2021-06-08T19:56:00Z", "updated_at": "2021-06-22T17:27:27Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "Hi,\r\n\r\nfantastic framework for working with Sqlite3 databases!!!\r\n\r\nI tried to update spezific rows in a table and used\r\n\r\nfor row in db[tablename]:\r\n newValue = row[\"counter\"] * row[\"prize\"] \r\n row.update({\"Fieldname\": newValue})\r\n print(row)\r\n\r\n\r\nThis updates the value in the printet row, but not in the database. So I switched to\r\n\r\ndb[tablename].update(id, {\"Filedname\": newValue})\r\n\r\nThis works fine. But row.update would be nicer, because no need for the id (its that row), no need for the tablename and the db (all defined in the for row ... loop).\r\n\r\nThx\r\n\r\n", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/267/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 952179830, "node_id": "MDU6SXNzdWU5NTIxNzk4MzA=", "number": 2, "title": "Command for fetching Hacker News threads from the search API", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2021-07-25T02:00:45Z", "updated_at": "2021-07-25T03:12:57Z", "closed_at": null, "author_association": "MEMBER", "pull_request": null, "body": "I want to be able to fetch every item for a domain, e.g. https://news.ycombinator.com/from?site=simonwillison.net", "repo": {"value": 248903544, "label": "hacker-news-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/hacker-news-to-sqlite/issues/2/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 959137143, "node_id": "MDU6SXNzdWU5NTkxMzcxNDM=", "number": 1415, "title": "feature request: document minimum permissions for service account for cloudrun", "user": {"value": 536941, "label": "fgregg"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2021-08-03T13:48:43Z", "updated_at": "2023-11-05T16:46:59Z", "closed_at": null, "author_association": "CONTRIBUTOR", "pull_request": null, "body": "Thanks again for such a powerful project.\r\n\r\nFor deploying to cloudrun from github actions, I'd like to create a service account with minimal permissions.\r\n\r\nIt would be great to document what those minimum permission that need to be set in the IAM.\r\n\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1415/reactions\", \"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 963527045, "node_id": "MDU6SXNzdWU5NjM1MjcwNDU=", "number": 1424, "title": "Document exceptions that can be raised by db.execute() and friends", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2021-08-08T22:23:25Z", "updated_at": "2021-08-08T22:27:31Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "Not currently covered here: https://docs.datasette.io/en/stable/internals.html#await-db-execute-sql", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1424/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1065429936, "node_id": "I_kwDOBm6k_c4_gSuw", "number": 1532, "title": "Use datasette-table Web Component to guide the design of the JSON API for 1.0", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": {"value": 3268330, "label": "Datasette 1.0"}, "comments": 4, "created_at": "2021-11-28T20:37:18Z", "updated_at": "2022-03-16T20:13:34Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "I realized that one of the reasons I'm having trouble committing to nailing down the JSON API for 1.0 is that I don't use it much myself - I use the `?_shape=array` one quite often, but I don't have any projects that are using the default, more fully-featured API.\r\n\r\nAs an experiment I built a Web Component for embedding Datasette tables on pages - https://github.com/simonw/datasette-table - and I think it's actually going to be a really useful tool for helping me dog food the v1.0 API design.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1532/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1072792507, "node_id": "I_kwDOCGYnMM4_8YO7", "number": 352, "title": "`sqlite-utils insert --extract colname`", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2021-12-07T00:55:44Z", "updated_at": "2022-02-03T22:59:36Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "Is there a reason I've not added `--extract` as an option for `sqlite-utils insert` next? There's a `extracts=` option for the various `table.insert()` etc methods - last line in this code block:\r\n\r\nhttps://github.com/simonw/sqlite-utils/blob/213a0ff177f23a35f3b235386366ff132eb879f1/sqlite_utils/db.py#L2483-L2495", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/352/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1083246400, "node_id": "PR_kwDOBm6k_c4wAMK8", "number": 1562, "title": "Update janus requirement from <0.8,>=0.6.2 to >=0.6.2,<1.1", "user": {"value": 49699333, "label": "dependabot[bot]"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2021-12-17T13:11:10Z", "updated_at": "2021-12-17T23:08:29Z", "closed_at": "2021-12-17T23:08:28Z", "author_association": "CONTRIBUTOR", "pull_request": "simonw/datasette/pulls/1562", "body": "Updates the requirements on [janus](https://github.com/aio-libs/janus) to permit the latest version.\n
\nRelease notes\n

Sourced from janus's releases.

\n
\n

janus 1.0.0 release

\n
    \n
  • Dropped Python 3.6 support
    • \n
  • Janus is marked as stable, no API changes was made for years
    • \n
\n
\n
\n
\nChangelog\n

Sourced from janus's changelog.

\n
\n

1.0.0 (2021-12-17)

\n
    \n
  • Drop Python 3.6 support
    • \n
\n

0.7.0 (2021-11-24)

\n
    \n
  • Add SyncQueue and AsyncQueue Protocols to provide type hints for sync and async queues #374
    • \n
\n

0.6.2 (2021-10-24)

\n
    \n
  • Fix Python 3.10 compatibility #358
    • \n
\n

0.6.1 (2020-10-26)

\n
    \n
  • \n

    Raise RuntimeError on queue.join() after queue closing. #295

    \n
    • \n
  • \n

    Replace timeout type from Optional[int] to Optional[float] #267

    \n
    • \n
\n

0.6.0 (2020-10-10)

\n
    \n
  • \n

    Drop Python 3.5, the minimal supported version is Python 3.6

    \n
    • \n
  • \n

    Support Python 3.9

    \n
    • \n
  • \n

    Refomat with black

    \n
    • \n
\n

0.5.0 (2020-04-23)

\n
    \n
  • Remove explicit loop arguments and forbid creating queues outside event loops #246
    • \n
\n

0.4.0 (2018-07-28)

\n
    \n
  • \n

    Add py.typed macro #89

    \n
    • \n
  • \n

    Drop python 3.4 support and fix minimal version python3.5.3 #88

    \n
    • \n
  • \n

    Add property with that indicates if queue is closed #86

    \n
    • \n
\n

0.3.2 (2018-07-06)

\n
    \n
  • Fixed python 3.7 support #97
    • \n
\n\n
\n

... (truncated)

\n
\n
\nCommits\n\n
\n
\n\n\nDependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.\n\n[//]: # (dependabot-automerge-start)\n[//]: # (dependabot-automerge-end)\n\n---\n\n
\nDependabot commands and options\n
\n\nYou can trigger Dependabot actions by commenting on this PR:\n- `@dependabot rebase` will rebase this PR\n- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it\n- `@dependabot merge` will merge this PR after your CI passes on it\n- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it\n- `@dependabot cancel merge` will cancel a previously requested merge and block automerging\n- `@dependabot reopen` will reopen this PR if it is closed\n- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually\n- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)\n- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)\n- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)\n\n\n
", "repo": {"value": 107914493, "label": "datasette"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1562/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null} {"id": 1087919372, "node_id": "I_kwDOBm6k_c5A2FUM", "number": 1578, "title": "Confirm if documented nginx proxy config works for row pages with escaped characters in their primary key", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2021-12-23T18:27:59Z", "updated_at": "2021-12-24T21:33:19Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "Found this while working on https://github.com/simonw/datasette-tiddlywiki\r\n\r\n\"image\"\r\n\r\nThen clicking on `/tiddlywiki/tiddlers/%24%3A%2FDefaultTiddlers` returns a 404.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1578/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1131295060, "node_id": "I_kwDOBm6k_c5DbjFU", "number": 1634, "title": "Update Dockerfile generated by `datasette publish`", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": {"value": 3268330, "label": "Datasette 1.0"}, "comments": 4, "created_at": "2022-02-11T00:07:26Z", "updated_at": "2022-03-11T17:38:08Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "The generated `Dockerfile` currently looks something like this:\r\n```Dockerfile\r\nFROM python:3.8\r\nCOPY . /app\r\nWORKDIR /app\r\n\r\nENV DATASETTE_SECRET 'edab49cbc5d5f6f33238f54852037e3fee710821960b73edd2ce743454182ae2'\r\nRUN pip install -U datasette datasette-auth-passwords datasette-tiddlywiki datasette-graphql\r\nRUN datasette inspect fixtures.db other.db --inspect-file inspect-data.json\r\nENV PORT 8080\r\nEXPOSE 8080\r\nCMD datasette serve --host 0.0.0.0 -i fixtures.db -i other.db --cors --inspect-file inspect-data.json --metadata metadata.json --create --port $PORT /data/*.db\r\n```\r\nThis is still on Python 3.8, and it generates a pretty large image compared to the `Dockerfile` used for https://hub.docker.com/datasetteproject/datasette - https://github.com/simonw/datasette/blob/0.60.2/Dockerfile\r\n\r\nHere's the code that generates it: https://github.com/simonw/datasette/blob/7d24fd405f3c60e4c852c5d746c91aa2ba23cf5b/datasette/utils/__init__.py#L389-L400", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1634/reactions\", \"total_count\": 2, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 2, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1154399841, "node_id": "I_kwDOBm6k_c5Ezr5h", "number": 1645, "title": "Sensible `cache-control` headers for static assets, including those served by plugins", "user": {"value": 697092, "label": "curiousleo"}, "state": "open", "locked": 0, "assignee": null, "milestone": {"value": 3268330, "label": "Datasette 1.0"}, "comments": 4, "created_at": "2022-02-28T18:12:03Z", "updated_at": "2022-03-08T02:59:29Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "## What I'm seeing\r\n\r\nWith `default_cache_ttl = 86400`, I see the following:\r\n\r\nA table view returns `Cache-control: max-age=86400`:\r\n\r\n![Screenshot_20220228_190000](https://user-images.githubusercontent.com/697092/156034352-4d64683e-39c8-49af-81df-0217a5957bbd.png)\r\n\r\nA static asset returns no `Cache-control` header:\r\n\r\n![Screenshot_20220228_185933](https://user-images.githubusercontent.com/697092/156034363-d0b03cc2-5889-4ed2-b601-8c1846b8469a.png)\r\n\r\n## What I expected to see\r\n\r\nI expected the static asset to return a `Cache-control` header indicating that this response can be cached.\r\n\r\n## Why this matters\r\n\r\nI'm productionising a Datasette deployment right now and was looking into putting it behind a Varnish instance. I was surprised to see requests for static assets being served from Datasette rather than Varnish, this is what led me to look more closely at the response headers.\r\n\r\nWhile Datasette serves those static assets pretty quickly, I don't see why Datasette should serve them. By their nature, static assets like images and JS files are very cacheable, so it should be easy to serve them from a cache like Varnish.\r\n\r\n(Note that Varnish can easily be configured to override this header, enabling caching for static assets. But it would be better if this override was not necessary.)\r\n\r\n## Discussion\r\n\r\nIt seems clear to me that serving static assets without a `Cache-control` header is not ideal.\r\n\r\nI see two options here:\r\n\r\nA. Static assets use the same logic as table / SQL views to set the `Cache-control` header based on `default_cache_ttl`.\r\nB. An additional setting for static assets is introduced (`default_static_cache_ttl`, say).", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1645/reactions\", \"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1198822563, "node_id": "I_kwDOBm6k_c5HdJSj", "number": 1706, "title": "[feature] immutable mode for a directory, not just individual sqlite file", "user": {"value": 9020979, "label": "hydrosquall"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2022-04-10T00:50:57Z", "updated_at": "2022-12-09T19:11:40Z", "closed_at": null, "author_association": "CONTRIBUTOR", "pull_request": null, "body": "## Motivation\r\n\r\n- I have a directory of sqlite databases\r\n- I'd like to use immutable mode when opening them for better performance [docs](https://docs.datasette.io/en/0.54/performance.html#immutable-mode)\r\n- Currently using this flag throws the following error\r\n\r\n IsADirectoryError: [Errno 21] Is a directory: '/name-of-directory'\r\n\r\n## Proposal\r\n\r\nImmutable flag works for both single files and directories\r\n\r\n datasette -i /folder-of-sqlite-files", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1706/reactions\", \"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1237586379, "node_id": "I_kwDOBm6k_c5JxBHL", "number": 1742, "title": "?_trace=1 fails with datasette-geojson for some reason", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2022-05-16T19:06:05Z", "updated_at": "2022-05-16T19:42:13Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "view-source:https://calands.datasettes.com/calands/CPAD_2020a_SuperUnits.geojson?_sort=id&id__exact=4&_labels=on&_trace=1 is showing me a blank page.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1742/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1326087800, "node_id": "PR_kwDOCGYnMM48hI-_", "number": 460, "title": "Cross-link CLI to Python docs", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2022-08-02T16:18:28Z", "updated_at": "2022-08-18T21:58:10Z", "closed_at": "2022-08-18T21:58:07Z", "author_association": "OWNER", "pull_request": "simonw/sqlite-utils/pulls/460", "body": "Work in progress, partly to test the ReadTheDocs preview link action.\r\n\r\nRefs:\r\n- #426\r\n\r\n\r\n----\n:books: Documentation preview :books:: https://readthedocs-preview--460.org.readthedocs.build/en/460/\n\r\n", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/460/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null} {"id": 1355433619, "node_id": "PR_kwDOCGYnMM4-B7Mc", "number": 480, "title": "search_sql add include_rank option", "user": {"value": 7908073, "label": "chapmanjacobd"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2022-08-30T09:10:29Z", "updated_at": "2022-08-31T03:40:35Z", "closed_at": "2022-08-31T03:40:35Z", "author_association": "CONTRIBUTOR", "pull_request": "simonw/sqlite-utils/pulls/480", "body": "I haven't tested this yet but wanted to get a heads-up whether this kind of change would be useful or if I should just duplicate the function and tweak it within my code\r\n\r\n\r\n----\n:books: Documentation preview :books:: https://sqlite-utils--480.org.readthedocs.build/en/480/\n\r\n", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/480/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null} {"id": 1373210675, "node_id": "I_kwDODD6af85R2Ygz", "number": 13, "title": "fails before generating views. ERR: table sqlite_master may not be modified", "user": {"value": 116795, "label": "pax"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2022-09-14T15:41:50Z", "updated_at": "2023-04-11T03:46:17Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "generates checkins.db but seems to fail before generating views \r\n\r\nnote: it worked on an Ubuntu WSL but fails on macOS 12.5.1\r\n\r\nlater edit: I suspect this is a problem with my local set-up, `dogsheep-beta index` also throws the same error\r\n\r\nfull error:\r\n\r\n Importing 2591 checkins [###################################-] 98% 00:00:00\r\n Traceback (most recent call last):\r\n File \"/Users/pax/devbox/envAll/bin/swarm-to-sqlite\", line 8, in \r\n sys.exit(cli())\r\n File \"/Users/pax/devbox/envAll/lib/python3.8/site-packages/click/core.py\", line 829, in __call__\r\n return self.main(*args, **kwargs)\r\n File \"/Users/pax/devbox/envAll/lib/python3.8/site-packages/click/core.py\", line 782, in main\r\n rv = self.invoke(ctx)\r\n File \"/Users/pax/devbox/envAll/lib/python3.8/site-packages/click/core.py\", line 1066, in invoke\r\n return ctx.invoke(self.callback, **ctx.params)\r\n File \"/Users/pax/devbox/envAll/lib/python3.8/site-packages/click/core.py\", line 610, in invoke\r\n return callback(*args, **kwargs)\r\n File \"/Users/pax/devbox/envAll/lib/python3.8/site-packages/swarm_to_sqlite/cli.py\", line 77, in cli\r\n ensure_foreign_keys(db)\r\n File \"/Users/pax/devbox/envAll/lib/python3.8/site-packages/swarm_to_sqlite/utils.py\", line 145, in ensure_foreign_keys\r\n db[fk.table].add_foreign_key(fk.column, fk.other_table, fk.other_column)\r\n File \"/Users/pax/devbox/envAll/lib/python3.8/site-packages/sqlite_utils/db.py\", line 2123, in add_foreign_key\r\n self.db.add_foreign_keys([(self.name, column, other_table, other_column)])\r\n File \"/Users/pax/devbox/envAll/lib/python3.8/site-packages/sqlite_utils/db.py\", line 1086, in add_foreign_keys\r\n cursor.execute(\r\n sqlite3.OperationalError: table sqlite_master may not be modified", "repo": {"value": 205429375, "label": "swarm-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/13/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1374626873, "node_id": "I_kwDOBm6k_c5R7yQ5", "number": 1810, "title": "Featured table(s) on the homepage", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2022-09-15T14:30:49Z", "updated_at": "2022-09-15T15:51:25Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "Many Datasette instances mainly exist to serve a single table - for example:\r\n\r\n- https://global-power-plants.datasettes.com/global-power-plants/global-power-plants\r\n- https://laion-aesthetic.datasette.io/laion-aesthetic-6pls/images\r\n\r\nIt would be neat if the / homepage of those instances could be configured to highlight that specific table.\r\n\r\nOr maybe more than one?", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1810/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1384549993, "node_id": "I_kwDOBm6k_c5Sho5p", "number": 1818, "title": "Setting to turn off table row counts entirely", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2022-09-24T06:39:22Z", "updated_at": "2022-12-11T02:03:09Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "There are situations - such as loading SQLite files remotely using HTTP range headers - where counting all of the rows in a table should be avoided entirely.\r\n\r\n> > Also, this chunked inefficiency means that I have to hack the URL to not load tables of a database as it seems to try to load the whole database when I click on a database.\r\n>\r\n> I bet that's because Datasette tries to show a count of all of the rows in each table when it shows the list on that page, which triggers a full table scan.\r\n>\r\n> Would be great to have a setting that turns that feature off, which could then be exposed as a query string option for Datasette Lite.\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/datasette-lite/issues/49#issuecomment-1256880715_\r\n ", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1818/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1393202060, "node_id": "I_kwDOCGYnMM5TCpOM", "number": 496, "title": "devrel/python api: Pylance type hinting", "user": {"value": 7908073, "label": "chapmanjacobd"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2022-10-01T03:03:34Z", "updated_at": "2023-05-03T05:53:27Z", "closed_at": null, "author_association": "CONTRIBUTOR", "pull_request": null, "body": "Pylance is generally pretty good at figuring out stuff but `sqlite-utils` has some quirks which make type hinting kinda useless. Maybe you don't care but I thought I would bring it to your attention.\r\n\r\nFor example:\r\n\r\n```\r\ndb[\"subs\"].insert_all(subs, pk=\"index\")\r\n```\r\n\r\n```\r\nCannot access member \"insert_all\" for type \"View\"\r\n Member \"insert_all\" is unknown\r\n```\r\n\r\n`insert_all` and all the other methods show up as a type issues because the program can't know whether something is a View or a Table. Fair enough. But that basically throws all type checking out the window.\r\n\r\n`pk=\"index\"` also shows up as a type issue:\r\n\r\n```\r\nArgument of type \"Literal['index']\" cannot be assigned to parameter \"pk\" of type \"Default\" in function \"insert_all\"\r\n \"Literal['index']\" is incompatible with \"Default\"\r\n```\r\n\r\nI think this is because DEFAULT is an empty class? \r\n\r\nmaybe a few small changes could be made to make the library more type-friendly\r\n\r\nThe interim solution is of course to turn off type hints completely for the line\r\n```\r\ndb[\"subs\"].insert_all(subs, pk=\"index\") # type: ignore\r\n```\r\n", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/496/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1433576351, "node_id": "I_kwDOBm6k_c5VcqOf", "number": 1880, "title": "Datasette with many and large databases > Memory use", "user": {"value": 525934, "label": "amitkoth"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2022-11-02T18:10:27Z", "updated_at": "2022-11-16T17:50:29Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "> Datasette maintains an in-memory SQLite database with details of the the databases, tables and columns for all of the attached databases.\r\n\r\nThe above is from the docs ^. There's two problems here - the number of datasette \"instances\" in a single server/VM and the size of the database itself. We want the **opposite** of in-memory, including what happens on SQLlite - documented in https://www.sqlite.org/inmemorydb.html\r\n\r\nFrom the context in https://github.com/simonw/datasette/issues/1150 - does it mean datasette is memory-bound to the size of the dataset - which might be a deal-breaker for many large-scale use cases?\r\n\r\nIn an extreme case - let's say a single server had 100 SQLlite databases, which would enable 100 \"instances\" of datasette to run, one per client (e.g. in a SaaS multi-tenant environment). How could we achieve all these goals:\r\n\r\n1. Allow any _one_ of these 100 databases to grow to say 2Tb in size \r\n2. Have one datasette instance, which connects to 1 of the 100 instances, based on incoming credentials/tenant ID\r\n3. Minimize memory use entirely - both by datasette and SQLlite, such that almost all operations are executed in real-time on-disk with little to no memory consumption per-tenant, or per-database.\r\n\r\nAny ideas appreciated - we're looking to use this in a SaaS type of setting - many instances, single server.\r\n\r\n@simonw great work on datasette, in general! Possibly related to https://github.com/simonw/datasette/issues/1480 but we don't want use any kind of serverless infra - this is a long-running VM/server.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1880/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1479914599, "node_id": "I_kwDOCGYnMM5YNbRn", "number": 516, "title": "Feature request: output number of ignored/replaced rows for insert command", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2022-12-06T18:59:21Z", "updated_at": "2022-12-06T19:08:14Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "https://hachyderm.io/@briandorsey/109468185742876820\r\n\r\n> I'm fiddling with piping json to `insert -ignore` I'd love to see the count of records inserted & ignored, but didn't see a way to do that in the help/docs.\r\n>\r\n> Example: `xh \"https://hachyderm.io/api/v1/timelines/tag/rust?max_id=109443380308326328\" | sqlite-utils insert aoc.db aoc - --pk=id --ignore`", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/516/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1515815014, "node_id": "I_kwDOBm6k_c5aWYBm", "number": 1973, "title": "render_cell plugin hook's row object is not a sqlite.Row", "user": {"value": 193185, "label": "cldellow"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2023-01-01T20:27:46Z", "updated_at": "2023-01-29T00:40:31Z", "closed_at": null, "author_association": "CONTRIBUTOR", "pull_request": null, "body": "From https://docs.datasette.io/en/stable/plugin_hooks.html#render-cell-row-value-column-table-database-datasette:\r\n\r\n> row - sqlite.Row\r\n> The SQLite row object that the value being rendered is part of\r\n\r\nThis appears to actually be a [CustomRow](https://github.com/simonw/datasette/blob/f0fadc28ddb9f82e5cc1ecaa51e8a342eb6dc528/datasette/utils/__init__.py#L773-L789), but I think that's unrelated to my issue.\r\n\r\nI have a table:\r\n\r\n```sql\r\nCREATE TABLE IF NOT EXISTS \"dss_job_stats\"(\r\n job_id integer not null references dss_job(id) on delete cascade,\r\n host text not null,\r\n // other columns elided as irrelevant\r\n primary key (job_id, host)\r\n);\r\n```\r\n\r\nOn datasette 0.63.2, the `render_cell` hook receives a `row` value that looks like:\r\n\r\n```\r\nCustomRow([('job_id', {'value': 2, 'label': '2'}), ('host', 'cldellow.com')])\r\n```\r\n\r\nI expected the `job_id` value to be `2`, but it's actually `{'value': 2, 'label': '2'}`.\r\n\r\nI can work around this, but was wondering if this was intended behaviour?", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1973/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1529707837, "node_id": "I_kwDOBm6k_c5bLX09", "number": 1988, "title": "Reconsider pattern where plugins could break existing template context", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": {"value": 3268330, "label": "Datasette 1.0"}, "comments": 4, "created_at": "2023-01-11T21:13:43Z", "updated_at": "2023-01-11T21:25:05Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "> I hadn't run into an issue with plugins like `datasette-template-sql` interfering with the existing context for other features before! Definitely not a good thing.\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/datasette-write/issues/6#issuecomment-1379490596_\r\n ", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1988/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1553615704, "node_id": "I_kwDOBm6k_c5cmktY", "number": 2001, "title": "Datasette is not compatible with SQLite's strict quoting compilation option", "user": {"value": 406380, "label": "gwk"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2023-01-23T19:10:07Z", "updated_at": "2023-01-25T04:59:58Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "I have linked Python3.11 on macOS against recent SQLite that was compiled using `-DSQLITE_DQS=0`. This option disables interpretation of double-quoted identifiers as string literals, described in the SQLite docs as a \"MySQL 3.x misfeature\". See https://www.sqlite.org/quirks.html#dblquote for background.\r\n\r\nDatasette uses the double-quote syntax in a number of key places, and is thus completely broken in this environment.\r\n\r\nMy experience was to `pip install datasette`, then run `datasette serve -I my-data.db`. When I visit `http://127.0.0.1:8001` I get a 500 response.\r\n\r\nThe error: `sqlite3.OperationalError: no such column: geometry_columns`\r\n\r\nThe responsible SQL: `'select 1 from sqlite_master where tbl_name = \"geometry_columns\"'`\r\n\r\nI then installed datasette from GitHub master in development mode and changed the offending SQL to use correct quotes: `\"select 1 from sqlite_master where tbl_name = 'geometry_columns'\"`.\r\n\r\nWith this change, I get a little further, but have the same problem with the first table name in my database (in my case, \"Meta\"):\r\n```\r\nOperationalError: no such column: Meta\r\nTraceback (most recent call last):\r\n File \"/Users/gwk/external/datasette/datasette/app.py\", line 1522, in route_path\r\n response = await view(request, send)\r\n ^^^^^^^^^^^^^^^^^^^^^^^^^\r\n File \"/Users/gwk/external/datasette/datasette/views/base.py\", line 151, in view\r\n return await self.dispatch_request(request)\r\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\r\n File \"/Users/gwk/external/datasette/datasette/views/base.py\", line 105, in dispatch_request\r\n response = await handler(request)\r\n ^^^^^^^^^^^^^^^^^^^^^^\r\n File \"/Users/gwk/external/datasette/datasette/views/index.py\", line 70, in get\r\n \"fts_table\": await db.fts_table(table),\r\n ^^^^^^^^^^^^^^^^^^^^^^^^^\r\n File \"/Users/gwk/external/datasette/datasette/database.py\", line 363, in fts_table\r\n return await self.execute_fn(lambda conn: detect_fts(conn, table))\r\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\r\n File \"/Users/gwk/external/datasette/datasette/database.py\", line 213, in execute_fn\r\n return await asyncio.get_event_loop().run_in_executor(\r\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\r\n File \"/usr/local/py/Python.framework/Versions/3.11/lib/python3.11/concurrent/futures/thread.py\", line 58, in run\r\n result = self.fn(*self.args, **self.kwargs)\r\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\r\n File \"/Users/gwk/external/datasette/datasette/database.py\", line 211, in in_thread\r\n return fn(conn)\r\n ^^^^^^^^\r\n File \"/Users/gwk/external/datasette/datasette/database.py\", line 363, in \r\n return await self.execute_fn(lambda conn: detect_fts(conn, table))\r\n ^^^^^^^^^^^^^^^^^^^^^^^\r\n File \"/Users/gwk/external/datasette/datasette/utils/__init__.py\", line 588, in detect_fts\r\n rows = conn.execute(detect_fts_sql(table)).fetchall()\r\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\r\nsqlite3.OperationalError: no such column: Meta\r\nINFO: 127.0.0.1:50258 - \"GET / HTTP/1.1\" 500 Internal Server Error\r\n```\r\n\r\nI will try to continue playing with this, but I also hope that the datasette developers will enable this mode in a test environment as I am unlikely to be able to exercise all of the SQL in the codebase, or make a pull request very soon.\r\n\r\nNote that the DQS setting compile-time option can be overridden at runtime with calls to the C API:\r\n```\r\nsqlite3_db_config(db, SQLITE_DBCONFIG_DQS_DDL, 0, (void*)0);\r\nsqlite3_db_config(db, SQLITE_DBCONFIG_DQS_DML, 0, (void*)0);\r\n```\r\n\r\nAs far as I can tell, `sqlite3_db_config` is not exposed in Python, but perhaps we could figure out how to invoke it using `ctypes`.\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/2001/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1590183272, "node_id": "I_kwDOBm6k_c5eyEVo", "number": 2027, "title": "How to redirect from \"/\" to a specific db/table", "user": {"value": 1350673, "label": "dmick"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2023-02-18T03:14:01Z", "updated_at": "2023-03-08T04:42:22Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "Using nginx to redirect public IP to the local uvicorn server as 'normal'. I can't figure out how to redirect such that '/' results in accessing the one db/table I want to serve; redirecting / to /db/table breaks some of the CSS; fooling with base_url doesn't seem to help. Can someone explain this, if it's possible?", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/2027/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1620164673, "node_id": "PR_kwDOCGYnMM5L08O8", "number": 531, "title": "Add paths for homebrew on Apple silicon", "user": {"value": 25778, "label": "eyeseast"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2023-03-11T22:27:52Z", "updated_at": "2023-04-09T01:49:44Z", "closed_at": "2023-04-09T01:49:43Z", "author_association": "CONTRIBUTOR", "pull_request": "simonw/sqlite-utils/pulls/531", "body": "This also passes in the extension path when specified in GIS methods. Wherever we know an extension path, we use `db.init_spatialite(find_spatialite() or load_extension)`.\r\n\r\n\r\n----\n:books: Documentation preview :books:: https://sqlite-utils--531.org.readthedocs.build/en/531/\n\r\n", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/531/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null} {"id": 1646734246, "node_id": "I_kwDOBm6k_c5iJyum", "number": 2049, "title": "Custom SQL queries should use new JSON ?_extra= format", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": {"value": 8755003, "label": "Datasette 1.0a-next"}, "comments": 4, "created_at": "2023-03-30T00:42:53Z", "updated_at": "2023-04-05T23:29:27Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "Related:\r\n- #262\r\n\r\nI've made the change to the table view, now I need the new format to work for arbitrary SQL queries too.\r\n\r\nNote that this incorporates both arbitrary SQL queries and canned queries.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/2049/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1816917522, "node_id": "PR_kwDOCGYnMM5WJ6Jm", "number": 573, "title": "feat: Implement a prepare_connection plugin hook", "user": {"value": 15178711, "label": "asg017"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2023-07-22T22:48:44Z", "updated_at": "2023-07-22T22:59:09Z", "closed_at": "2023-07-22T22:59:09Z", "author_association": "CONTRIBUTOR", "pull_request": "simonw/sqlite-utils/pulls/573", "body": "Just like the [Datasette prepare_connection hook](https://docs.datasette.io/en/stable/plugin_hooks.html#prepare-connection-conn-database-datasette), this PR adds a similar hook for the `sqlite-utils` plugin system. \r\n\r\nThe sole argument is `conn`, since I don't believe a `database` or `datasette` argument would be relevant here. \r\n\r\nI want to do this so I can release `sqlite-utils` plugins for my [SQLite extensions](https://github.com/asg017/sqlite-ecosystem), similar to the Datasette plugins I've release for them. \r\n\r\nAn example plugin: https://gist.github.com/asg017/d7cdf0d56e2be87efda28cebee27fa3c\r\n\r\n```bash\r\n$ sqlite-utils install https://gist.github.com/asg017/d7cdf0d56e2be87efda28cebee27fa3c/archive/5f5ad549a40860787629c69ca120a08c32519e99.zip\r\n\r\n$ sqlite-utils memory 'select hello(\"alex\") as response'\r\n[{\"response\": \"Hello, alex!\"}]\r\n```\r\nRefs:\r\n- #574 \r\n\r\n\r\n----\r\n:books: Documentation preview :books:: https://sqlite-utils--573.org.readthedocs.build/en/573/\r\n\r\n", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/573/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null} {"id": 1865649347, "node_id": "I_kwDOBm6k_c5vM4zD", "number": 2156, "title": "datasette -s/--setting option for setting nested configuration options", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2023-08-24T18:09:27Z", "updated_at": "2023-08-28T19:33:05Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "> I've been thinking about what it might look like to allow command-line arguments to be used to define _any_ of the configuration options in `datasette.yml`, as alternative and more convenient syntax.\r\n>\r\n> Here's what I've come up with:\r\n> ```\r\n> datasette \\\r\n> -s settings.sql_time_limit_ms 1000 \\\r\n> -s plugins.datasette-auth-tokens.manage_tokens true \\\r\n> -s plugins.datasette-auth-tokens.manage_tokens_database tokens \\\r\n> mydatabase.db tokens.db\r\n> ```\r\n> Which would be equivalent to `datasette.yml` containing this:\r\n> ```yaml\r\n> plugins:\r\n> datasette-auth-tokens:\r\n> manage_tokens: true\r\n> manage_tokens_database: tokens\r\n> settings:\r\n> sql_time_limit_ms: 1000\r\n> ```\r\nMore details in https://github.com/simonw/datasette/issues/2143#issuecomment-1690792514\r\n ", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/2156/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1870672704, "node_id": "PR_kwDOBm6k_c5Y-7Em", "number": 2162, "title": "Add new `--internal internal.db` option, deprecate legacy `_internal` database", "user": {"value": 15178711, "label": "asg017"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2023-08-29T00:05:07Z", "updated_at": "2023-08-29T03:24:23Z", "closed_at": "2023-08-29T03:24:23Z", "author_association": "CONTRIBUTOR", "pull_request": "simonw/datasette/pulls/2162", "body": "refs #2157 \r\n\r\nThis PR adds a new `--internal` option to datasette serve. If provided, it is the path to a persistent internal database that Datasette core and Datasette plugins can use to store data, as discussed in the proposal issue. \r\n\r\nThis PR also removes and deprecates the previous in-memory `_internal` database. Those tables now appear in the `internal` database, with `core_` prefixes (ex `tables` in `_internal` is now `core_tables` in `internal`).\r\n\r\n\r\n## A note on the new `core_` tables\r\nHowever, one important notes about those new `core_` tables: If a `--internal` DB is passed in, that means those `core_` tables will persist across multiple Datasette instances. This wasn't the case before, since `_internal` was always an in-memory database created from scratch.\r\n\r\nI tried to put those `core_` tables as `TEMP` tables - after all, there's always one 1 `internal` DB connection at a time, so I figured it would work. But, since we use the `Database()` wrapper for the internal DB, it has two separate connections: a default read-only connection and a write connection that is created when a write operation occurs. Which meant the `TEMP` tables would be created by the write connection, but not available in the read-only connection. \r\n\r\nSo I had a brillant idea: Attach an in-memory named database with `cache=shared`, and create those tables there! \r\n\r\n```sql\r\nATTACH DATABASE 'file:datasette_internal_core?mode=memory&cache=shared' AS core;\r\n```\r\n\r\nWe'd run this on both the read-only connection and the write-only connection. That way, those tables would stay in memory, they'd communicate with the `cache=shared` feature, and we'd be good to go.\r\n\r\n\r\nHowever, I couldn't find an easy way to run a `ATTACH DATABASE` command on the read-only query. \r\n\r\nUsing `Database()` as a wrapper for the internal DB is pretty limiting - it's meant for Datasette \"data\" databases, where we want multiple readers and possibly 1 write connection at a time. But the internal database doesn't really require that kind of support - I think we could get away with a single read/write connection, but it seemed like too big of a rabbithole to go through now. \r\n\r\n\r\n\r\n----\r\n:books: Documentation preview :books:: https://datasette--2162.org.readthedocs.build/en/2162/\r\n\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/2162/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null} {"id": 1891212159, "node_id": "PR_kwDOBm6k_c5aD33C", "number": 2183, "title": "`datasette.yaml` plugin support", "user": {"value": 15178711, "label": "asg017"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2023-09-11T20:26:04Z", "updated_at": "2023-09-13T21:06:25Z", "closed_at": "2023-09-13T21:06:25Z", "author_association": "CONTRIBUTOR", "pull_request": "simonw/datasette/pulls/2183", "body": "Part of #2093\r\n\r\nIn #2149 , we ported over `\"settings.json\"` into the new `datasette.yaml` config file, with a top-level `\"settings\"` key. This PR ports over plugin configuration into top-level `\"plugins\"` key, as well as nested database/table plugin config.\r\n\r\nFrom now on, no plugin-related configuration is allowed in `metadata.yaml`, and must be in `datasette.yaml` in this new format. This is a pretty significant breaking change. Thankfully, you should be able to copy-paste your legacy plugin key/values into the new `datasette.yaml` format.\r\n\r\nAn example of what `datasette.yaml` would look like with this new plugin config:\r\n\r\n```yaml\r\n\r\nplugins:\r\n datasette-my-plugin:\r\n config_key: value\r\n\r\ndatabases:\r\n fixtures:\r\n plugins: \r\n datasette-my-plugin:\r\n config_key: fixtures-db-value\r\n tables:\r\n students:\r\n plugins:\r\n datasette-my-plugin:\r\n config_key: fixtures-students-table-value\r\n\r\n```\r\n\r\nAs an additional benefit, this now works with the new `-s` flag:\r\n\r\n```bash\r\ndatasette --memory -s 'plugins.datasette-my-plugin.config_key' new_value\r\n```\r\n\r\n\r\nMarked as a \"Draft\" right now until I add better documentation. We also should have a plan for the next alpha release to document and publicize this change, especially for plugin authors (since their docs will have to change to say `datasette.yaml` instead of `metadata.yaml`\r\n\r\n\r\n----\n:books: Documentation preview :books:: https://datasette--2183.org.readthedocs.build/en/2183/\n\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/2183/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null} {"id": 1901768721, "node_id": "PR_kwDOBm6k_c5anSg5", "number": 2191, "title": "Move `permissions`, `allow` blocks, canned queries and more out of `metadata.yaml` and into `datasette.yaml`", "user": {"value": 15178711, "label": "asg017"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2023-09-18T21:21:16Z", "updated_at": "2023-10-12T16:16:38Z", "closed_at": "2023-10-12T16:16:38Z", "author_association": "CONTRIBUTOR", "pull_request": "simonw/datasette/pulls/2191", "body": "The PR moves the following fields from `metadata.yaml` to `datasette.yaml`:\r\n\r\n```\r\npermissions\r\nallow\r\nallow_sql\r\nqueries\r\nextra_css_urls\r\nextra_js_urls\r\n```\r\n\r\nThis is a significant breaking change that users will need to upgrade their `metadata.yaml` files for. But the format/locations are similar to the previous version, so it shouldn't be too difficult to upgrade.\r\n\r\nOne note: I'm still working on the Configuration docs, specifically the \"reference\" section. Though it's pretty small, the rest of read to review", "repo": {"value": 107914493, "label": "datasette"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/2191/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null} {"id": 2001006157, "node_id": "PR_kwDOCGYnMM5f2OZC", "number": 604, "title": "Add more STRICT table support", "user": {"value": 16437338, "label": "tkhattra"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2023-11-19T19:38:53Z", "updated_at": "2023-12-08T05:17:20Z", "closed_at": "2023-12-08T05:05:27Z", "author_association": "CONTRIBUTOR", "pull_request": "simonw/sqlite-utils/pulls/604", "body": "- https://github.com/simonw/sqlite-utils/issues/344#issuecomment-982014776\r\n\r\nMake `table.transform()` preserve STRICT mode.\r\n\r\n\r\n----\r\n:books: Documentation preview :books:: https://sqlite-utils--604.org.readthedocs.build/en/604/\r\n\r\n", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/604/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null} {"id": 268469569, "node_id": "MDU6SXNzdWUyNjg0Njk1Njk=", "number": 39, "title": "Protect against malicious SQL that causes damage even though our DB is immutable", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 2857392, "label": "Ship first public release"}, "comments": 4, "created_at": "2017-10-25T16:44:27Z", "updated_at": "2021-08-17T23:52:07Z", "closed_at": "2017-11-05T02:53:47Z", "author_association": "OWNER", "pull_request": null, "body": "I\u2019m currently operating under the assumption that it\u2019s safe to allow arbitrary SQL statements because we are dealing with an immutable database. But this might not be the case - there are some pretty weird SQLite language extensions (ATTACH, PRAGMA etc) and I\u2019m not certain they cannot be used to break things in a way that would affect future requests to the API.\r\n\r\nSolution: provide a \u201csafe mode\u201d option which disables the ?sql= mechanism. This still leaves the URL filter lookups, so I need to make sure that those are \u201csafe\u201d.\r\n\r\nIn the future I may also implement a whitelist option where datasets can be configured to only allow specific filters against specific columns.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/39/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 268591332, "node_id": "MDU6SXNzdWUyNjg1OTEzMzI=", "number": 42, "title": "Homepage UI for editing metadata file", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2017-10-26T00:22:03Z", "updated_at": "2017-12-10T03:02:14Z", "closed_at": "2017-12-10T03:02:14Z", "author_association": "OWNER", "pull_request": null, "body": "Since we are going to have a metadata file which sets the title/description/etc for each database, why not allow you to run the app in \u2014dev mode which makes the homepage into a WYSIWYG editor that can save to that file format.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/42/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 272391665, "node_id": "MDU6SXNzdWUyNzIzOTE2NjU=", "number": 48, "title": "Switch to ujson", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2017-11-08T23:50:29Z", "updated_at": "2019-06-24T06:57:54Z", "closed_at": "2019-06-24T06:57:43Z", "author_association": "OWNER", "pull_request": null, "body": "ujson is already a dependency of Sanic, and should be quite a bit faster.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/48/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 272661336, "node_id": "MDU6SXNzdWUyNzI2NjEzMzY=", "number": 49, "title": "Pick a name", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 2857392, "label": "Ship first public release"}, "comments": 4, "created_at": "2017-11-09T17:56:17Z", "updated_at": "2017-11-10T18:33:22Z", "closed_at": "2017-11-10T18:33:22Z", "author_association": "OWNER", "pull_request": null, "body": "Options so far:\r\n\r\n* immutabase\r\n* datasite\r\n* sqlstatic\r\n* dbserve\r\n* sqlserve\r\n\r\nTerms to play with:\r\n\r\n* immutable\r\n* sqlite\r\n* dataset\r\n* json\r\n* static\r\n* serve", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/49/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 273157085, "node_id": "MDU6SXNzdWUyNzMxNTcwODU=", "number": 59, "title": "datasette publish hyper", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2017-11-11T16:27:26Z", "updated_at": "2019-05-13T19:01:00Z", "closed_at": "2019-05-13T19:00:44Z", "author_association": "OWNER", "pull_request": null, "body": "This is a bit tricky, because unlike Now there doesn't seem to be a way to tell Hyper to \"build this Dockerfile and deploy the resulting image\". They expect you to build a container and publish it to a registry instead.\r\n\r\nhttps://docs.hyper.sh/Reference/CLI/load.html allows you to publish an image directly from a tarball, but that still leaves the challenge of creating that image. The nice thing about the Now integration is that you don't need to have Docker installed on your local machine.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/59/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 273247186, "node_id": "MDU6SXNzdWUyNzMyNDcxODY=", "number": 68, "title": "Support for title/source/license metadata", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 2857392, "label": "Ship first public release"}, "comments": 4, "created_at": "2017-11-12T17:04:21Z", "updated_at": "2017-12-04T04:55:43Z", "closed_at": "2017-11-13T15:26:11Z", "author_association": "OWNER", "pull_request": null, "body": "I've decided this is important for launch: I want to set a precedent for people citing, licensing and documenting their datasets.\r\n\r\nNot sure how best to go about supporting this. I'd like to allow for the following data to be optionally attached to any given database:\r\n\r\n- Title\r\n- Description, potentially in markdown?\r\n- Original source URL\r\n- License\r\n\r\nI'd also like the ability to attach descriptions to individual tables - and maybe even to table columns?\r\n\r\nThe question then becomes: how should this information be stored. A few options:\r\n\r\n- In the SQLite database itself, in a specially named table. Problem here is that this means having to modify SQLite databases before publishing them.\r\n- In a separate SQLite database that can be published alongside the databases we are publishing.\r\n- In a JSON file. This is neat, but JSON files are not a great editing experience once you start including multiple lines (e.g. a markdown description).\r\n- In a YAML file. This is a better format for multi-line descriptions, but still isn't a great editing experience.\r\n\r\nWhatever the format, it can be made much more usable by offering a web-based editing UI for populating it (a special mode the server can be run in).", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/68/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 273248366, "node_id": "MDU6SXNzdWUyNzMyNDgzNjY=", "number": 69, "title": "Enforce pagination (or at least limits) for arbitrary custom SQL", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 2857392, "label": "Ship first public release"}, "comments": 4, "created_at": "2017-11-12T17:21:33Z", "updated_at": "2017-11-13T20:32:47Z", "closed_at": "2017-11-13T19:35:47Z", "author_association": "OWNER", "pull_request": null, "body": "It's way too easy to accidentally trigger a page that returns 100,000 rows at the moment. I need to use the LIMIT clause on views and custom SQL - I can support pagination \"next\" links using offset as well.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/69/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 273775212, "node_id": "MDU6SXNzdWUyNzM3NzUyMTI=", "number": 88, "title": "Add NHS England Hospitals example to wiki", "user": {"value": 15543, "label": "tomdyson"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2017-11-14T12:29:10Z", "updated_at": "2021-03-22T23:46:36Z", "closed_at": "2017-11-14T22:54:06Z", "author_association": "CONTRIBUTOR", "pull_request": null, "body": "https://nhs-england-hospitals.now.sh\r\n\r\nand an associated map visualisation:\r\n\r\nhttp://run.plnkr.co/preview/cj9zlf1qc0003414y90ajkwpk/\r\n\r\nDatasette is wonderful!\r\n\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/88/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 274314940, "node_id": "MDU6SXNzdWUyNzQzMTQ5NDA=", "number": 105, "title": "Consider data-package as a format for metadata", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2017-11-15T21:43:34Z", "updated_at": "2017-11-20T19:50:53Z", "closed_at": "2017-11-20T19:50:53Z", "author_association": "OWNER", "pull_request": null, "body": "http://frictionlessdata.io/specs/data-package/", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/105/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 275087397, "node_id": "MDU6SXNzdWUyNzUwODczOTc=", "number": 120, "title": "Plugin that adds an authentication layer of some sort", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2017-11-18T15:39:13Z", "updated_at": "2020-03-16T18:48:06Z", "closed_at": "2020-03-16T18:48:06Z", "author_association": "OWNER", "pull_request": null, "body": "Would allow people who want to host private data to do so.\r\n.sh ", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/120/reactions\", \"total_count\": 7, \"+1\": 5, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 2, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 275089535, "node_id": "MDU6SXNzdWUyNzUwODk1MzU=", "number": 121, "title": "?_json=foo&_json=bar query string argument ", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2017-11-18T16:09:55Z", "updated_at": "2018-05-31T13:48:12Z", "closed_at": "2018-05-28T18:11:51Z", "author_association": "OWNER", "pull_request": null, "body": "Causes the specified columns in the output to be treated as JSON, and returned deserialized in the .json or .jsono response.\r\n\r\nThis will be particularly powerful when combined with https://sqlite.org/json1.html", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/121/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 277589569, "node_id": "MDU6SXNzdWUyNzc1ODk1Njk=", "number": 155, "title": "A primary key column that has foreign key restriction associated won't rendering label column", "user": {"value": 388154, "label": "wsxiaoys"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 2949431, "label": "Custom templates edition"}, "comments": 4, "created_at": "2017-11-29T00:40:02Z", "updated_at": "2017-12-07T05:39:53Z", "closed_at": "2017-12-07T05:39:53Z", "author_association": "NONE", "pull_request": null, "body": "", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/155/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 278814220, "node_id": "MDU6SXNzdWUyNzg4MTQyMjA=", "number": 161, "title": "Support WITH query ", "user": {"value": 388154, "label": "wsxiaoys"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2017-12-03T20:00:40Z", "updated_at": "2017-12-08T06:18:12Z", "closed_at": "2017-12-04T04:52:41Z", "author_association": "NONE", "pull_request": null, "body": "Currently datasettle failed with error message: Statement must begin with SELECT\r\n\r\nExample query\r\n```sql\r\nWITH RECURSIVE\r\n cnt(x) AS (\r\n SELECT 1\r\n UNION ALL\r\n SELECT x+1 FROM cnt\r\n LIMIT 1000000\r\n )\r\nSELECT x FROM cnt;\r\n```", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/161/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 292011379, "node_id": "MDU6SXNzdWUyOTIwMTEzNzk=", "number": 184, "title": "500 from missing table name", "user": {"value": 222245, "label": "carlmjohnson"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2018-01-26T19:46:45Z", "updated_at": "2019-05-21T16:17:29Z", "closed_at": "2018-04-13T18:18:59Z", "author_association": "NONE", "pull_request": null, "body": "https://github.com/simonw/datasette/blob/56623e48da5412b25fb39cc26b9c743b684dd968/datasette/app.py#L517-L519 throws an error if it gets an empty list back. Simplest solution is to write a helper func that just says \r\n\r\n```python\r\nresult = list(await self.execute(name, sql, params)\r\nif result:\r\n return result[0][0]\r\n```\r\n\r\nand use it anywhere `[0][0]` is now.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/184/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 312620566, "node_id": "MDU6SXNzdWUzMTI2MjA1NjY=", "number": 199, "title": "Ability to apply sort on mobile in portrait mode", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2018-04-09T17:35:04Z", "updated_at": "2018-04-10T00:37:53Z", "closed_at": "2018-04-10T00:34:38Z", "author_association": "OWNER", "pull_request": null, "body": "Missed this in #189... on mobile in portrait mode we hide the column headers, which means you can't click them to sort! You can sort in landscape mode at least.\r\n\r\nNeed to come up with an alternative sort UI for portrait on mobile.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/199/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 316323336, "node_id": "MDU6SXNzdWUzMTYzMjMzMzY=", "number": 231, "title": "metadata.json support for plugin configuration options", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2018-04-20T15:58:47Z", "updated_at": "2019-05-13T18:56:21Z", "closed_at": "2019-05-13T18:56:21Z", "author_association": "OWNER", "pull_request": null, "body": "My [datasette-cluster-map](https://github.com/simonw/datasette-cluster-map) plugin currently works by detecting `latitude` and `longitude` columns. I'd like to be able to configure it to look for different column names.\r\n\r\nOne way to do this could be to support optional plugin configuration as part of `metadata.json`. Something like this:\r\n\r\n {\r\n \"title\": \"Polar Bear Ear Tags, 2009-2011\",\r\n \"source\": \"USGS Alaska Science Center, Polar Bear Research Program\",\r\n \"source_url\": \"https://alaska.usgs.gov/products/data.php?dataid=130\",\r\n \"plugins\": {\r\n \"datasette_cluster_map\": {\r\n \"latitude_columns\": [\r\n \"latitude\",\r\n \"Capture Latitude\"\r\n ],\r\n \"longitude_columns\": [\r\n \"longitude\",\r\n \"Capture Longitude\"\r\n ]\r\n }\r\n }\r\n }\r\n\r\nThese settings should be supported at the root level or at the individual database or table level.\r\n\r\nThey could also be exposed in the https://datasette-cluster-map-demo.now.sh/-/plugins debug tool.\r\n\r\nRefs #14", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/231/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 320592643, "node_id": "MDU6SXNzdWUzMjA1OTI2NDM=", "number": 251, "title": "Explore \"distinct values for column\" in inspect()", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2018-05-06T13:27:24Z", "updated_at": "2018-05-14T22:47:55Z", "closed_at": "2018-05-14T22:47:55Z", "author_association": "OWNER", "pull_request": null, "body": "A lot of datasets have columns which have a small number of possible values in them - this one for example: https://fivethirtyeight.datasettes.com/fivethirtyeight-2628db9?sql=select+distinct+category+from+%5Binconvenient-sequel%2Fratings%5D%3B\r\n\r\nDetecting these could be interesting as part of `.inspect()`, since it would allow for various UI enhancements like autocomplete / select box filters for those columns.\r\n\r\nThe problem is detecting them efficiently. `.inspect()` shouldn't spend 5 minutes churning through columns on giant tables trying to determine if they have a small collection of unique values.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/251/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 322283067, "node_id": "MDU6SXNzdWUzMjIyODMwNjc=", "number": 254, "title": "Escaping named parameters in canned queries", "user": {"value": 247131, "label": "philroche"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2018-05-11T12:43:30Z", "updated_at": "2020-05-10T14:54:14Z", "closed_at": "2020-05-10T14:54:13Z", "author_association": "NONE", "pull_request": null, "body": "Thank you very much for this project.\r\n\r\nI have created some canned queries but some of the filters include a colon eg. \"com.ubuntu.cloud:server:18.04:amd64\". When saved these colons are parsed as named parameters. \r\n\r\nIs there a way to escape colons in a canned query?", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/254/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 324162476, "node_id": "MDU6SXNzdWUzMjQxNjI0NzY=", "number": 271, "title": "Mechanism for automatically picking up changes when on-disk .db file changes", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2018-05-17T19:53:15Z", "updated_at": "2019-01-10T21:35:18Z", "closed_at": "2019-01-10T21:35:18Z", "author_association": "OWNER", "pull_request": null, "body": "It would be useful if Datasette could spot when a SQLite database file changes on disk and restart itself (hence re-running .inspect() and picking up the new content hash). Ideally this could happen in an atomic way so no requests get dropped during the switch-over.\r\n\r\nThis may not play well with SQLite opening databases in immutable mode. Research required.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/271/reactions\", \"total_count\": 2, \"+1\": 2, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 328172521, "node_id": "MDU6SXNzdWUzMjgxNzI1MjE=", "number": 303, "title": "Support table names ending with .json or .csv", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2018-05-31T14:53:23Z", "updated_at": "2018-06-15T06:55:50Z", "closed_at": "2018-06-15T06:55:50Z", "author_association": "OWNER", "pull_request": null, "body": "This is needed for #266 - if a table name ends with `.json` or `.csv` right now our URL pattern matching will do the wrong thing.\r\n\r\nWe should be smarter about this. This does mean we will have some URLs that look like this:\r\n\r\n http://localhost:8001/dbname/weird.json - returning HTML, not JSON\r\n http://localhost:8001/dbname/weird.json.json - returning JSON\r\n http://localhost:8001/dbname/weird.json.csv - returning CSV\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/303/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 336464733, "node_id": "MDU6SXNzdWUzMzY0NjQ3MzM=", "number": 328, "title": "Installation instructions, including how to use the docker image", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2018-06-28T03:59:33Z", "updated_at": "2023-09-05T14:10:39Z", "closed_at": "2018-06-28T04:02:10Z", "author_association": "OWNER", "pull_request": null, "body": "", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/328/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 336924199, "node_id": "MDU6SXNzdWUzMzY5MjQxOTk=", "number": 330, "title": "Limit text display in cells containing large amounts of text", "user": {"value": 82988, "label": "psychemedia"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2018-06-29T09:15:22Z", "updated_at": "2018-07-24T04:53:20Z", "closed_at": "2018-07-10T16:20:48Z", "author_association": "CONTRIBUTOR", "pull_request": null, "body": "The default preview of a database shows all columns (is the row count limited?) which is fine in many cases but can take a long time to load / offer a large overhead if the table is a SpatiaLite table containing geometry columns that include large shapefiles.\r\n\r\nWould it make sense to have a setting that can limit the amount of text displayed in any given cell in the table preview, or (less useful?) suppress (with notification) the display of overlong columns unless enabled by the user?\r\n\r\nAn issue then arises if a user does want to see all the text in a cell:\r\n\r\n 1) for a particular cell;\r\n 2) for every cell in the table;\r\n 3) for all cells in a particular column or columns\r\n\r\n(I haven't checked but what if a column contains e.g. raw image data? Does this display as raw data? Or can this be rendered in a context aware way as an image preview? I guess a custom template would be one way to do that?)", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/330/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 338768551, "node_id": "MDU6SXNzdWUzMzg3Njg1NTE=", "number": 333, "title": "Datasette on Zeit Now returns http URLs for facet and next links", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2018-07-06T00:40:49Z", "updated_at": "2018-07-24T04:53:20Z", "closed_at": "2018-07-24T01:51:53Z", "author_association": "OWNER", "pull_request": null, "body": "e.g. on https://fivethirtyeight.datasettes.com/fivethirtyeight-ac35616/nba-elo%2Fnbaallelo.json?_facet=lg_id&_size=0\r\n```\r\n{\r\n \"facet_results\": {\r\n \"lg_id\": {\r\n \"name\": \"lg_id\",\r\n \"results\": [\r\n {\r\n \"value\": \"NBA\",\r\n \"label\": \"NBA\",\r\n \"count\": 118016,\r\n \"toggle_url\": \"http://fivethirtyeight.datasettes.com/fivethirtyeight-ac35616/nba-elo%2Fnbaallelo.json?_facet=lg_id&_size=1&lg_id=NBA\",\r\n \"selected\": false\r\n },\r\n {\r\n \"value\": \"ABA\",\r\n \"label\": \"ABA\",\r\n \"count\": 8298,\r\n \"toggle_url\": \"http://fivethirtyeight.datasettes.com/fivethirtyeight-ac35616/nba-elo%2Fnbaallelo.json?_facet=lg_id&_size=1&lg_id=ABA\",\r\n \"selected\": false\r\n }\r\n ],\r\n \"truncated\": false\r\n }\r\n },\r\n \"suggested_facets\": [\r\n {\r\n \"name\": \"_iscopy\",\r\n \"toggle_url\": \"/fivethirtyeight-ac35616/nba-elo%2Fnbaallelo.json?_facet=lg_id&_size=1&_facet=_iscopy\"\r\n }\r\n ],\r\n \"next_url\": \"http://fivethirtyeight.datasettes.com/fivethirtyeight-ac35616/nba-elo%2Fnbaallelo.json?_facet=lg_id&_size=1&_next=1\",\r\n}\r\n```\r\n`next_url` and `facet_results` both link to `http://` when they should link to `https://`.\r\n\r\nNote that suggested facets doesn't include the full URL at all, which is a consistency bug.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/333/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 340396247, "node_id": "MDU6SXNzdWUzNDAzOTYyNDc=", "number": 339, "title": "Expose SANIC_RESPONSE_TIMEOUT config option in a sensible way", "user": {"value": 12617395, "label": "bsilverm"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2018-07-11T20:38:06Z", "updated_at": "2022-03-21T22:22:40Z", "closed_at": "2022-03-21T22:22:34Z", "author_association": "NONE", "pull_request": null, "body": "Is it possible to configure the sql_time_limit_ms beyond 60 seconds? It seems queries are still timing out at 60 seconds when sql_time_limit_ms is set to 180000. We have a very large data set and often encounter timeouts when testing new queries from the datasette UI. We are optimizing our database as much as we can, but still may require more than 60 seconds for complex queries.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/339/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 341123355, "node_id": "MDU6SXNzdWUzNDExMjMzNTU=", "number": 342, "title": "Requesting support for query description", "user": {"value": 12617395, "label": "bsilverm"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2018-07-13T18:50:16Z", "updated_at": "2018-07-24T04:53:21Z", "closed_at": "2018-07-16T02:33:54Z", "author_association": "NONE", "pull_request": null, "body": "It would be great if the metadata file allowed you to enter a description for the query. We have a lot of pre-defined queries that can only be so descriptive by their name. It would be nice if an optional description could be included underneath the name within the UI, or on hover where it currently shows the SQL.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/342/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 345821500, "node_id": "MDU6SXNzdWUzNDU4MjE1MDA=", "number": 352, "title": "render_cell(value) plugin hook", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2018-07-30T15:56:20Z", "updated_at": "2020-02-10T16:18:58Z", "closed_at": "2018-08-05T00:14:57Z", "author_association": "OWNER", "pull_request": null, "body": "To allow plugins to customize how values matching a specific pattern are displayed in the HTML table view.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/352/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 346028655, "node_id": "MDU6SXNzdWUzNDYwMjg2NTU=", "number": 356, "title": "Ability to display facet counts for many-to-many relationships", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2018-07-31T04:14:26Z", "updated_at": "2019-05-29T21:39:12Z", "closed_at": "2019-05-25T16:30:09Z", "author_association": "OWNER", "pull_request": null, "body": "Parent: #354", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/356/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 413867537, "node_id": "MDU6SXNzdWU0MTM4Njc1Mzc=", "number": 16, "title": "add_column() should support REFERENCES {other_table}({other_column})", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2019-02-24T21:00:45Z", "updated_at": "2019-05-29T05:17:59Z", "closed_at": "2019-05-29T04:56:18Z", "author_association": "OWNER", "pull_request": null, "body": "Related to #2 ", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/16/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 423316403, "node_id": "MDU6SXNzdWU0MjMzMTY0MDM=", "number": 422, "title": "Figure out what to do about table counts in a mutable world", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2019-03-20T15:27:15Z", "updated_at": "2019-05-02T05:43:11Z", "closed_at": "2019-05-02T05:43:11Z", "author_association": "OWNER", "pull_request": null, "body": "In moving away from the existing static inspect method (see #420 and #419) the biggest thing lost is full table row counts. These can be expensive against large tables, but currently Datasette runs the `count (*) from x` query once at inspection time and then reuses it for every page.\r\n\r\nWe can run those counts with a timelimit, but this means that for larger tables we won't be able to show a count at all, which is disappointing.\r\n\r\nIs there a way we can find an approximate or lower bound count for a table?", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/422/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 435531034, "node_id": "MDU6SXNzdWU0MzU1MzEwMzQ=", "number": 435, "title": "Tracing support for seeing what SQL queries were executed", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 4305096, "label": "0.28"}, "comments": 4, "created_at": "2019-04-21T17:37:37Z", "updated_at": "2019-05-11T20:32:21Z", "closed_at": "2019-05-11T19:07:42Z", "author_association": "OWNER", "pull_request": null, "body": "Features like faceting, foreign key expansions and now the inspect-less index view mean Datasette can end up executing a surprisingly large number of SQL queries to render a single page.\r\n\r\nPast experience with projects like [tikbar](https://github.com/simonw/tikibar) have shown that being able to see what actually went into rendering a page can be critical for optimizing performance and generally understanding how everything works.\r\n\r\nSupport a tracing mode (probably via a `?_trace=1` querystring) which adds information about what is actually going on to both the HTML and the JSON.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/435/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 443023308, "node_id": "MDU6SXNzdWU0NDMwMjMzMDg=", "number": 462, "title": "Replace most of `.inspect()` (and `datasette inspect`) with table counting", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 4305096, "label": "0.28"}, "comments": 4, "created_at": "2019-05-11T18:26:06Z", "updated_at": "2019-05-16T14:31:05Z", "closed_at": "2019-05-16T14:31:05Z", "author_association": "OWNER", "pull_request": null, "body": "This is the last part of #419 - with the move to supporting mutable databases by default, the inspect-data mechanism currently in use no-longer makes much sense.\r\n\r\nThe one optimization I think it's worth keeping for databases opened in immutable mode is the cached table counts. I think `datasette inspect` should cut down to only counting the rows in the tables - the other things done by inspect (figuring out columns, foreign key relationships, FTS etc) should all be fast enough that they can be reliably performed at runtime even against large databases.\r\n\r\nIf performing them at run-time has performance issues, I would rather cache those results internally within Datasette after they are first calculated than continue to support them in the `datasette inspect` command - to keep things simpler.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/462/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 443038584, "node_id": "MDU6SXNzdWU0NDMwMzg1ODQ=", "number": 465, "title": "Decide what to do about /-/inspect", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2019-05-11T21:39:46Z", "updated_at": "2019-06-28T16:34:33Z", "closed_at": "2019-06-28T16:34:33Z", "author_association": "OWNER", "pull_request": null, "body": "It's not clear to me what this endpoint should do now as a result of #419 - it's still useful to be able to introspect databases for tools like datasette-registry, but since we aren't pre-calculating introspection data any more I need to rethink the approach.\r\n\r\nFor one thing, this endpoint may need to be paginated. Or maybe it should be split up into separate endpoints for each connected database? Those should probably be paginated too seeing as fivethirtyeight has 400+ tables.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/465/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}