{"id": 999902754, "node_id": "I_kwDOBm6k_c47mU4i", "number": 1473, "title": "base logo link visits `undefined` rather than href url", "user": {"value": 192568, "label": "mroswell"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2021-09-18T04:17:04Z", "updated_at": "2021-09-19T00:45:32Z", "closed_at": null, "author_association": "CONTRIBUTOR", "pull_request": null, "body": "I have two connected sites:\r\nhttp://www.SaferOrToxic.org \r\n(a Hugo website)\r\nand:\r\nhttp://disinfectants.SaferOrToxic.org/disinfectants/listN\r\n(a datasette table page)\r\n\r\nThe latter is linked as \"The List\" in the former's menu.\r\n(I'd love a prettier URL, but that's what I've got.)\r\n\r\nOn:\r\nhttp://disinfectants.SaferOrToxic.org/disinfectants/listN\r\n... all the other menu links should point back to:\r\nhttps://www.SaferOrToxic.org \r\nAnd they do!\r\n\r\nBut the logo, for some reason--though it has an href pointing to:\r\nhttps://www.SaferOrToxic.org\r\nKeeps going to this instead:\r\nhttps://disinfectants.saferortoxic.org/disinfectants/undefined\r\n\r\nWhat is causing that? How can I fix it?\r\n\r\nIn #1284 back in March, I was doing battle with the index.html template, in a still unresolved issue. (I wanted only a single table page at the root.)\r\n\r\nBut I thought, well, if I can't resolve that, at least I could just point the main website to the datasette page (\"The List,\") and then have the List point back to the home website.\r\n\r\nThe menu hrefs to https://www.SaferOrToxic.org work just fine, exactly as they should, from the datasette page. Even the Home link works properly.\r\n\r\nBut the logo link keeps rewriting to: https://disinfectants.saferortoxic.org/disinfectants/undefined\r\n\r\nThis is the HTML:\r\n```\r\n\"Logo:\r\n```\r\n\r\n\r\nIs this somehow related to cloudflare?\r\nOr something in the datasette code?\r\n\r\nI'm starting to think it's a cloudflare issue.\r\n\"Screen\r\n\r\nCan I at least rule out it being a datasette issue?\r\n\r\nMy repository is here:\r\nhttps://github.com/mroswell/list-N\r\n\r\n(BTW, I couldn't figure out how to reference a local image, either, on the datasette side, which is why I'm using the image from the www home page.)\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1473/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1021550542, "node_id": "I_kwDOBm6k_c4845_O", "number": 1482, "title": "Support Python 3.10", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2021-10-09T00:30:52Z", "updated_at": "2021-10-24T22:21:40Z", "closed_at": "2021-10-24T22:19:55Z", "author_association": "OWNER", "pull_request": null, "body": "I started work on this in #1481 where I found a Python 3.10 bug that needs a workaround in Janus, see:\r\n\r\n- https://github.com/aio-libs/janus/issues/358\r\n\r\nThis is a tracking issue for anything else that shows up.\r\n\r\nThis is also needed for the Homebrew package to upgrade to 3.10:\r\n\r\n- https://github.com/Homebrew/homebrew-core/pull/86932", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1482/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1057996111, "node_id": "I_kwDOBm6k_c4_D71P", "number": 1517, "title": "Let `register_routes()` over-ride default routes within Datasette", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 3268330, "label": "Datasette 1.0"}, "comments": 2, "created_at": "2021-11-19T00:22:15Z", "updated_at": "2021-11-19T03:20:00Z", "closed_at": "2021-11-19T03:07:27Z", "author_association": "OWNER", "pull_request": null, "body": "See https://github.com/simonw/datasette/issues/878#issuecomment-973554024_ - right now `register_routes()` can't replace default Datasette routes.\r\n\r\nIt would be neat if plugins could do this - especially if there was a neat documented way for them to then re-dispatch to the original route code after making some kind of modification.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1517/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1059549523, "node_id": "I_kwDOBm6k_c4_J3FT", "number": 1526, "title": "Add to vercel.json, rather than overwriting it.", "user": {"value": 192568, "label": "mroswell"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2021-11-22T00:47:12Z", "updated_at": "2021-11-22T04:49:45Z", "closed_at": "2021-11-22T04:13:47Z", "author_association": "CONTRIBUTOR", "pull_request": null, "body": "I'd like to be able to add to vercel.json. But Datasette overwrites whatever I put in that file. I originally reported this here:\r\nhttps://github.com/simonw/datasette-publish-vercel/issues/51\r\n\r\nIn that case, I wanted to do a rewrite... and now I need to do 301 redirects (because we had to rename our site).\r\n\r\nCan this be addressed?\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1526/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1073712378, "node_id": "I_kwDOBm6k_c4__4z6", "number": 1544, "title": "Code that detects the label column for a table is case-sensitive", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2021-12-07T20:01:25Z", "updated_at": "2021-12-07T20:03:43Z", "closed_at": "2021-12-07T20:03:43Z", "author_association": "OWNER", "pull_request": null, "body": "I just noticed that a column called `Name` is not being picked up as the label column for a table.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1544/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1077628073, "node_id": "I_kwDOBm6k_c5AO0yp", "number": 1550, "title": "Research option for returning all rows from arbitrary query", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2021-12-11T19:31:11Z", "updated_at": "2021-12-11T23:43:24Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "Inspired by thinking about #1549 - returning ALL rows from an arbitrary query is a lot easier if you just run that query and keep iterating over the cursor.\r\n\r\nI've avoided doing that in the past because it could tie up a connection for a long time - but in private instances this wouldn't be such a problem.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1550/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1083573206, "node_id": "I_kwDOBm6k_c5AlgPW", "number": 1563, "title": "Datasette(... files=) should not be a required argument", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 7571612, "label": "Datasette 0.60"}, "comments": 2, "created_at": "2021-12-17T19:54:18Z", "updated_at": "2022-01-13T22:27:18Z", "closed_at": "2021-12-18T02:19:40Z", "author_association": "OWNER", "pull_request": null, "body": "```pycon\r\n>>> ds = Datasette(memory=True)\r\nTraceback (most recent call last):\r\n File \"\", line 1, in \r\nTypeError: __init__() missing 1 required positional argument: 'files'\r\n>>> ds = Datasette(memory=True, files=[])\r\n```\r\nI wanted to create an in-memory Datasette for running some tests, no point in forcing me to pass `files=[]` to do that.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1563/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1083921371, "node_id": "I_kwDOBm6k_c5Am1Pb", "number": 1570, "title": "Separate db.execute_write() into three methods", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 7571612, "label": "Datasette 0.60"}, "comments": 2, "created_at": "2021-12-18T18:45:54Z", "updated_at": "2022-01-13T22:27:38Z", "closed_at": "2021-12-18T18:57:25Z", "author_association": "OWNER", "pull_request": null, "body": "> Rather than adding a `executemany=True` parameter, I'm now thinking a better design might be to have three methods:\r\n>\r\n> - `db.execute_write(sql, params=None, block=False)`\r\n> - `db.execute_write_script(sql, block=False)`\r\n> - `db.execute_write_many(sql, params_seq, block=False)`\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1555#issuecomment-997267416_", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1570/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1083726550, "node_id": "I_kwDOBm6k_c5AmFrW", "number": 1568, "title": "Trace should show queries on the write connection too", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 7571612, "label": "Datasette 0.60"}, "comments": 2, "created_at": "2021-12-18T02:34:12Z", "updated_at": "2022-01-13T22:27:23Z", "closed_at": "2021-12-18T02:42:34Z", "author_association": "OWNER", "pull_request": null, "body": "> Here's why - `trace` only applies to read, not write SQL operations: https://github.com/simonw/datasette/blob/7c8f8aa209e4ba7bf83976f8495d67c28fbfca24/datasette/database.py#L209-L211\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1555#issuecomment-997128508_", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1568/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1083895395, "node_id": "I_kwDOBm6k_c5Amu5j", "number": 1569, "title": "db.execute_write(..., executescript=True) parameter", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 7571612, "label": "Datasette 0.60"}, "comments": 2, "created_at": "2021-12-18T18:20:47Z", "updated_at": "2022-01-13T22:27:27Z", "closed_at": "2021-12-18T18:34:18Z", "author_association": "OWNER", "pull_request": null, "body": "> Idea: teach `execute_write` to accept an optional `executescript=True` parameter, like this:\r\n```diff\r\ndiff --git a/datasette/database.py b/datasette/database.py\r\nindex 468e936..1a424f5 100644\r\n--- a/datasette/database.py\r\n+++ b/datasette/database.py\r\n@@ -94,10 +94,14 @@ class Database:\r\n f\"file:{self.path}{qs}\", uri=True, check_same_thread=False\r\n )\r\n \r\n- async def execute_write(self, sql, params=None, block=False):\r\n+ async def execute_write(self, sql, params=None, executescript=False, block=False):\r\n+ assert not executescript and params, \"Cannot use params with executescript=True\"\r\n def _inner(conn):\r\n with conn:\r\n- return conn.execute(sql, params or [])\r\n+ if executescript:\r\n+ return conn.executescript(sql)\r\n+ else:\r\n+ return conn.execute(sql, params or [])\r\n \r\n with trace(\"sql\", database=self.name, sql=sql.strip(), params=params):\r\n results = await self.execute_write_fn(_inner, block=block)\r\n```\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1555#issuecomment-997248364_", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1569/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1097040427, "node_id": "I_kwDOBm6k_c5BY4Ir", "number": 1587, "title": "Add `sqlite_stat1`(-4) tables to hidden table list", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2022-01-08T21:28:20Z", "updated_at": "2022-01-20T04:12:59Z", "closed_at": "2022-01-20T04:12:59Z", "author_association": "OWNER", "pull_request": null, "body": "> Running `ANALYZE` creates a new visible table called `sqlite_stat1`: https://www.sqlite.org/fileformat.html#the_sqlite_stat1_table\r\n>\r\n> This should be added to the default list of hidden tables in Datasette.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1587/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1122416919, "node_id": "I_kwDOBm6k_c5C5rkX", "number": 1623, "title": "/-/patterns returns link: alternate JSON header to 404", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 3268330, "label": "Datasette 1.0"}, "comments": 2, "created_at": "2022-02-02T21:42:49Z", "updated_at": "2022-03-19T04:04:49Z", "closed_at": "2022-02-02T21:48:56Z", "author_association": "OWNER", "pull_request": null, "body": "Bug from:\r\n- #1620\r\n\r\n```\r\n% curl -s -I 'https://latest.datasette.io/-/patterns' | grep link\r\nlink: https://latest.datasette.io/-/patterns.json; rel=\"alternate\"; type=\"application/json+datasette\"\r\n```", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1623/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1122427321, "node_id": "I_kwDOBm6k_c5C5uG5", "number": 1624, "title": "Index page `/` has no CORS headers", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2022-02-02T21:56:10Z", "updated_at": "2022-09-28T16:54:22Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "Compare the following:\r\n```\r\n% curl -I 'https://latest.datasette.io/fixtures'\r\nHTTP/1.1 200 OK\r\nlink: https://latest.datasette.io/fixtures.json; rel=\"alternate\"; type=\"application/json+datasette\"\r\ncache-control: max-age=5\r\nreferrer-policy: no-referrer\r\naccess-control-allow-origin: *\r\naccess-control-allow-headers: Authorization\r\naccess-control-expose-headers: Link\r\ncontent-type: text/html; charset=utf-8\r\nx-databases: _memory, _internal, fixtures, extra_database\r\nDate: Wed, 02 Feb 2022 21:55:49 GMT\r\nServer: Google Frontend\r\nTransfer-Encoding: chunked\r\n\r\n% curl -I 'https://latest.datasette.io/' \r\nHTTP/1.1 200 OK\r\nlink: https://latest.datasette.io/.json; rel=\"alternate\"; type=\"application/json+datasette\"\r\ncontent-type: text/html; charset=utf-8\r\nx-databases: _memory, _internal, fixtures, extra_database\r\nDate: Wed, 02 Feb 2022 21:55:52 GMT\r\nServer: Google Frontend\r\nTransfer-Encoding: chunked\r\n```", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1624/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1109783030, "node_id": "I_kwDOBm6k_c5CJfH2", "number": 1607, "title": "More detailed information about installed SpatiaLite version", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 3268330, "label": "Datasette 1.0"}, "comments": 2, "created_at": "2022-01-20T21:28:03Z", "updated_at": "2022-02-09T06:42:02Z", "closed_at": "2022-02-09T06:32:28Z", "author_association": "OWNER", "pull_request": null, "body": "https://www.gaia-gis.it/gaia-sins/spatialite-sql-5.0.0.html#version has a whole bunch of interesting functions for things like `freexl_version()` and `geos_version()` and `HasMathSQL()` and suchlike.\r\n\r\nThese could be shown on the `/-/versions` page.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1607/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1117132741, "node_id": "I_kwDOBm6k_c5ClhfF", "number": 1615, "title": "Potential simplified publishing mechanism", "user": {"value": 369053, "label": "aidansteele"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2022-01-28T08:34:50Z", "updated_at": "2022-02-02T07:34:21Z", "closed_at": "2022-02-02T07:34:17Z", "author_association": "NONE", "pull_request": null, "body": "Hi,\r\n\r\nForewarning: this idea is one I've only been thinking about for a while and it's not fully fleshed-out yet.\r\n\r\nI love Datasette and what it stands for. I was thinking about how we could make it accessible to more people, especially those without access to credit cards required for a lot of hosting options. Or they might not feel comfortable signing up for said services.\r\n\r\nSo I was thinking I might create a service that hosts Datasette instances for folks. I'd probably stick it on AWS Lambda and limit requests to something like n/month to avoid bankrupting myself. If I did build such a hypothetical service, I was thinking I would rely on GitHub Actions to do the heavy lifting. \r\n\r\nE.g. user `johndoe` creates a repo `my-animals` with a couple of files: `dogs.csv`, `cats.csv` and the following GitHub Actions workflow:\r\n\r\n```yaml\r\n# .github/workflows/push.yml\r\non:\r\n push\r\n\r\n# this allows the publish action to use OIDC to authenticate johndoe/my-animals\r\npermissions:\r\n id-token: write\r\n contents: read\r\n\r\njobs:\r\n publish:\r\n runs-on: ubuntu-latest\r\n steps:\r\n - uses: actions/setup-python@v2\r\n\r\n - run: pip install sqlite-utils\r\n\r\n - uses: actions/checkout@v2\r\n\r\n - run: |\r\n set -eux\r\n \r\n sqlite-utils create-database animals.db \r\n sqlite-utils insert animals.db dogs dogs.csv --csv \r\n sqlite-utils insert animals.db cats cats.csv --csv\r\n\r\n - uses: datasette-hub/publish@v1\r\n with:\r\n db: animals.db\r\n metadata: meta.yml\r\n\r\n # this step is helpful for debugging why the\r\n # generated sqlite db was rejected\r\n - uses: actions/upload-artifact@v2\r\n if: failure()\r\n with:\r\n path: animals.db\r\n retention-days: 1\r\n```\r\n\r\nThis would then cause a Datasette instance to be available at `https://johndoe-my-animals.datasette-hub.test/`. It feels like this could significantly reduce the friction to someone being able to go from data set to Datasette.\r\n\r\nWhat do you think? Does this address a real need? Or am I perhaps misunderstanding the main friction points? As a bonus: it feels like this would pair well with [git scraping](https://simonwillison.net/2020/Oct/9/git-scraping/).", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1615/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1129052172, "node_id": "I_kwDOBm6k_c5DS_gM", "number": 1633, "title": "base_url or prefix does not work with _exact match", "user": {"value": 6613091, "label": "henrikek"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2022-02-09T21:45:07Z", "updated_at": "2022-04-28T09:12:56Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "When i hit \"Apply\" button to search with \"_exact\" for a column syntax the URL prefix is removed from the url.\r\n\r\n![image](https://user-images.githubusercontent.com/6613091/153293758-0b757d55-5757-4987-992e-9426e69a7956.png)\r\n\r\nAnd the result is:\r\n![image](https://user-images.githubusercontent.com/6613091/153294672-87be7809-bb7b-455d-bf1a-41e90bbfa4ae.png)\r\n\r\nIf I add the marked row to url_builder.py it seams to work:\r\n![image](https://user-images.githubusercontent.com/6613091/153295231-bdd52e37-efcf-4b21-9d37-69f182a922f4.png)\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1633/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1148725876, "node_id": "I_kwDOBm6k_c5EeCp0", "number": 1640, "title": "Support static assets where file length may change, e.g. logs", "user": {"value": 57859326, "label": "broccolihighkicks"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2022-02-24T00:34:42Z", "updated_at": "2022-03-05T01:19:25Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "This is a bit of an oxymoron.\r\n\r\nI am serving a log.txt file for a background process using the Datasette --static CLI. This is useful as I can observe a background process from the web UI to see any errors that occur (instead of spelunking the logs via docker exec/ssh etc).\r\n\r\nI get this error, which I think is because Datasette assumes that the size of the content does not change (but appending new log lines means the content length changes).\r\n\r\n\r\n```python\r\nTraceback (most recent call last):\r\n File \"/usr/local/lib/python3.9/site-packages/datasette/app.py\", line 1181, in route_path\r\n response = await view(request, send)\r\n File \"/usr/local/lib/python3.9/site-packages/datasette/utils/asgi.py\", line 305, in inner_static\r\n await asgi_send_file(send, full_path, chunk_size=chunk_size)\r\n File \"/usr/local/lib/python3.9/site-packages/datasette/utils/asgi.py\", line 280, in asgi_send_file\r\n await send(\r\n File \"/usr/local/lib/python3.9/site-packages/asgi_csrf.py\", line 104, in wrapped_send\r\n await send(event)\r\n File \"/usr/local/lib/python3.9/site-packages/uvicorn/protocols/http/h11_impl.py\", line 460, in send\r\n output = self.conn.send(event)\r\n File \"/usr/local/lib/python3.9/site-packages/h11/_connection.py\", line 468, in send\r\n data_list = self.send_with_data_passthrough(event)\r\n File \"/usr/local/lib/python3.9/site-packages/h11/_connection.py\", line 501, in send_with_data_passthrough\r\n writer(event, data_list.append)\r\n File \"/usr/local/lib/python3.9/site-packages/h11/_writers.py\", line 58, in __call__\r\n self.send_data(event.data, write)\r\n File \"/usr/local/lib/python3.9/site-packages/h11/_writers.py\", line 78, in send_data\r\n raise LocalProtocolError(\"Too much data for declared Content-Length\")\r\nh11._util.LocalProtocolError: Too much data for declared Content-Length\r\nERROR: Exception in ASGI application\r\nTraceback (most recent call last):\r\n File \"/usr/local/lib/python3.9/site-packages/datasette/app.py\", line 1181, in route_path\r\n response = await view(request, send)\r\n File \"/usr/local/lib/python3.9/site-packages/datasette/utils/asgi.py\", line 305, in inner_static\r\n await asgi_send_file(send, full_path, chunk_size=chunk_size)\r\n File \"/usr/local/lib/python3.9/site-packages/datasette/utils/asgi.py\", line 280, in asgi_send_file\r\n await send(\r\n File \"/usr/local/lib/python3.9/site-packages/asgi_csrf.py\", line 104, in wrapped_send\r\n await send(event)\r\n File \"/usr/local/lib/python3.9/site-packages/uvicorn/protocols/http/h11_impl.py\", line 460, in send\r\n output = self.conn.send(event)\r\n File \"/usr/local/lib/python3.9/site-packages/h11/_connection.py\", line 468, in send\r\n data_list = self.send_with_data_passthrough(event)\r\n File \"/usr/local/lib/python3.9/site-packages/h11/_connection.py\", line 501, in send_with_data_passthrough\r\n writer(event, data_list.append)\r\n File \"/usr/local/lib/python3.9/site-packages/h11/_writers.py\", line 58, in __call__\r\n self.send_data(event.data, write)\r\n File \"/usr/local/lib/python3.9/site-packages/h11/_writers.py\", line 78, in send_data\r\n raise LocalProtocolError(\"Too much data for declared Content-Length\")\r\nh11._util.LocalProtocolError: Too much data for declared Content-Length\r\n```\r\n\r\nThanks, I am finding Datasette very useful.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1640/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1161937073, "node_id": "I_kwDOBm6k_c5FQcCx", "number": 1653, "title": "Mechanism to default a table to sorting by multiple columns", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2022-03-07T21:20:11Z", "updated_at": "2022-03-07T21:23:39Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "### Discussed in https://github.com/simonw/datasette/discussions/1652\r\n\r\n
\r\n\r\nOriginally posted by **zaneselvans** March 7, 2022\r\nIt's easy to tell datasette to sort tables using a single column, as [described in the docs](https://docs.datasette.io/en/stable/metadata.html#setting-a-default-sort-order):\r\n\r\n```yaml\r\ndatabases:\r\n ferc1:\r\n tables:\r\n f1_edcfu_epda:\r\n sort: created_time\r\n```\r\n\r\nBut is there some way to tell it to sort using a composite key, like you would in an `ORDER BY` clause instead? For example, the way it's being done **[in this query](https://data.catalyst.coop/ferc1?sql=select%0D%0A++rowid%2C%0D%0A++respondent_id%2C%0D%0A++report_year%2C%0D%0A++spplmnt_num%2C%0D%0A++row_number%2C%0D%0A++row_seq%2C%0D%0A++row_prvlg%2C%0D%0A++acct_num%2C%0D%0A++depr_plnt_base%2C%0D%0A++est_avg_srvce_lf%2C%0D%0A++net_salvage%2C%0D%0A++apply_depr_rate%2C%0D%0A++mrtlty_crv_typ%2C%0D%0A++avg_remaining_lf%2C%0D%0A++report_prd%0D%0Afrom%0D%0A++f1_edcfu_epda%0D%0Awhere%0D%0A++respondent_id+%3D+210%0D%0A++AND+report_year+%3D+2020%0D%0Aorder+by%0D%0A++report_year%2C+report_prd%2C+respondent_id%2C+spplmnt_num%2C+row_number%0D%0Alimit%0D%0A++1000)** on our Datasette?\r\n\r\n```sql\r\nSELECT\r\n respondent_id,\r\n report_year,\r\n spplmnt_num,\r\n row_number,\r\n row_seq,\r\n row_prvlg,\r\n acct_num,\r\n depr_plnt_base,\r\n est_avg_srvce_lf,\r\n net_salvage,\r\n apply_depr_rate,\r\n mrtlty_crv_typ,\r\n avg_remaining_lf,\r\n report_prd\r\nFROM\r\n f1_edcfu_epda\r\nWHERE\r\n respondent_id = 210\r\n AND report_year = 2020\r\nORDER BY\r\n report_year, report_prd, respondent_id, spplmnt_num, row_number\r\nLIMIT\r\n 1000\r\n```\r\n\r\nThe problem here is that by default it's using `rowid` (the SQLite assigned autoincrementing integer key) to order the records, but the table **should** have a natural composite primary key, but the original database that this data is being migrated from doesn't enforce unique primary keys, so there are dupes, and we don't want to drop those rows, and the records are somehow getting jumbled in the database (the `rowid` ordering isn't lined up with the expected ordering based on the composite primary key, though it's close) and this jumbling is confusing to users that expect to see the data ordered based on the natural primary key.\r\n\r\nI've tried setting the `sort` metadata parameter to a list of column names, a tuple of column names, a quoted string of comma-separated column names, a quoted string of a tuple of column names...\r\n\r\n```yaml\r\ndatabases:\r\n ferc1:\r\n tables:\r\n f1_edcfu_epda:\r\n sort: \"(report_year, report_prd, respondent_id, spplmnt_num, row_number)\"\r\n```\r\n\r\nand they all give me server errors like:\r\n\r\n```\r\nCannot sort table by (report_year, report_prd, respondent_id, spplmnt_num, row_number)\r\n```
", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1653/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1174404647, "node_id": "I_kwDOBm6k_c5F__4n", "number": 1669, "title": "Release 0.61 alpha", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2022-03-20T00:35:35Z", "updated_at": "2022-03-20T01:24:36Z", "closed_at": "2022-03-20T01:24:36Z", "author_association": "OWNER", "pull_request": null, "body": "> I'm going to release this as a 0.61 alpha so I can more easily depend on it from `datasette-hashed-urls`.\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1668#issuecomment-1073136896_", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1669/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1170554975, "node_id": "I_kwDOBm6k_c5FxUBf", "number": 1663, "title": "Document the internals that were used in datasette-hashed-urls", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 3268330, "label": "Datasette 1.0"}, "comments": 2, "created_at": "2022-03-16T05:17:08Z", "updated_at": "2022-03-19T04:04:50Z", "closed_at": "2022-03-17T21:32:38Z", "author_association": "OWNER", "pull_request": null, "body": "The https://github.com/simonw/datasette-hashed-urls used a couple of currently undocumented features:\r\n- `db.hash`\r\n- `Datasette(..., immutables=[...])`", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1663/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1177101697, "node_id": "I_kwDOBm6k_c5GKSWB", "number": 1681, "title": "Potential bug in numeric handling where_clause for filters", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2022-03-22T17:43:50Z", "updated_at": "2022-03-22T17:49:09Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "> Note that Datasette does already have special logic to convert parameters to integers for numeric comparisons like `>`:\r\n>\r\n> https://github.com/simonw/datasette/blob/c4c9dbd0386e46d2bf199f0ed34e4895c98cb78c/datasette/filters.py#L203-L212\r\n> \r\n> Though... it looks like there's a bug in that? It doesn't account for `float` values - `\"3.5\".isdigit()` return `False` - probably for the best, because `int(3.5)` would break that value anyway.\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1671#issuecomment-1075432283_", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1681/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1182227211, "node_id": "I_kwDOBm6k_c5Gd1sL", "number": 1692, "title": "[plugins][feature request]: Support additional script tag attributes when loading custom JS", "user": {"value": 9020979, "label": "hydrosquall"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2022-03-27T01:16:03Z", "updated_at": "2022-03-30T06:14:51Z", "closed_at": null, "author_association": "CONTRIBUTOR", "pull_request": null, "body": "## Motivation\r\n\r\n- The build system for my new [plugin](https://github.com/hydrosquall/datasette-nteract-data-explorer) has two output JS files, one for browsers that support ES modules, one for browsers that don't. At present, I'm only passing one of them into Datasette.\r\n- I'd like to specify the non-es-module script as a fallback for older browsers. I don't want to load it by default, because browsers will only need one, and it's heavy, so for now I'm only supporting modern browsers. \r\n\r\nTo be able to support legacy browsers without slowing down users with modern browsers, I would like to be able to set additional HTML attributes on the tag fallback script, `nomodule` and `defer`. My injected scripts should look something like this:\r\n\r\n```html\r\n\r\n\r\n```\r\n\r\n## Proposal\r\n\r\nTo achieve this, I propose additional optional properties to the API accepted by the `extra_js_urls` hook and custom JS field the `metadata.json` [described here](https://docs.datasette.io/en/stable/custom_templates.html#custom-css-and-javascript). \r\n\r\nUnder this API, I'd write something like this to get the above HTML rendered in Datasette.\r\n\r\n```json\r\n{\r\n \"extra_js_urls\": [\r\n {\r\n \"url\": \"/index.my-es-module-bundle.js\",\r\n \"module\": true,\r\n },\r\n {\r\n \"url\": \"/index.my-legacy-fallback-bundle.js\",\r\n \"nomodule\": \"\",\r\n \"defer\": true\r\n }\r\n ]\r\n}\r\n```\r\n\r\n## Resources\r\n\r\n- [MDN on the script tag](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/script)\r\n - There may be other properties that could be added that are potentially valuable, like `async` or `referrerpolicy`, but I don't have an immediate need for those.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1692/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1182141761, "node_id": "I_kwDOBm6k_c5Gdg1B", "number": 1690, "title": "Idea: `datasette.set_actor_cookie(response, actor)`", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2022-03-26T22:41:52Z", "updated_at": "2022-03-26T22:43:00Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "I just wrote this code in a plugin and it felt like it could benefit from an abstraction: https://github.com/simonw/datasette-auth0/blob/152e6eb21e96e9b73bd9c205f9749a1297d0ef0b/datasette_auth0/__init__.py#L79-L92\r\n\r\n```python\r\n redirect_response = Response.redirect(\"/\")\r\n expires_at = int(time.time()) + (24 * 60 * 60)\r\n redirect_response.set_cookie(\r\n \"ds_actor\",\r\n datasette.sign(\r\n {\r\n \"a\": profile_response.json(),\r\n \"e\": baseconv.base62.encode(expires_at),\r\n },\r\n \"actor\",\r\n ),\r\n )\r\n return redirect_response\r\n```\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1690/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1185868354, "node_id": "I_kwDOBm6k_c5GrupC", "number": 1695, "title": "Option to un-filter facet not shown for `?col__exact=value`", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2022-03-30T04:44:02Z", "updated_at": "2022-03-30T04:46:18Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "Spotted this on a page with `COUNTY__exact=Lee` in the URL:\r\n\r\n![CleanShot 2022-03-29 at 21 41 46@2x](https://user-images.githubusercontent.com/9599/160752849-a9039343-3770-4655-920b-f19e25687a57.png)\r\n\r\nWith `COUNTY=Lee` you get this instead:\r\n\r\n\"image\"\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1695/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1186696202, "node_id": "I_kwDOBm6k_c5Gu4wK", "number": 1696, "title": "Show foreign key label when filtering", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2022-03-30T16:18:54Z", "updated_at": "2023-01-29T20:56:20Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "For example here:\r\n\r\n\"image\"\r\n\r\n3 corresponds to \"Human Related: Other\" - it would be neat to display this in this area of the page somehow.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1696/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1200649124, "node_id": "I_kwDOBm6k_c5HkHOk", "number": 1708, "title": "Datasette 1.0 alpha upcoming release notes", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": {"value": 8755003, "label": "Datasette 1.0a-next"}, "comments": 2, "created_at": "2022-04-11T22:57:12Z", "updated_at": "2022-12-13T05:29:06Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "I'm going to try writing the release notes first, to see if that helps unblock me.\r\n\r\n# \u26a0\ufe0f Any release notes in this issue are a draft, and should not be treated as the real thing \u26a0\ufe0f ", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1708/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1221849746, "node_id": "I_kwDOBm6k_c5I0_KS", "number": 1732, "title": "Custom page variables aren't decoded", "user": {"value": 52649, "label": "tannewt"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2022-04-30T14:55:46Z", "updated_at": "2022-05-03T01:50:45Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "I have a page `templates/filer/{filer_id}.html`. It uses `filer_id` in a `sql()` call to fetch data. With 0.61.1 this no longer works because the spaces in IDs isn't preserved. Instead, the escaped version is passed into the template and the id isn't present in my db.\r\n\r\nDatasette should unescape the url component before passing them into the template.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1732/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1223527226, "node_id": "I_kwDOBm6k_c5I7Ys6", "number": 1738, "title": "\"Cannot use _sort and _sort_desc at the same time\"", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 8303187, "label": "Datasette 0.62"}, "comments": 2, "created_at": "2022-05-03T01:06:24Z", "updated_at": "2022-08-14T16:13:55Z", "closed_at": "2022-08-14T16:13:55Z", "author_association": "OWNER", "pull_request": null, "body": "Triggered this error while playing with the sort desc checkbox and the apply button that are only visible on this page at mobile screen width:\r\n\r\nhttps://latest.datasette.io/fixtures/compound_three_primary_keys?_sort_desc=pk1\r\n\r\nNavigate to that page (with the browser narrow enough to show the box), un-check the box and click Apply:\r\n\r\n![sort-bug](https://user-images.githubusercontent.com/9599/166390804-cb289b29-63dc-4986-b7f9-81cf2ae04914.gif)\r\n\r\nAlso notable: I managed to get to a page with `?_sort_desk=pk1` in the URL three times by clicking around with that button.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1738/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1243517592, "node_id": "I_kwDOBm6k_c5KHpKY", "number": 1748, "title": "Add copy buttons next to code examples in the documentation", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2022-05-20T19:09:00Z", "updated_at": "2022-05-20T19:15:00Z", "closed_at": "2022-05-20T19:11:32Z", "author_association": "OWNER", "pull_request": null, "body": "Similar to the ones in `datasette-copyable` which are implemented here: https://github.com/executablebooks/sphinx-copybutton/tree/f84c001a0507f8ec46779d0701b079a265564583", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1748/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1339444565, "node_id": "I_kwDOBm6k_c5P1k1V", "number": 1783, "title": "Better guidance as to what to do after you've installed Datasette", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2022-08-15T20:11:06Z", "updated_at": "2022-08-15T20:14:01Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "Feedback [from Discord](https://discord.com/channels/823971286308356157/823971286941302908/1008822978793984060):\r\n\r\n> hello, love the project and came for help and to point out a possible gap in the docs. starting with \"getting started\" and \"installation\" every thing looks great, but then there's a giant leap after you have it installed and running. from the user perspective of \"i have a csv of set of csvs that i want to turn into a table(s), what do i do next?\" --- so something like maybe a page for creating your first project should go after \"installation\".\r\n\r\n- https://docs.datasette.io/en/0.62/getting_started.html\r\n- https://docs.datasette.io/en/0.62/installation.html", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1783/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1339663518, "node_id": "I_kwDOBm6k_c5P2aSe", "number": 1784, "title": "Include \"entrypoint\" option on `--load-extension`?", "user": {"value": 15178711, "label": "asg017"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2022-08-16T00:22:57Z", "updated_at": "2022-08-23T18:34:31Z", "closed_at": "2022-08-23T18:34:31Z", "author_association": "CONTRIBUTOR", "pull_request": null, "body": "## Problem \r\n\r\nSQLite extensions have the option to define multiple \"entrypoints\" in each loadable extension. For example, the upcoming version of `sqlite-lines` will have 2 entrypoints: the default `sqlite3_lines_init` (which SQLite will automatically guess for) and `sqlite3_lines_noread_init`. The `sqlite3_lines_noread_init` version omits functions that read from the filesystem, which is necessary for security purposes when running untrusted SQL (which Datasette does).\r\n\r\n(Similar multiple entrypoints will also be added for sqlite-http).\r\n\r\n\r\n\r\nThe `--load-extension` flag, however, doesn't give the option to specify a different entrypoint, so the default one is always used. \r\n\r\n## Proposal\r\n\r\nI want there to be a new command line option of the `--load-extension` flag to specify a custom entrypoint like so:\r\n```\r\ndatasette my.db \\\r\n --load-extension ./lines0 sqlite3_lines0_noread_init\r\n```\r\n\r\nThen, under the hood, this line of code:\r\n\r\nhttps://github.com/simonw/datasette/blob/7af67b54b7d9bca43e948510fc62f6db2b748fa8/datasette/app.py#L562\r\n\r\nWould look something like this:\r\n\r\n```python\r\n conn.execute(\"SELECT load_extension(?, ?)\", [extension, entrypoint]) \r\n```\r\n\r\nOne potential problem: For backward compatibility, I'm not sure if Click allows cli flags to have variable number of options (\"arity\"). So I guess it could also use a `:` delimiter like `--static`:\r\n\r\n```\r\ndatasette my.db \\\r\n --load-extension ./lines0:sqlite3_lines0_noread_init\r\n```\r\n\r\nOr maybe even a new flag name?\r\n\r\n```\r\ndatasette my.db \\\r\n --load-extension-entrypoint ./lines0 sqlite3_lines0_noread_init\r\n```\r\n\r\n\r\nPersonally I prefer the `:` option... and maybe even `--load-extension` -> `--load`? Definitely out of scope for this issue tho\r\n\r\n```\r\ndatasette my.db \\\r\n --load./lines0:sqlite3_lines0_noread_init\r\n```", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1784/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1338278056, "node_id": "I_kwDOBm6k_c5PxICo", "number": 1782, "title": "Release notes for Datasette 0.62", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 8303187, "label": "Datasette 0.62"}, "comments": 2, "created_at": "2022-08-14T15:26:45Z", "updated_at": "2022-08-14T17:40:45Z", "closed_at": "2022-08-14T17:32:54Z", "author_association": "OWNER", "pull_request": null, "body": "I've written a lot of these already for the alphas:\r\n\r\n- https://github.com/simonw/datasette/releases/tag/0.62a0\r\n- https://github.com/simonw/datasette/releases/tag/0.62a1", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1782/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1378495690, "node_id": "I_kwDOBm6k_c5SKizK", "number": 1814, "title": "Static files not served", "user": {"value": 4068, "label": "frafra"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2022-09-19T20:38:17Z", "updated_at": "2022-09-19T23:35:06Z", "closed_at": "2022-09-19T23:34:30Z", "author_association": "NONE", "pull_request": null, "body": "Folder structure:\r\n\r\n```\r\nbibliography/\r\nbibliography/static-files\r\nbibliography/static-files/styles.css\r\nbibliography/bibliography.db\r\nbibliography/metadata.json\r\nbibliography/settings.json\r\n```\r\n\r\n```\r\n$ cat bibliography/settings.json \r\n{\r\n \"suggest_facets\": false,\r\n \"truncate_cells_html\": 1000,\r\n \"static\": \"assets:static-files/\"\r\n}\r\n```\r\n\r\nFile `/assets/styles.css` is not found (HTTP 404, `Database not found: assets`).\r\n\r\nUsing datasette revision d0737e4de51ce178e556fc011ccb8cc46bbb6359.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1814/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1378640768, "node_id": "I_kwDOBm6k_c5SLGOA", "number": 1816, "title": "Validate settings.json on startup in configuration directory mode", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2022-09-19T23:35:18Z", "updated_at": "2022-09-20T01:15:48Z", "closed_at": "2022-09-20T01:15:48Z", "author_association": "OWNER", "pull_request": null, "body": "> It might have been useful for Datasette to show an error when started against a `settings.json` file that contains an invalid setting though.\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1814#issuecomment-1251677554_", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1816/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1393903845, "node_id": "I_kwDOBm6k_c5TFUjl", "number": 1828, "title": "word-wrap: anywhere resulting in weird display", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2022-10-02T21:25:03Z", "updated_at": "2022-10-02T23:01:17Z", "closed_at": "2022-10-02T23:01:17Z", "author_association": "OWNER", "pull_request": null, "body": "e.g. on https://github-to-sqlite.dogsheep.net/github/commits\r\n\r\n\"image\"\r\n\r\nThis is from a change introduced here: https://github.com/simonw/datasette/commit/bf8d84af5422606597be893cedd375020cb2b369 in #1805\r\n\r\nhttps://github.com/simonw/datasette/blob/bf8d84af5422606597be893cedd375020cb2b369/datasette/static/app.css#L447-L450", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1828/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1397193691, "node_id": "I_kwDOBm6k_c5TR3vb", "number": 1832, "title": "__bool__ method on Results", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2022-10-05T04:18:12Z", "updated_at": "2022-10-05T04:32:33Z", "closed_at": "2022-10-05T04:32:33Z", "author_association": "OWNER", "pull_request": null, "body": "Wrote this code today: https://github.com/simonw/datasette-public/blob/1401bfae50e71c1dfd2bfb6954f2e86d5a7ab21b/datasette_public/__init__.py#L41\r\n```python\r\n results = await db.execute(\r\n \"select 1 from _public_tables where table_name = ?\", [table_name]\r\n )\r\n if len(results):\r\n return True\r\n```\r\nWould be nice if I could use `if results` there instead.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1832/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1420055377, "node_id": "I_kwDOBm6k_c5UpFNR", "number": 1847, "title": "Both _local_metadata and _metadata_local?", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2022-10-24T01:43:08Z", "updated_at": "2022-10-24T01:53:13Z", "closed_at": "2022-10-24T01:53:13Z", "author_association": "OWNER", "pull_request": null, "body": "Spotted this in the debugger against the `datasette` object while running tests (`pytest -k test_permissions_cascade` to be exact):\r\n\r\n```\r\n(Pdb) [p for p in dir(self) if p.startswith('_') and '__' not in p]\r\n['_actor', '_asset_urls', '_connected_databases', '_crumb_items', '_local_metadata', '_metadata', '_metadata_local', '_metadata_recursive_update', '_permission_checks', '_plugins', '_prepare_connection', '_refresh_schemas', '_refresh_schemas_lock', '_register_custom_units', '_register_renderers', '_root_token', '_routes', '_secret', '_settings', '_show_messages', '_startup_hook_calculation', '_startup_hook_fired', '_startup_invoked', '_threads', '_versions', '_write_messages_to_response']\r\n```\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1847/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1431786951, "node_id": "I_kwDOBm6k_c5VV1XH", "number": 1876, "title": "SQL query should wrap on SQL interrupted screen", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2022-11-01T17:14:01Z", "updated_at": "2022-11-01T17:22:33Z", "closed_at": "2022-11-01T17:22:33Z", "author_association": "OWNER", "pull_request": null, "body": "Just saw this:\r\n\r\n\"CleanShot", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1876/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1447388809, "node_id": "I_kwDOBm6k_c5WRWaJ", "number": 1887, "title": "Add a confirm step to the drop table API", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 8658075, "label": "Datasette 1.0a0"}, "comments": 2, "created_at": "2022-11-14T04:59:53Z", "updated_at": "2022-11-15T19:59:59Z", "closed_at": "2022-11-14T05:18:51Z", "author_association": "OWNER", "pull_request": null, "body": "> In playing with the API explorer just now I realized it's way too easy to accidentally drop a table using it.\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1871#issuecomment-1313097057_\r\n\r\nAdded drop table API in:\r\n- #1874", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1887/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1455928469, "node_id": "I_kwDOBm6k_c5Wx7SV", "number": 1903, "title": "Refactor all error classes into a datasette.exceptions module", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": {"value": 3268330, "label": "Datasette 1.0"}, "comments": 2, "created_at": "2022-11-18T22:44:45Z", "updated_at": "2022-11-20T22:35:01Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "While working on this issue:\r\n- #1896\r\n\r\nI realized that Datasette has error classes scattered around a fair bit, including some in the `datasette.utils.asgi` module for some reason.\r\n\r\nI should clean these up.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1903/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1468519699, "node_id": "I_kwDOBm6k_c5Xh9UT", "number": 1911, "title": "`/db/-/create` should support creating tables with compound primary keys", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 8658075, "label": "Datasette 1.0a0"}, "comments": 2, "created_at": "2022-11-29T18:30:47Z", "updated_at": "2022-11-29T18:50:58Z", "closed_at": "2022-11-29T18:48:05Z", "author_association": "OWNER", "pull_request": null, "body": "Found myself needing this to write the tests for:\r\n- #1864 ", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1911/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1469043836, "node_id": "I_kwDOBm6k_c5Xj9R8", "number": 1917, "title": "Don't allow writable API to edit the `_memory` database", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 7867486, "label": "Datasette 1.0a1"}, "comments": 2, "created_at": "2022-11-30T04:51:59Z", "updated_at": "2022-11-30T05:07:56Z", "closed_at": "2022-11-30T05:07:55Z", "author_association": "OWNER", "pull_request": null, "body": "It shows up on https://latest.datasette.io/-/api (once you are signed in as root) - but there's no point in creating tables in it because they likely won't persist from one request to the next, as it's not a shared named database.\r\n\r\n\"image\"\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1917/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1483250004, "node_id": "I_kwDOBm6k_c5YaJlU", "number": 1936, "title": "Fix /db/table/-/upsert in the API explorer", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": {"value": 3268330, "label": "Datasette 1.0"}, "comments": 2, "created_at": "2022-12-08T00:59:34Z", "updated_at": "2022-12-08T01:36:02Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "Split from:\r\n- #1931\r\n- #1878\r\n\r\nThis is a bit tricky because the code needs to figure out what the primary keys are for an item, and whether or not `rowid` should be included.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1936/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1483320357, "node_id": "I_kwDOBm6k_c5Yaawl", "number": 1937, "title": "/db/-/create API should require insert-rows permission to use row: or rows: option", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 8711695, "label": " Datasette 1.0a2"}, "comments": 2, "created_at": "2022-12-08T01:33:09Z", "updated_at": "2022-12-14T20:21:26Z", "closed_at": "2022-12-14T20:21:26Z", "author_association": "OWNER", "pull_request": null, "body": "Otherwise someone with `create-table` but no` insert-rows` permission could abuse it to insert data.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1937/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1493339206, "node_id": "I_kwDOBm6k_c5ZAoxG", "number": 1946, "title": "`datasette --get` mechanism for sending tokens", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 8711695, "label": " Datasette 1.0a2"}, "comments": 2, "created_at": "2022-12-13T04:25:05Z", "updated_at": "2022-12-13T04:36:57Z", "closed_at": "2022-12-13T04:36:57Z", "author_association": "OWNER", "pull_request": null, "body": "> For the tests for `datasette create-token` it would be useful if `datasette --get` had a mechanism for sending an `Authorization: Bearer X` header.\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1855#issuecomment-1347731288_\r\n ", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1946/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1495241162, "node_id": "I_kwDOBm6k_c5ZH5HK", "number": 1950, "title": "Bad ?_sort returns a 500 error, should be a 400", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2022-12-13T22:08:16Z", "updated_at": "2022-12-13T22:23:22Z", "closed_at": "2022-12-13T22:23:22Z", "author_association": "OWNER", "pull_request": null, "body": "https://latest.datasette.io/fixtures/facetable?_sort=bad\r\n\r\n\"image\"\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1950/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1495821607, "node_id": "I_kwDOBm6k_c5ZKG0n", "number": 1953, "title": "Release notes for Datasette 1.0a2", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 8711695, "label": " Datasette 1.0a2"}, "comments": 2, "created_at": "2022-12-14T06:26:40Z", "updated_at": "2022-12-15T02:02:15Z", "closed_at": "2022-12-15T02:01:08Z", "author_association": "OWNER", "pull_request": null, "body": "https://github.com/simonw/datasette/milestone/27?closed=1\r\n\r\nhttps://github.com/simonw/datasette/compare/1.0a1...9ad76d279e2c3874ca5070626a25458ce129f126", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1953/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1497288666, "node_id": "I_kwDOBm6k_c5ZPs_a", "number": 1956, "title": "Handle abbreviations properly in permission_allowed_actor_restrictions", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 8711695, "label": " Datasette 1.0a2"}, "comments": 2, "created_at": "2022-12-14T19:54:21Z", "updated_at": "2022-12-14T20:04:29Z", "closed_at": "2022-12-14T20:04:28Z", "author_association": "OWNER", "pull_request": null, "body": "This code currently assumes abbreviations are:\r\n\r\n ```pyton\r\naction_initials = \"\".join([word[0] for word in action.split(\"-\")])\r\n```\r\n\r\nhttps://github.com/simonw/datasette/blob/1a3dcf494376e32f7cff110c86a88e5b0a3f3924/datasette/default_permissions.py#L182-L208\r\n\r\nThat's no longer correct, they are now registered by the new plugin hook:\r\n- #1939 ", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1956/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1501713288, "node_id": "I_kwDOBm6k_c5ZglOI", "number": 1963, "title": "0.63.3 bugfix release", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2022-12-18T02:48:15Z", "updated_at": "2022-12-18T03:26:55Z", "closed_at": "2022-12-18T03:26:55Z", "author_association": "OWNER", "pull_request": null, "body": "I'm going to ship a release which back-ports these two fixes:\r\n\r\n- https://github.com/simonw/datasette/issues/1958\r\n- https://github.com/simonw/datasette/issues/1955", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1963/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1524867951, "node_id": "I_kwDOBm6k_c5a46Nv", "number": 1980, "title": "\"Cannot sort table by id\" when sortable_columns is used", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2023-01-09T03:21:33Z", "updated_at": "2023-01-09T03:23:53Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "I had an instance with this in `metadata.yml`:\r\n\r\n```yaml\r\ndatabases:\r\n timezones:\r\n tables:\r\n timezones:\r\n sortable_columns:\r\n - tzid\r\n```\r\nWhen I clicked on the \"Apply\" button here:\r\n\r\n\"image\"\r\n\r\nIt sent me to `/timezones/timezones?_sort=id&id__exact=133` with the error message:\r\n\r\n> 500: Cannot sort table by id", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1980/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1515182998, "node_id": "I_kwDOBm6k_c5aT9uW", "number": 1970, "title": "Path \"None\" in _internal database table", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2022-12-31T18:51:05Z", "updated_at": "2022-12-31T19:22:58Z", "closed_at": "2022-12-31T18:52:49Z", "author_association": "OWNER", "pull_request": null, "body": "See https://latest.datasette.io/_internal/databases (after https://latest.datasette.io/login-as-root)\r\n\r\n\"image\"\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1970/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1522778923, "node_id": "I_kwDOBm6k_c5aw8Mr", "number": 1978, "title": "Document datasette.urls.row and row_blob", "user": {"value": 25778, "label": "eyeseast"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2023-01-06T15:45:51Z", "updated_at": "2023-01-09T14:30:00Z", "closed_at": "2023-01-09T14:30:00Z", "author_association": "CONTRIBUTOR", "pull_request": null, "body": "These are in the codebase but not in documentation. I think everything else in this class is documented.\r\n\r\n```python\r\nclass Urls:\r\n ...\r\n def row(self, database, table, row_path, format=None):\r\n path = f\"{self.table(database, table)}/{row_path}\"\r\n if format is not None:\r\n path = path_with_format(path=path, format=format)\r\n return PrefixedUrlString(path)\r\n\r\n def row_blob(self, database, table, row_path, column):\r\n return self.table(database, table) + \"/{}.blob?_blob_column={}\".format(\r\n row_path, urllib.parse.quote_plus(column)\r\n )\r\n```\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1978/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "not_planned"} {"id": 1558644003, "node_id": "I_kwDOBm6k_c5c5wUj", "number": 2006, "title": "Teach `datasette publish` to pin to `datasette<1.0` in a 0.x release", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": {"value": 3268330, "label": "Datasette 1.0"}, "comments": 2, "created_at": "2023-01-26T19:17:40Z", "updated_at": "2023-01-26T19:20:53Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "I just realized that when I ship Datasette 1.0 there may be automated deployments out there which could deploy the 1.0 version by accident, potentially breaking any customizations that aren't compatible with the 1.0 changes.\r\n\r\nI can hopefully help avoid that by shipping one last entry in the `0.x` series that ensures `datasette publish` pins to `<1.0` when it installs Datasette itself.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/2006/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1551113681, "node_id": "I_kwDOBm6k_c5cdB3R", "number": 1998, "title": "`datasette --version` should also show the SQLite version", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2023-01-20T16:11:30Z", "updated_at": "2023-01-20T18:19:06Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "Idea came up here: https://discord.com/channels/823971286308356157/823971286941302908/1066026473003159783", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1998/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1579695809, "node_id": "I_kwDOBm6k_c5eKD7B", "number": 2023, "title": "Error: Invalid setting 'hash_urls' in settings.json in 0.64.1", "user": {"value": 80409402, "label": "mlaparie"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2023-02-10T13:35:01Z", "updated_at": "2023-02-10T15:40:00Z", "closed_at": "2023-02-10T15:39:59Z", "author_association": "NONE", "pull_request": null, "body": "On a Debian machine, using datasette 0.64.1 installed with `pip3`, I am getting a `datasette[114272]: Error: Invalid setting 'hash_urls' in settings.json` in `journalctl -xe`. The same settings work on 0.54.1 on another Debian server.\r\n\r\nThis is my `settings.json`:\r\n\r\n```json\r\n{\r\n \"default_page_size\": 200,\r\n \"max_returned_rows\": 8000,\r\n \"num_sql_threads\": 3,\r\n \"sql_time_limit_ms\": 1000,\r\n \"default_facet_size\": 30,\r\n \"facet_time_limit_ms\": 200,\r\n \"facet_suggest_time_limit_ms\": 50,\r\n \"hash_urls\": false,\r\n \"allow_facet\": true,\r\n \"allow_download\": true,\r\n \"suggest_facets\": true,\r\n \"default_cache_ttl\": 5,\r\n \"default_cache_ttl_hashed\": 31536000,\r\n \"cache_size_kb\": 0,\r\n \"allow_csv_stream\": true,\r\n \"max_csv_mb\": 100,\r\n \"truncate_cells_html\": 2048,\r\n \"force_https_urls\": false,\r\n \"template_debug\": false,\r\n \"base_url\": \"/pclim/db/\"\r\n}\r\n```\r\n\r\nThis looks ok to me. Would you have any ideas?", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/2023/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1594383280, "node_id": "I_kwDOBm6k_c5fCFuw", "number": 2030, "title": "How to use Datasette with apache webserver on GCP?", "user": {"value": 19700859, "label": "gk7279"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2023-02-22T03:08:49Z", "updated_at": "2023-02-22T21:54:39Z", "closed_at": "2023-02-22T21:54:39Z", "author_association": "NONE", "pull_request": null, "body": "Hi Simon and Datasette team-\r\n\r\nI have installed apache2 webserver inside GCP VM using apt.\r\n\r\nI can see my \"Hello World\" index.html if I use the external IP of this GCP in a browser.\r\n\r\nHowever, when I try to run datasette with different combinations of -h and -p, I am still unable to access the webpage.\r\n\r\nI cannot invest Docker on this VM.\r\n\r\nAny pointers to use datasette with already existing apache2 webserver on GCP is appreciated.\r\n\r\nThanks.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/2030/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1612296210, "node_id": "I_kwDOBm6k_c5gGbAS", "number": 2033, "title": "`datasette install -r requirements.txt`", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2023-03-06T22:17:17Z", "updated_at": "2023-03-06T22:54:52Z", "closed_at": "2023-03-06T22:27:34Z", "author_association": "OWNER", "pull_request": null, "body": "Would be useful for cases where you want to install a whole set of plugins in one go, e.g. when running tutorials in GitHub Codespaces.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/2033/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1618249044, "node_id": "I_kwDOBm6k_c5gdIVU", "number": 2038, "title": "Consider a `strict_templates` setting", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2023-03-10T02:09:13Z", "updated_at": "2023-03-10T02:11:06Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "A setting which turns on Jinja strict mode, so any templates that access undefined variables raise a hard error.\r\n\r\nPrototype here:\r\n```diff\r\ndiff --git a/datasette/app.py b/datasette/app.py\r\nindex 40416713..1428a3f0 100644\r\n--- a/datasette/app.py\r\n+++ b/datasette/app.py\r\n@@ -200,6 +200,7 @@ SETTINGS = (\r\n \"Allow display of SQL trace debug information with ?_trace=1\",\r\n ),\r\n Setting(\"base_url\", \"/\", \"Datasette URLs should use this base path\"),\r\n+ Setting(\"strict_templates\", False, \"Raise errors for undefined template variables\"),\r\n )\r\n _HASH_URLS_REMOVED = \"The hash_urls setting has been removed, try the datasette-hashed-urls plugin instead\"\r\n OBSOLETE_SETTINGS = {\r\n@@ -399,11 +400,14 @@ class Datasette:\r\n ),\r\n ]\r\n )\r\n+ env_extras = {}\r\n+ if self.setting(\"strict_templates\"):\r\n+ env_extras[\"undefined\"] = StrictUndefined\r\n self.jinja_env = Environment(\r\n loader=template_loader,\r\n autoescape=True,\r\n enable_async=True,\r\n- undefined=StrictUndefined,\r\n+ **env_extras,\r\n )\r\n self.jinja_env.filters[\"escape_css_string\"] = escape_css_string\r\n self.jinja_env.filters[\"quote_plus\"] = urllib.parse.quote_plus\r\n```\r\nExplored this idea a bit in:\r\n- #1999", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/2038/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1633077183, "node_id": "I_kwDOBm6k_c5hVse_", "number": 2041, "title": "Remove obsolete table POST code", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 8755003, "label": "Datasette 1.0a-next"}, "comments": 2, "created_at": "2023-03-21T01:01:40Z", "updated_at": "2023-03-21T01:17:44Z", "closed_at": "2023-03-21T01:17:43Z", "author_association": "OWNER", "pull_request": null, "body": "Spotted this in:\r\n- #1999\r\n\r\n`POST /db/table` currently executes obsolete code for inserting a row - I replaced that with `/db/table/-/insert` in \r\nhttps://github.com/simonw/datasette/commit/6e788b49edf4f842c0817f006eb9d865778eea5e but forgot to remove the old code.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/2041/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1702354223, "node_id": "I_kwDOBm6k_c5ld90v", "number": 2070, "title": "Mechanism for deploying a preview of a branch using Vercel", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2023-05-09T16:21:45Z", "updated_at": "2023-05-09T16:25:00Z", "closed_at": "2023-05-09T16:24:31Z", "author_association": "OWNER", "pull_request": null, "body": "I prototyped that here: https://github.com/simonw/one-off-actions/blob/main/.github/workflows/deploy-datasette-branch-preview.yml\r\n\r\nIt deployed the `json-extras-query` branch here: https://datasette-preview-json-extras-query.vercel.app/", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/2070/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1708030220, "node_id": "I_kwDOBm6k_c5lznkM", "number": 2073, "title": "Faceting doesn't work against integer columns in views", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2023-05-12T18:20:10Z", "updated_at": "2023-05-12T18:24:07Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "Spotted this issue here: https://til.simonwillison.net/datasette/baseline\r\n\r\nI had to do this workaround:\r\n```sql\r\ncreate view baseline as select\r\n _key,\r\n spec,\r\n '' || json_extract(status, '$.is_baseline') as is_baseline,\r\n json_extract(status, '$.since') as baseline_since,\r\n json_extract(status, '$.support.chrome') as baseline_chrome,\r\n json_extract(status, '$.support.edge') as baseline_edge,\r\n json_extract(status, '$.support.firefox') as baseline_firefox,\r\n json_extract(status, '$.support.safari') as baseline_safari,\r\n compat_features,\r\n caniuse,\r\n usage_stats,\r\n status\r\nfrom\r\n [index]\r\n```\r\nI think the core issue here is that, against a table, `select * from x where integer_column = '1'` works correctly, due to some kind of column type conversion mechanism... but this mechanism doesn't work against views.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/2073/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1765870617, "node_id": "I_kwDOBm6k_c5pQQwZ", "number": 2087, "title": "`--settings settings.json` option", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2023-06-20T17:48:45Z", "updated_at": "2023-07-14T17:02:03Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "https://discord.com/channels/823971286308356157/823971286941302908/1120705940728066080\r\n\r\n> May I add a request to the whole metadata / settings ? Allow to pass `--settings path/to/settings.json` instead of having to rely exclusively on directory mode to centralize settings (this would reflect the behavior of providing metadata)", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/2087/reactions\", \"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1781005740, "node_id": "I_kwDOBm6k_c5qJ_2s", "number": 2090, "title": "Adopt ruff for linting", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2023-06-29T14:56:43Z", "updated_at": "2023-06-29T15:05:04Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "https://beta.ruff.rs/docs/", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/2090/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1781047747, "node_id": "I_kwDOBm6k_c5qKKHD", "number": 2092, "title": "test_homepage intermittent failure", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2023-06-29T15:20:37Z", "updated_at": "2023-06-29T15:26:28Z", "closed_at": "2023-06-29T15:24:13Z", "author_association": "OWNER", "pull_request": null, "body": "e.g. in https://github.com/simonw/datasette/actions/runs/5413590227/jobs/9839373852\r\n\r\n```\r\n=================================== FAILURES ===================================\r\n________________________________ test_homepage _________________________________\r\n[gw0] linux -- Python 3.7.17 /opt/hostedtoolcache/Python/3.7.17/x64/bin/python\r\n\r\nds_client = \r\n\r\n @pytest.mark.asyncio\r\n async def test_homepage(ds_client):\r\n response = await ds_client.get(\"/.json\")\r\n assert response.status_code == 200\r\n assert \"application/json; charset=utf-8\" == response.headers[\"content-type\"]\r\n data = response.json()\r\n assert data.keys() == {\"fixtures\": 0}.keys()\r\n d = data[\"fixtures\"]\r\n assert d[\"name\"] == \"fixtures\"\r\n assert d[\"tables_count\"] == 24\r\n assert len(d[\"tables_and_views_truncated\"]) == 5\r\n assert d[\"tables_and_views_more\"] is True\r\n # 4 hidden FTS tables + no_primary_key (hidden in metadata)\r\n assert d[\"hidden_tables_count\"] == 6\r\n # 201 in no_primary_key, plus 6 in other hidden tables:\r\n> assert d[\"hidden_table_rows_sum\"] == 207, data\r\nE AssertionError: {'fixtures': {'color': '9403e5', 'hash': None, 'hidden_table_rows_sum': 0, 'hidden_tables_count': 6, ...}}\r\nE assert 0 == 207\r\n```\r\nMy guess is that this is a timing error, where very occasionally the \"count rows but stop counting if it exceeds a time limit\" thing fails.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/2092/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1808215339, "node_id": "I_kwDOBm6k_c5rxy0r", "number": 2104, "title": "Tables starting with an underscore should be treated as hidden", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2023-07-17T17:13:53Z", "updated_at": "2023-07-18T22:41:37Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "Plugins can then take advantage of this pattern, for example:\r\n- https://github.com/simonw/datasette-auth-tokens/pull/8", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/2104/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1825007061, "node_id": "I_kwDOBm6k_c5sx2XV", "number": 2123, "title": "datasette serve when invoked with --reload interprets the serve command as a file", "user": {"value": 79087, "label": "cadeef"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2023-07-27T19:07:22Z", "updated_at": "2023-09-18T13:02:46Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "When running `datasette serve` with the `--reload` flag, the serve command is picked up as a file argument:\r\n\r\n```\r\n$ datasette serve --reload test_db\r\nStarting monitor for PID 13574.\r\nError: Invalid value for '[FILES]...': Path 'serve' does not exist.\r\nPress ENTER or change a file to reload.\r\n```\r\n\r\nIf a 'serve' file is created it launches properly (albeit with an empty database called serve):\r\n\r\n```\r\n$ touch serve; datasette serve --reload test_db\r\nStarting monitor for PID 13628.\r\nINFO: Started server process [13628]\r\nINFO: Waiting for application startup.\r\nINFO: Application startup complete.\r\nINFO: Uvicorn running on http://127.0.0.1:8001 (Press CTRL+C to quit)\r\n```\r\n\r\nVersion (running from HEAD on main):\r\n\r\n```\r\n$ datasette --version\r\ndatasette, version 1.0a2\r\n```\r\n\r\nThis issue appears to have existed for awhile as https://github.com/simonw/datasette/issues/1380#issuecomment-953366110 mentions the error in a different context.\r\n\r\nI'm happy to debug and land a patch if it's welcome.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/2123/reactions\", \"total_count\": 2, \"+1\": 2, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1844213115, "node_id": "I_kwDOBm6k_c5t7HV7", "number": 2138, "title": "on_success_message_sql option for writable canned queries", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 8755003, "label": "Datasette 1.0a-next"}, "comments": 2, "created_at": "2023-08-10T00:20:14Z", "updated_at": "2023-08-10T00:39:40Z", "closed_at": "2023-08-10T00:34:26Z", "author_association": "OWNER", "pull_request": null, "body": "> Or... how about if the `on_success_message` option could define a SQL query to be executed to generate that message? Maybe `on_success_message_sql`.\r\n\r\n- https://github.com/simonw/datasette/issues/2134", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/2138/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1840329615, "node_id": "I_kwDOBm6k_c5tsTOP", "number": 2130, "title": "Render plugin mechanism needs `error` and `truncated` fields", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 9700784, "label": "Datasette 1.0a3"}, "comments": 2, "created_at": "2023-08-07T23:19:19Z", "updated_at": "2023-08-08T01:51:54Z", "closed_at": "2023-08-08T01:47:42Z", "author_association": "OWNER", "pull_request": null, "body": "While working on:\r\n- https://github.com/simonw/datasette/pull/2118\r\n\r\nIt became clear that the `render` callback function documented here: https://docs.datasette.io/en/0.64.3/plugin_hooks.html#register-output-renderer-datasette\r\n\r\nNeeds to grow the ability to be told if an error occurred (an `error` string) and if the results were truncated (a `truncated` boolean).", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/2130/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1846076261, "node_id": "I_kwDOBm6k_c5uCONl", "number": 2139, "title": "border-color: ##ff0000 bug - two hashes", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 8755003, "label": "Datasette 1.0a-next"}, "comments": 2, "created_at": "2023-08-11T01:22:58Z", "updated_at": "2023-08-11T05:16:24Z", "closed_at": "2023-08-11T05:16:24Z", "author_association": "OWNER", "pull_request": null, "body": "Spotted this on https://latest.datasette.io/extra_database\r\n\r\n```html\r\n
\r\n```", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/2139/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1910269679, "node_id": "I_kwDOBm6k_c5x3Gbv", "number": 2196, "title": "Discord invite link returns 401", "user": {"value": 1892194, "label": "Olshansk"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2023-09-24T15:16:54Z", "updated_at": "2023-10-13T00:07:08Z", "closed_at": "2023-10-12T21:54:54Z", "author_association": "NONE", "pull_request": null, "body": "I found the link to the datasette discord channel via [this query](https://github.com/search?q=repo%3Asimonw%2Fdatasette%20discord&type=code).\r\n\r\nThe following video should be self explanatory:\r\n\r\nhttps://github.com/simonw/datasette/assets/1892194/8cd33e88-bcaa-41f3-9818-ab4d589c3f02\r\n\r\n\r\nLink for reference: https://discord.com/invite/ktd74dm5mw", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/2196/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1898927976, "node_id": "I_kwDOBm6k_c5xL1do", "number": 2186, "title": "Mechanism for register_output_renderer hooks to access full count", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": {"value": 3268330, "label": "Datasette 1.0"}, "comments": 2, "created_at": "2023-09-15T18:57:54Z", "updated_at": "2023-09-15T19:27:59Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "The cause of this bug:\r\n- https://github.com/simonw/datasette-export-notebook/issues/17\r\n\r\nIs that `datasette-export-notebook` was consulting `data[\"filtered_table_rows_count\"]` in the render output plugin function in order to show the total number of rows that would be exported.\r\n\r\nThat field is no longer available by default - the `\"count\"` field is only available if `?_extra=count` was passed.\r\n\r\nIt would be useful if plugins like this could access the total count on demand, should they need to.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/2186/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1900026059, "node_id": "I_kwDOBm6k_c5xQBjL", "number": 2188, "title": "Plugin Hooks for \"compile to SQL\" languages", "user": {"value": 15178711, "label": "asg017"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2023-09-18T01:37:15Z", "updated_at": "2023-09-18T06:58:53Z", "closed_at": null, "author_association": "CONTRIBUTOR", "pull_request": null, "body": "There's a ton of tools/languages that compile to SQL, which may be nice in Datasette. Some examples:\r\n\r\n- Logica https://logica.dev\r\n- PRQL https://prql-lang.org\r\n- Malloy, but not sure if it works with SQLite? https://github.com/malloydata/malloy\r\n\r\nIt would be cool if plugins could extend Datasette to use these languages, in both the code editor and API usage.\r\n\r\nA few things I'd imagine a `datasette-prql` or `datasette-logica` plugin would do:\r\n\r\n- `prql=` instead of `sql=`\r\n- Code editor support (syntax highlighting, autocomplete)\r\n- Hide/show SQL", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/2188/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1515883470, "node_id": "I_kwDOC8tyDs5aWovO", "number": 24, "title": "DOC: xml.etree.ElementTree.ParseError due to healthkit version 12 ", "user": {"value": 6231413, "label": "mmngreco"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2023-01-01T23:00:38Z", "updated_at": "2023-03-30T10:17:31Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "Hi @simonw \r\n\r\nI hope you find this issue ok, the idea is provide some documentation to other users like me about how to solve this problem and save some time.\r\n\r\nFollowing the instructions from the `README.md` I've faced this error:\r\n\r\n```bash\r\n(venv) mgreco@pop-os apple-health master* (23:44|0s)\r\n$ healthkit-to-sqlite apple_health_export/export.xml healthkit.db --xml\r\nImporting from HealthKit [------------------------------------] 0%\r\nTraceback (most recent call last):\r\n File \"/home/mgreco/github/mmngreco/apple-health/venv/bin/healthkit-to-sqlite\", line 33, in \r\n sys.exit(load_entry_point('healthkit-to-sqlite', 'console_scripts', 'healthkit-to-sqlite')())\r\n File \"/home/mgreco/github/mmngreco/apple-health/venv/lib/python3.10/site-packages/click/core.py\", line 1130, in __call__\r\n return self.main(*args, **kwargs)\r\n File \"/home/mgreco/github/mmngreco/apple-health/venv/lib/python3.10/site-packages/click/core.py\", line 1055, in main\r\n rv = self.invoke(ctx)\r\n File \"/home/mgreco/github/mmngreco/apple-health/venv/lib/python3.10/site-packages/click/core.py\", line 1404, in invoke\r\n return ctx.invoke(self.callback, **ctx.params)\r\n File \"/home/mgreco/github/mmngreco/apple-health/venv/lib/python3.10/site-packages/click/core.py\", line 760, in invoke\r\n return __callback(*args, **kwargs)\r\n File \"/home/mgreco/github/mmngreco/apple-health/.deps/healthkit-to-sqlite/healthkit_to_sqlite/cli.py\", line 57, in cli\r\n convert_xml_to_sqlite(fp, db, progress_callback=bar.update, zipfile=zf)\r\n File \"/home/mgreco/github/mmngreco/apple-health/.deps/healthkit-to-sqlite/healthkit_to_sqlite/utils.py\", line 25, in convert_xml_to_sqlite\r\n for tag, el in find_all_tags(\r\n File \"/home/mgreco/github/mmngreco/apple-health/.deps/healthkit-to-sqlite/healthkit_to_sqlite/utils.py\", line 12, in find_all_tags\r\n for event, el in parser.read_events():\r\n File \"/home/mgreco/github/mmngreco/apple-health/venv/lib/python3.10/xml/etree/ElementTree.py\", line 1324, in read_events\r\n raise event\r\n File \"/home/mgreco/github/mmngreco/apple-health/venv/lib/python3.10/xml/etree/ElementTree.py\", line 1296, in feed\r\n self._parser.feed(data)\r\nxml.etree.ElementTree.ParseError: syntax error: line 156, column 0\r\n```\r\n\r\nSo, after debugging and searching on internet I found this useful link: https://discussions.apple.com/thread/254202523 (etresoft, the real hero). Which basically says that the xml given by the health app (healthkit version 12) has some bugs but fortunately, they can be solved with a couple of commads:\r\n\r\n1. Uncompress the zip and move the new folder where `export.xml` is.\r\n1. Create a `patch.txt` with the following content\r\n\r\n ```diff\r\n --- export.xml\t2022-09-18 15:17:09.000000000 -0400\r\n +++ export-fixed.xml\t2022-09-18 16:37:08.000000000 -0400\r\n @@ -15,6 +15,7 @@\r\n HKCharacteristicTypeIdentifierBiologicalSex CDATA #REQUIRED\r\n HKCharacteristicTypeIdentifierBloodType CDATA #REQUIRED\r\n HKCharacteristicTypeIdentifierFitzpatrickSkinType CDATA #REQUIRED\r\n + HKCharacteristicTypeIdentifierCardioFitnessMedicationsUse CDATA #IMPLIED\r\n >\r\n \r\n \r\n -\r\n +\r\n \r\n -\r\n +\r\n \r\n \r\n \r\n \r\n \r\n - device CDATA #IMPLIED\r\n -\r\n -\r\n ->\r\n ]>\r\n \r\n \r\n ```\r\n1. Apply the path with the command: `patch < patch.txt`\r\n1. Fix endDates with the command `sed 's/startDate/endDate/2' export.xml > export-fixed.xml`\r\n1. Try again `healthkit-to-sqlite export-fixed.xml healthkit.db --xml`", "repo": {"value": 197882382, "label": "healthkit-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/24/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1026794056, "node_id": "I_kwDOCGYnMM49M6JI", "number": 331, "title": "Mypy error: found module but no type hints or library stubs", "user": {"value": 53032010, "label": "andreaslongo"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2021-10-14T20:29:50Z", "updated_at": "2021-11-14T23:21:08Z", "closed_at": "2021-11-14T23:21:08Z", "author_association": "NONE", "pull_request": null, "body": "```\r\nPython 3.9.5\r\nmypy 0.910\r\nsqlite-utils 3.17.1\r\n```\r\n\r\nWhile using sqlite-utils as a library, when I use mypy for static type checking, it throws an error:\r\n\r\n```\r\nmypy .\r\nsrc/etl.py:5: error: Skipping analyzing \"sqlite_utils\": found module but no type hints or library stubs\r\n import sqlite_utils\r\n ^\r\nsrc/etl.py:5: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports\r\ntest/test_etl.py:4: error: Skipping analyzing \"sqlite_utils\": found module but no type hints or library stubs\r\n import sqlite_utils\r\n ^\r\nFound 2 errors in 2 files (checked 7 source files)\r\n```\r\n\r\n\r\nWhen I add a `py.typed` file to the sqlite-utils package to mark it as PEP 561 compatible, the error goes away.\r\n\r\n```\r\nal@nbal ..b/python3.9/site-packages/sqlite_utils (git)-[main] % la\r\ntotal 200\r\ndrwx------ 3 al al 4096 Oct 14 22:00 .\r\ndrwx------ 117 al al 4096 Oct 12 21:12 ..\r\n-rw------- 1 al al 64409 Oct 12 21:11 cli.py\r\n-rw------- 1 al al 109092 Oct 12 21:11 db.py\r\n-rw------- 1 al al 0 Oct 14 22:00 py.typed\r\n-rw------- 1 al al 684 Oct 12 21:11 recipes.py\r\n-rw------- 1 al al 7988 Oct 12 21:11 utils.py\r\n-rw------- 1 al al 113 Oct 12 21:11 __init__.py\r\n```\r\n\r\nI would like to suggest adding a `py.typed` file to the repository.\r\n\r\nSee also the mypy docs on creating PEP 561 compatible packages:\r\nhttps://mypy.readthedocs.io/en/stable/installed_packages.html#creating-pep-561-compatible-packages\r\n\r\n", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/331/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1066501534, "node_id": "I_kwDOCGYnMM4_kYWe", "number": 345, "title": "`table.strict` introspection boolean for identifying STRICT mode tables", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2021-11-29T21:05:10Z", "updated_at": "2021-11-29T22:45:26Z", "closed_at": "2021-11-29T22:44:36Z", "author_association": "OWNER", "pull_request": null, "body": "> From the STRICT docs:\r\n\r\n>> The SQLite parser accepts a comma-separated list of table options after the final close parenthesis in a CREATE TABLE statement. As of this writing (2021-08-23) only two options are recognized:\r\n>> \r\n>> - STRICT\r\n>> - [WITHOUT ROWID](https://www.sqlite.org/withoutrowid.html)\r\n> \r\n> So I think I need to read the `CREATE TABLE` statement from the `sqlite_master` table, split on the last `)`, split those tokens on `,` and see if `create` is in there (case insensitive).\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/344#issuecomment-982020757_", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/345/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 2029161033, "node_id": "I_kwDOCGYnMM548opJ", "number": 606, "title": "str and int as aliases for text and integer", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2023-12-06T18:35:49Z", "updated_at": "2023-12-06T19:44:04Z", "closed_at": "2023-12-06T18:49:32Z", "author_association": "OWNER", "pull_request": null, "body": "I keep making this mistake:\r\n```bash\r\nsqlite-utils add-column content.db assets _since int\r\n```\r\n```\r\nUsage: sqlite-utils add-column [OPTIONS] PATH TABLE COL_NAME [[integer|float|b\r\n lob|text|INTEGER|FLOAT|BLOB|TEXT]]\r\nTry 'sqlite-utils add-column -h' for help.\r\n\r\nError: Invalid value for '[[integer|float|blob|text|INTEGER|FLOAT|BLOB|TEXT]]':\r\n 'int' is not one of 'integer', 'float', 'blob', 'text', 'INTEGER', 'FLOAT', 'BLOB', 'TEXT'.\r\n```", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/606/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1099584685, "node_id": "I_kwDOCGYnMM5BilSt", "number": 381, "title": "`sqlite-utils rows` options `--limit` and `--offset`", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2022-01-11T20:23:12Z", "updated_at": "2022-01-11T23:33:37Z", "closed_at": "2022-01-11T23:19:36Z", "author_association": "OWNER", "pull_request": null, "body": "Because I often want to use it just to preview a few rows from the database. Piping through `| head -n 20` works for JSON and CSV (they stream) but not for `--table`.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/381/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1114544727, "node_id": "I_kwDOCGYnMM5CbppX", "number": 389, "title": "Plausible analytics for documentation", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2022-01-26T01:58:35Z", "updated_at": "2022-01-26T02:07:41Z", "closed_at": "2022-01-26T02:07:41Z", "author_association": "OWNER", "pull_request": null, "body": "```html\r\n\r\n```\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/388#issuecomment-1021785268_", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/389/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1114638930, "node_id": "I_kwDOCGYnMM5CcApS", "number": 391, "title": "`sqlite-utils bulk` progress bar", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2022-01-26T05:14:49Z", "updated_at": "2022-01-26T05:17:20Z", "closed_at": "2022-01-26T05:16:51Z", "author_association": "OWNER", "pull_request": null, "body": "It can easily have a progress bar because it works by looping through an iterator: https://github.com/simonw/sqlite-utils/blob/a9fca7efa4184fbb2a65ca1275c326950ed9d3c1/sqlite_utils/cli.py#L1014-L1018\r\n\r\nShould also support the `--silent` option if I add this.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/391/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1125081640, "node_id": "I_kwDOCGYnMM5DD2Io", "number": 401, "title": "Update SpatiaLite example in the documentation", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2022-02-06T02:02:07Z", "updated_at": "2022-02-06T02:05:03Z", "closed_at": "2022-02-06T02:03:24Z", "author_association": "OWNER", "pull_request": null, "body": "This one here: https://sqlite-utils.datasette.io/en/3.23/python-api.html#converting-column-values-using-sql-functions\r\n\r\nIt should take advantage of the new methods from:\r\n- #79", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/401/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1128120451, "node_id": "I_kwDOCGYnMM5DPcCD", "number": 404, "title": "Add example of `--convert` to the help for `sqlite-utils insert`", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2022-02-09T06:49:09Z", "updated_at": "2022-02-09T06:56:35Z", "closed_at": "2022-02-09T06:55:16Z", "author_association": "OWNER", "pull_request": null, "body": "https://sqlite-utils.datasette.io/en/3.23/cli-reference.html#insert would be more useful if it included an example of `--convert` in action.\r\n\r\nI can maybe use an example from https://simonwillison.net/2022/Jan/11/sqlite-utils/", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/404/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1128139375, "node_id": "I_kwDOCGYnMM5DPgpv", "number": 405, "title": "`Database(memory_name=\"name\")` constructor argument", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2022-02-09T07:15:03Z", "updated_at": "2022-02-16T01:23:16Z", "closed_at": "2022-02-16T01:23:16Z", "author_association": "OWNER", "pull_request": null, "body": "SQLite in-memory databases can be named, in which case multiple connections can be opened to a shared in-memory database running within the same process.\r\n\r\nDatasette supports this - SQLite could support it too.\r\n\r\nhttps://docs.datasette.io/en/0.60.2/internals.html#database-ds-path-none-is-mutable-false-is-memory-false-memory-name-none", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/405/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1171599874, "node_id": "I_kwDOCGYnMM5F1TIC", "number": 415, "title": "Convert with `--multi` and `--dry-run` flag does not work", "user": {"value": 3976183, "label": "dotcs"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2022-03-16T21:59:46Z", "updated_at": "2022-03-21T04:18:24Z", "closed_at": "2022-03-21T04:18:24Z", "author_association": "NONE", "pull_request": null, "body": "It's not possible to combine `--multi` and `--dry-run` flag in the `convert` command.\r\n\r\nLet's first create a simple database from JSON string\r\n\r\n```console\r\n$ echo '[{\"foo\": \"abc\"}]' | sqlite-utils insert demo.db demo -\r\n$ sqlite-utils query demo.db \"SELECT * FROM demo\" \r\n[{\"foo\": \"abc\"}]\r\n```\r\n\r\nand then try to convert the \"foo\" column with a static value \"bar\" (see docs [Converting a column into multiple columns](https://sqlite-utils.datasette.io/en/stable/cli.html#converting-a-column-into-multiple-columns))\r\n\r\n```console\r\n$ sqlite-utils convert demo.db demo foo '{\"foo\": \"bar\"}' --multi --dry-run\r\nTraceback (most recent call last):\r\n File \"/home/dotcs/anaconda3/envs/tools/bin/sqlite-utils\", line 8, in \r\n sys.exit(cli())\r\n File \"/home/dotcs/anaconda3/envs/tools/lib/python3.9/site-packages/click/core.py\", line 1128, in __call__\r\n return self.main(*args, **kwargs)\r\n File \"/home/dotcs/anaconda3/envs/tools/lib/python3.9/site-packages/click/core.py\", line 1053, in main\r\n rv = self.invoke(ctx)\r\n File \"/home/dotcs/anaconda3/envs/tools/lib/python3.9/site-packages/click/core.py\", line 1659, in invoke\r\n return _process_result(sub_ctx.command.invoke(sub_ctx))\r\n File \"/home/dotcs/anaconda3/envs/tools/lib/python3.9/site-packages/click/core.py\", line 1395, in invoke\r\n return ctx.invoke(self.callback, **ctx.params)\r\n File \"/home/dotcs/anaconda3/envs/tools/lib/python3.9/site-packages/click/core.py\", line 754, in invoke\r\n return __callback(*args, **kwargs)\r\n File \"/home/dotcs/anaconda3/envs/tools/lib/python3.9/site-packages/sqlite_utils/cli.py\", line 2686, in convert\r\n for row in db.conn.execute(sql, where_args).fetchall():\r\nsqlite3.OperationalError: user-defined function raised exception\r\n```\r\n\r\nBut without the `--dry-run` flag it does work as expected:\r\n\r\n```console\r\n$ sqlite-utils convert demo.db demo foo '{\"foo\": \"bar\"}' --multi\r\n$ sqlite-utils query demo.db \"SELECT * FROM demo\" \r\n[{\"foo\": \"bar\"}]\r\n```\r\n\r\n```console\r\n$ sqlite-utils --version\r\nsqlite-utils, version 3.25.1\r\n```", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/415/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1224112817, "node_id": "I_kwDOCGYnMM5I9nqx", "number": 430, "title": "Document how to use `PRAGMA temp_store` to avoid errors when running VACUUM against huge databases", "user": {"value": 9308268, "label": "rayvoelker"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2022-05-03T13:33:58Z", "updated_at": "2022-06-14T23:26:37Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "I'm trying to figure out a way to get the `table.extract()` method to complete successfully -- I'm not sure if maybe the cause (and a possible solution) of this on Ubuntu Server 22.04 is to adjust some of the PRAGMA values within SQLite itself ... on another Linux system (PopOS), using this method on this same database appears to work just fine.\r\n\r\nHere's the bit that's causing the error, and the resulting error output:\r\n```python\r\n# combine these columns into 1 table \"bib_properties\" :\r\n# best_title\r\n# bib_level_code\r\n# mat_type\r\n# material_code\r\n# best_author\r\ndb[\"circ_trans\"].extract(\r\n [\"best_title\", \"bib_level_code\", \"mat_type\", \"material_code\", \"best_author\"], \r\n table=\"bib_properties\", \r\n fk_column=\"bib_properties_id\"\r\n)\r\n\r\ndb[\"circ_trans\"].extract(\r\n [\"call_number\"], \r\n table=\"call_number\", \r\n fk_column=\"call_number_id\",\r\n rename={\"call_number\": \"value\"}\r\n)\r\n```\r\n\r\n```python\r\n---------------------------------------------------------------------------\r\nOperationalError Traceback (most recent call last)\r\nInput In [17], in ()\r\n 1 # combine these columns into 1 table \"bib_properties\" :\r\n 2 # best_title\r\n 3 # bib_level_code\r\n 4 # mat_type\r\n 5 # material_code\r\n 6 # best_author\r\n----> 7 db[\"circ_trans\"].extract(\r\n 8 [\"best_title\", \"bib_level_code\", \"mat_type\", \"material_code\", \"best_author\"], \r\n 9 table=\"bib_properties\", \r\n 10 fk_column=\"bib_properties_id\"\r\n 11 )\r\n 13 db[\"circ_trans\"].extract(\r\n 14 [\"call_number\"], \r\n 15 table=\"call_number\", \r\n 16 fk_column=\"call_number_id\",\r\n 17 rename={\"call_number\": \"value\"}\r\n 18 )\r\n\r\nFile ~/jupyter/venv/lib/python3.10/site-packages/sqlite_utils/db.py:1764, in Table.extract(self, columns, table, fk_column, rename)\r\n 1761 column_order.append(c.name)\r\n 1763 # Drop the unnecessary columns and rename lookup column\r\n-> 1764 self.transform(\r\n 1765 drop=set(columns),\r\n 1766 rename={magic_lookup_column: fk_column},\r\n 1767 column_order=column_order,\r\n 1768 )\r\n 1770 # And add the foreign key constraint\r\n 1771 self.add_foreign_key(fk_column, table, \"id\")\r\n\r\nFile ~/jupyter/venv/lib/python3.10/site-packages/sqlite_utils/db.py:1526, in Table.transform(self, types, rename, drop, pk, not_null, defaults, drop_foreign_keys, column_order)\r\n 1524 with self.db.conn:\r\n 1525 for sql in sqls:\r\n-> 1526 self.db.execute(sql)\r\n 1527 # Run the foreign_key_check before we commit\r\n 1528 if pragma_foreign_keys_was_on:\r\n\r\nFile ~/jupyter/venv/lib/python3.10/site-packages/sqlite_utils/db.py:465, in Database.execute(self, sql, parameters)\r\n 463 return self.conn.execute(sql, parameters)\r\n 464 else:\r\n--> 465 return self.conn.execute(sql)\r\n\r\nOperationalError: database or disk is full\r\n```\r\n\r\nThis database is about 17G in total size, so I'm assuming the error is coming from the vacuum ... where i'm assuming it's maybe trying to do the temp storage in a location that doesn't have sufficient room. The disk space is more than ample on the host in question (1.8T is free in the directory where the sqlite db resides) The `/tmp` directory however is limited on a smaller disk associated with the OS\r\n\r\nI'm trying to think if there's a way to set the `PRAGMA temp_store` or maybe if it's `temp_store_directory` that I'm after ... to use the same local directory of where the file is located (maybe this is a property of the version of sqlite on the system?) \r\n\r\n```python\r\n# SET the temp file store to be a file ...\r\nprint(db.execute('PRAGMA temp_store').fetchall())\r\nprint(db.execute('PRAGMA temp_store=FILE').fetchall())\r\n\r\nprint(db.execute('PRAGMA temp_store').fetchall())\r\n\r\n# the users home directory ...\r\nprint(db.execute(\"PRAGMA temp_store_directory='/home/plchuser/'\").fetchall())\r\nprint(db.execute(\"PRAGMA sqlite3_temp_directory='/home/plchuser/'\").fetchall())\r\n\r\nprint(db.execute(\"PRAGMA temp_store_directory\").fetchall())\r\nprint(db.execute(\"PRAGMA sqlite3_temp_directory\").fetchall())\r\n```\r\n```text\r\n[(1,)]\r\n[]\r\n[(1,)]\r\n[]\r\n[]\r\n[('/home/plchuser/',)]\r\n[]\r\n```\r\n\r\nHere's the docs on the Temporary File Storage Locations \r\nhttps://www.sqlite.org/tempfiles.html", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/430/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1243704847, "node_id": "I_kwDOCGYnMM5KIW4P", "number": 435, "title": "Switch to Furo documentation theme", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2022-05-20T21:46:39Z", "updated_at": "2022-05-20T21:56:10Z", "closed_at": "2022-05-20T21:54:43Z", "author_association": "OWNER", "pull_request": null, "body": "As seen in:\r\n- https://github.com/simonw/datasette/issues/1746\r\n- https://github.com/simonw/shot-scraper/issues/77", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/435/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1250161887, "node_id": "I_kwDOCGYnMM5Kg_Tf", "number": 438, "title": "illegal UTF-16 surrogate", "user": {"value": 4068, "label": "frafra"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2022-05-26T22:49:52Z", "updated_at": "2022-05-27T08:21:53Z", "closed_at": "2022-05-27T08:21:53Z", "author_association": "NONE", "pull_request": null, "body": "I am trying to insert `https://artsdatabanken.no/Fab2018/api/export/csv` into a SQLite database, but I have an error when using `sqlite-utils`:\r\n\r\n```\r\nsqlite-utils insert --csv --delimiter \";\" --encoding=\"utf-16-le\" --pk \"Id\" csv fremmedart test.db\r\n [------------------------------------] 0%\r\nError: 'utf-16-le' codec can't decode bytes in position 98-99: illegal UTF-16 surrogate\r\n\r\nThe input you provided uses a character encoding other than utf-8.\r\n\r\nYou can fix this by passing the --encoding= option with the encoding of the file.\r\n\r\nIf you do not know the encoding, running 'file filename.csv' may tell you.\r\n\r\nIt's often worth trying: --encoding=latin-1\r\n```\r\n\r\nI tried to convert the file using `iconv -f \"utf-16le\" -t \"utf-8\"`, but I still get a similar error (slightly different position):\r\n\r\n```\r\nsqlite-utils insert --csv --delimiter \";\" --encoding=utf-8 --pk \"Id\" csv_utf8 fremmedart test.db\r\n [------------------------------------] 0%\r\nError: 'utf-8' codec can't decode byte 0xd9 in position 99: invalid continuation byte\r\n\r\nThe input you provided uses a character encoding other than utf-8.\r\n\r\nYou can fix this by passing the --encoding= option with the encoding of the file.\r\n\r\nIf you do not know the encoding, running 'file filename.csv' may tell you.\r\n\r\nIt's often worth trying: --encoding=latin-1\r\n```\r\n\r\nI have no issues reading such file using this Python code:\r\n```python\r\ncontent = open('csv', encoding='utf-16-le').read())\r\n```\r\n\r\n`in2csv` works too.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/438/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1269886084, "node_id": "I_kwDOCGYnMM5LsOyE", "number": 442, "title": "`maximize_csv_field_size_limit()` utility function", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2022-06-13T19:54:54Z", "updated_at": "2022-06-14T21:55:15Z", "closed_at": "2022-06-14T21:31:49Z", "author_association": "OWNER", "pull_request": null, "body": "This code here runs only if `cli.py` is imported: https://github.com/simonw/sqlite-utils/blob/7ddf5300886a32d6daf60cf1d71efe492b65c87e/sqlite_utils/cli.py#L50-L59\r\n\r\nI found myself needing the same fix in another library:\r\n\r\n- https://github.com/simonw/datasette-socrata/issues/13\r\n\r\nIt should be a documented utility function.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/442/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1269998342, "node_id": "I_kwDOCGYnMM5LsqMG", "number": 443, "title": "Make `utils.rows_from_file()` a documented API", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2022-06-13T21:53:24Z", "updated_at": "2022-06-20T19:49:37Z", "closed_at": "2022-06-14T20:12:46Z", "author_association": "OWNER", "pull_request": null, "body": "> `rows_from_file()` isn't part of the documented API but maybe it should be!\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/440#issuecomment-1154385916_", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/443/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1277328147, "node_id": "I_kwDOCGYnMM5MInsT", "number": 446, "title": "Use Just to automate running tests and linters locally", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2022-06-20T19:51:09Z", "updated_at": "2022-06-21T19:28:35Z", "closed_at": "2022-06-20T19:54:50Z", "author_association": "OWNER", "pull_request": null, "body": "I keep committing code that fails additional tests like `mypy` and `flake8` and `black`. Automate those using Just.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/446/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1298531653, "node_id": "I_kwDOCGYnMM5NZgVF", "number": 451, "title": "Make sqlite_utils.utils.chunks a documented function", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2022-07-08T06:01:04Z", "updated_at": "2022-07-15T22:09:34Z", "closed_at": "2022-07-15T21:59:33Z", "author_association": "OWNER", "pull_request": null, "body": "I want to use it in another project: https://github.com/simonw/sqlite-utils/blob/8a9fe6498faf783a1fdeb1793e661ad194a05267/sqlite_utils/utils.py#L471-L474", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/451/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1352946135, "node_id": "I_kwDOCGYnMM5QpFHX", "number": 472, "title": "Reuse the locals/globals fix from --functions for other code accepting options", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 8355157, "label": "3.29"}, "comments": 2, "created_at": "2022-08-27T05:12:05Z", "updated_at": "2022-08-27T05:20:12Z", "closed_at": "2022-08-27T05:20:12Z", "author_association": "OWNER", "pull_request": null, "body": "I figured out a workaround for the ugly `global x` hack here:\r\n- https://github.com/simonw/sqlite-utils/issues/471#issuecomment-1229120653", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/472/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1353441389, "node_id": "I_kwDOCGYnMM5Qq-Bt", "number": 477, "title": "Conda Forge", "user": {"value": 49702524, "label": "thewchan"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2022-08-28T19:03:08Z", "updated_at": "2022-09-07T03:46:55Z", "closed_at": "2022-09-07T03:46:55Z", "author_association": "NONE", "pull_request": null, "body": "Hello! I have successfully put this package on to Conda Forge, and I have extending the invitation for the owner/maintainers of this package to be maintainers on Conda Forge as well. Let me know if you are interested! Thanks.\r\nhttps://github.com/conda-forge/sqlite-utils-feedstock", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/477/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1353196970, "node_id": "I_kwDOCGYnMM5QqCWq", "number": 476, "title": "Release notes for 3.29", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 8355157, "label": "3.29"}, "comments": 2, "created_at": "2022-08-27T23:21:21Z", "updated_at": "2022-08-28T04:07:15Z", "closed_at": "2022-08-28T04:07:03Z", "author_association": "OWNER", "pull_request": null, "body": "https://github.com/simonw/sqlite-utils/compare/3.28...104f37fa4d2e7e5999c1d829267b62c737f74d3e", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/476/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1363765916, "node_id": "I_kwDOCGYnMM5RSWqc", "number": 483, "title": "`sqlite-utils install` command", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2022-09-06T20:13:55Z", "updated_at": "2022-09-26T19:04:43Z", "closed_at": "2022-09-26T18:57:15Z", "author_association": "OWNER", "pull_request": null, "body": "With the addition of `--functions` in:\r\n- #471\r\n\r\nIn addition to the existing `convert` command, there are now very good reasons to want to install additional packages into the same virtual environment as `sqlite-utils` itself, to allow them to be used with those features.\r\n\r\nThis isn't easy if you installed the tool with `pipx` or `brew install sqlite-utils`.\r\n\r\nDatasette solved this problem with the `datasette install` command:\r\n\r\n- https://github.com/simonw/datasette/issues/925\r\n\r\n`sqlite-utils` could benefit from the same idea.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/483/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1367835380, "node_id": "I_kwDOCGYnMM5Rh4L0", "number": 487, "title": "Specify foreign key against compound key in other table", "user": {"value": 540968, "label": "ryanfox"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2022-09-09T13:32:09Z", "updated_at": "2022-09-11T04:00:44Z", "closed_at": "2022-09-11T04:00:44Z", "author_association": "NONE", "pull_request": null, "body": "When inserting rows via the library, is it possible to specify a foreign key to a compound primary key?\r\n\r\nFor example, suppose I create a table:\r\n```\r\ndb = Database('events.db')\r\ndb['events'].insert_all([\r\n {'venue': 'Times Square', 'date': '2022-12-31', 'title': 'Rockin New Year Eve'},\r\n {'venue': 'Wembley Stadium', 'date': '2022-06-05', 'title': 'FA Cup'},\r\n {'venue': 'Times Square', 'date': '2021-12-31', 'title': 'Rockin New Year Eve'},\r\n], pk=('date', 'venue'))\r\n```\r\n\r\nAnd I want to add related data in another table:\r\n```\r\nact = {'name': 'Rick Astley', 'venue': 'Times Square', 'date': '2021-12-31' }\r\ndb['performers'].insert(act, pk=)\r\n```\r\n\r\nIs it possible to specify a value for `pk` that will point to the compound primary key in `events`?\r\n\r\nSQLite does support it:\r\nhttps://www.sqlite.org/foreignkeys.html#fk_composite", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/487/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1386593843, "node_id": "I_kwDOCGYnMM5Spb4z", "number": 494, "title": "Document how to use Just", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2022-09-26T19:25:12Z", "updated_at": "2022-09-26T19:32:36Z", "closed_at": "2022-09-26T19:26:39Z", "author_association": "OWNER", "pull_request": null, "body": "I'm using `just` a lot know, based on this file - I should add that to https://sqlite-utils.datasette.io/en/latest/contributing.html\r\n\r\nhttps://github.com/simonw/sqlite-utils/blob/afbd2b2cba45cccb305c3d4638d18db4dd3d4bbd/Justfile#L1-L24", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/494/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1423182778, "node_id": "I_kwDOCGYnMM5U1Au6", "number": 505, "title": "Release sqlite-utils 3.30", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2022-10-25T22:20:05Z", "updated_at": "2022-10-25T22:41:26Z", "closed_at": "2022-10-25T22:41:16Z", "author_association": "OWNER", "pull_request": null, "body": "https://github.com/simonw/sqlite-utils/compare/3.29...defa2974c6d3abc19be28d6b319649b8028dc966", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/505/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1436539554, "node_id": "I_kwDOCGYnMM5Vn9qi", "number": 511, "title": "[insert_all, upsert_all] IntegrityError: constraint failed", "user": {"value": 7908073, "label": "chapmanjacobd"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2022-11-04T19:21:48Z", "updated_at": "2022-11-04T22:59:54Z", "closed_at": "2022-11-04T22:54:09Z", "author_association": "CONTRIBUTOR", "pull_request": null, "body": "My understand is that `INSERT OR IGNORE` will ignore when inserts would cause duplicate keys so I'm not sure exactly why the error is raised from `sqlite3`.\r\n\r\n```\r\nimport argparse\r\nfrom pathlib import Path\r\n\r\nfrom xklb import db, utils\r\nfrom xklb.utils import log\r\n\r\n\r\ndef parse_args() -> argparse.Namespace:\r\n parser = argparse.ArgumentParser()\r\n parser.add_argument(\"database\")\r\n parser.add_argument(\"dbs\", nargs=\"*\")\r\n parser.add_argument(\"--upsert\")\r\n parser.add_argument(\"--db\", \"-db\", help=argparse.SUPPRESS)\r\n parser.add_argument(\"--verbose\", \"-v\", action=\"count\", default=0)\r\n args = parser.parse_args()\r\n\r\n if args.db:\r\n args.database = args.db\r\n Path(args.database).touch()\r\n args.db = db.connect(args)\r\n log.info(utils.dict_filter_bool(args.__dict__))\r\n\r\n return args\r\n\r\n\r\ndef merge_db(args, source_db):\r\n source_db = str(Path(source_db).resolve())\r\n\r\n s_db = db.connect(argparse.Namespace(database=source_db, verbose=args.verbose))\r\n for table in [s for s in s_db.table_names() if not \"_fts\" in s and not s.startswith(\"sqlite_\")]:\r\n log.info(\"[%s]: %s\", source_db, table)\r\n with s_db.conn:\r\n data = s_db[table].rows\r\n\r\n with args.db.conn:\r\n if args.upsert:\r\n args.db[table].upsert_all(data, pk=args.upsert.split(\",\"), alter=True)\r\n else:\r\n args.db[table].insert_all(data, alter=True, replace=True)\r\n\r\n\r\ndef merge_dbs():\r\n args = parse_args()\r\n for s_db in args.dbs:\r\n merge_db(args, s_db)\r\n\r\n\r\nif __name__ == \"__main__\":\r\n merge_dbs()\r\n\r\n```\r\n\r\n```\r\n$ lb-dev merge video.db tube_71.db --upsert path -vv\r\nSQL: INSERT OR IGNORE INTO [media]([path]) VALUES(?); - params: ['https://archive.org/details/088ghostofachanceroygetssackedrevengeofthelivinglunchdvdripxvidphz']\r\n...\r\nFile ~/.local/lib/python3.10/site-packages/sqlite_utils/db.py:3122, in Table.insert_all(self, records, pk, foreign_keys, column_order, not_null, defaults, batch_size, hash_id, hash_id_columns, alter, ignore, replace, truncate, extracts, conversions, columns, upsert, analyze)\r\n 3116 all_columns += [\r\n 3117 column for column in record if column not in all_columns\r\n 3118 ]\r\n 3120 first = False\r\n-> 3122 self.insert_chunk(\r\n 3123 alter,\r\n 3124 extracts,\r\n 3125 chunk,\r\n 3126 all_columns,\r\n 3127 hash_id,\r\n 3128 hash_id_columns,\r\n 3129 upsert,\r\n 3130 pk,\r\n 3131 conversions,\r\n 3132 num_records_processed,\r\n 3133 replace,\r\n 3134 ignore,\r\n 3135 )\r\n 3137 if analyze:\r\n 3138 self.analyze()\r\n\r\nFile ~/.local/lib/python3.10/site-packages/sqlite_utils/db.py:2887, in Table.insert_chunk(self, alter, extracts, chunk, all_columns, hash_id, hash_id_columns, upsert, pk, conversions, num_records_processed, replace, ignore)\r\n 2885 for query, params in queries_and_params:\r\n 2886 try:\r\n-> 2887 result = self.db.execute(query, params)\r\n 2888 except OperationalError as e:\r\n 2889 if alter and (\" column\" in e.args[0]):\r\n 2890 # Attempt to add any missing columns, then try again\r\n\r\nFile ~/.local/lib/python3.10/site-packages/sqlite_utils/db.py:484, in Database.execute(self, sql, parameters)\r\n 482 self._tracer(sql, parameters)\r\n 483 if parameters is not None:\r\n--> 484 return self.conn.execute(sql, parameters)\r\n 485 else:\r\n 486 return self.conn.execute(sql)\r\n\r\nIntegrityError: constraint failed\r\n> /home/xk/.local/lib/python3.10/site-packages/sqlite_utils/db.py(484)execute()\r\n 482 self._tracer(sql, parameters)\r\n 483 if parameters is not None:\r\n--> 484 return self.conn.execute(sql, parameters)\r\n 485 else:\r\n 486 return self.conn.execute(sql)\r\n```\r\n\r\n```\r\nsqlite3 --version\r\n3.36.0 2021-06-18 18:36:39\r\n```", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/511/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1595340692, "node_id": "I_kwDOCGYnMM5fFveU", "number": 530, "title": "add ability to configure \"on delete\" and \"on update\" attributes of foreign keys:", "user": {"value": 536941, "label": "fgregg"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2023-02-22T15:44:14Z", "updated_at": "2023-05-08T20:39:01Z", "closed_at": null, "author_association": "CONTRIBUTOR", "pull_request": null, "body": "sqlite supports these, and it would be quite nice to be able to add them with sqlite-utils.\r\n\r\nhttps://www.sqlite.org/foreignkeys.html#fk_actions", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/530/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1620254998, "node_id": "I_kwDOCGYnMM5gkyEW", "number": 532, "title": "Show more information when JSON can't be imported with sqlite-utils insert", "user": {"value": 83080728, "label": "voltagex"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2023-03-12T06:41:44Z", "updated_at": "2023-05-08T20:32:16Z", "closed_at": "2023-05-08T20:32:02Z", "author_association": "NONE", "pull_request": null, "body": "I am currently trying to import the [JSON export of my data from Discord](https://support.discord.com/hc/en-us/articles/360004027692-Requesting-a-Copy-of-your-Data), specifically `activity/reporting/events-*.json`\r\n\r\n```\r\nsqlite-utils.exe insert test.db reporting events-2023-00000-of-00001.json\r\n [###################################-] 99% 00:00:00\r\nError: Invalid JSON - use --csv for CSV or --tsv for TSV files\r\n```\r\n\r\nPlease show more information as to *why* this is invalid, if possible.\r\n\r\nI am using version 3.30 with Python 3.10 on Windows 11.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/532/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}