{"id": 702386948, "node_id": "MDU6SXNzdWU3MDIzODY5NDg=", "number": 159, "title": ".delete_where() does not auto-commit (unlike .insert() or .upsert())", "user": {"value": 11712349, "label": "spdkils"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 9, "created_at": "2020-09-16T01:55:52Z", "updated_at": "2023-04-01T17:21:05Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "When you use the delete_where() function on a table, it never commits....\r\n\r\nIs that intentional?", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/159/reactions\", \"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 722816436, "node_id": "MDU6SXNzdWU3MjI4MTY0MzY=", "number": 186, "title": ".extract() shouldn't extract null values", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 7, "created_at": "2020-10-16T02:41:08Z", "updated_at": "2021-08-12T12:32:14Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "This almost works, but it creates a rogue `type` record with a value of None.\r\n```\r\nIn [1]: import sqlite_utils\r\nIn [2]: db = sqlite_utils.Database(memory=True)\r\nIn [5]: db[\"creatures\"].insert_all([\r\n    {\"id\": 1, \"name\": \"Simon\", \"type\": None},\r\n    {\"id\": 2, \"name\": \"Natalie\", \"type\": None},\r\n    {\"id\": 3, \"name\": \"Cleo\", \"type\": \"dog\"}], pk=\"id\")\r\nOut[5]: <Table creatures (id, name, type)>\r\nIn [7]: db[\"creatures\"].extract(\"type\")\r\nOut[7]: <Table creatures (id, name, type_id)>\r\nIn [8]: list(db[\"creatures\"].rows)\r\nOut[8]: \r\n[{'id': 1, 'name': 'Simon', 'type_id': None},\r\n {'id': 2, 'name': 'Natalie', 'type_id': None},\r\n {'id': 3, 'name': 'Cleo', 'type_id': 2}]\r\nIn [9]: db[\"type\"]\r\nOut[9]: <Table type (id, type)>\r\nIn [10]: list(db[\"type\"].rows)\r\nOut[10]: [{'id': 1, 'type': None}, {'id': 2, 'type': 'dog'}]\r\n```", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/186/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 1856075668, "node_id": "I_kwDOCGYnMM5uoXeU", "number": 586, "title": ".transform() fails to drop column if table is part of a view", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2023-08-18T05:25:22Z", "updated_at": "2023-08-18T06:13:47Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "I got this error trying to drop a column from a table that was part of a SQL view:\r\n\r\n> error in view plugins: no such table: main.pypi_releases\r\n\r\nUpon further investigation I found that this pattern seemed to fix it:\r\n```python\r\ndef transform_the_table(conn):\r\n    # Run this in a transaction:\r\n    with conn:\r\n        # We have to read all the views first, because we need to drop and recreate them\r\n        db = sqlite_utils.Database(conn)\r\n        views = {v.name: v.schema for v in db.views if table.lower() in v.schema.lower()}\r\n        for view in views.keys():\r\n            db[view].drop()\r\n        db[table].transform(\r\n            types=types,\r\n            rename=rename,\r\n            drop=drop,\r\n            column_order=[p[0] for p in order_pairs],\r\n        )\r\n        # Now recreate the views\r\n        for name, schema in views.items():\r\n            db.create_view(name, schema)\r\n```\r\nSo grab a copy of any view that might reference this table, start a transaction, drop those views, run the transform, recreate the views again.\r\n\r\n> I wonder if this should become an option in `sqlite-utils`? Maybe a `recreate_views=True` argument for `table.tranform(...)`? Should it be opt-in or opt-out?\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/datasette-edit-schema/issues/35#issuecomment-1683370548_\r\n            ", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/586/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 778530523, "node_id": "MDU6SXNzdWU3Nzg1MzA1MjM=", "number": 1172, "title": "/-/static should be excluded from auth and permission checks", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2021-01-05T02:53:41Z", "updated_at": "2021-01-05T02:53:41Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "I want to set far future / immutable cache headers on everything served from `/-/static` and `/-/static-plugins`\r\n\r\nThis has security implications since it will be possible to see what plugins are installed by checking for known static URLs. I'm fine with that - performance is more important here.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1172/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 462117311, "node_id": "MDU6SXNzdWU0NjIxMTczMTE=", "number": 531, "title": "/database/-/inspect", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2019-06-28T16:33:41Z", "updated_at": "2019-07-08T15:43:57Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "Build `/database/-/inspect` which shows tables, columns, column types and foreign keys\r\n\r\nIt won't show table counts. Or maybe it will include them optionally but only for `-i` databases, in a special area of the JSON reserved for immutable-only inspect details.\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/datasette/issues/465#issuecomment-506797086_", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/531/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 1426080014, "node_id": "I_kwDOBm6k_c5VAEEO", "number": 1867, "title": "/db/table/-/rename API (also allows atomic replace)", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": {"value": 8755003, "label": "Datasette 1.0a-next"}, "comments": 1, "created_at": "2022-10-27T18:13:23Z", "updated_at": "2023-01-09T15:34:12Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "> There's one catch with batched inserts: if your CLI tool fails half way through you could end up with a partially populated table - since a bunch of batches will have succeeded first.\r\n>\r\n> ...\r\n>\r\n> If people care about that kind of thing they could always push all of their inserts to a table called `_tablename` and then atomically rename that once they've uploaded all of the data (assuming I provide an atomic-rename-this-table mechanism).\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1866#issuecomment-1293893789_\r\n      ", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1867/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 951817328, "node_id": "MDU6SXNzdWU5NTE4MTczMjg=", "number": 12, "title": "403 when getting token", "user": {"value": 285352, "label": "treyhunner"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2021-07-23T18:43:26Z", "updated_at": "2021-10-12T18:31:57Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "I tried to use https://your-foursquare-oauth-token.glitch.me/ to get my Swarm auth token and got a 403 after I clicked the Allow button:\r\n\r\n![image](https://user-images.githubusercontent.com/285352/126826478-60e53614-263d-40bb-9f1d-c1a676644eb0.png)\r\n\r\nI'm not sure if this is the right repo to report this in", "repo": {"value": 205429375, "label": "swarm-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/12/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 1663399821, "node_id": "I_kwDOBm6k_c5jJXeN", "number": 2058, "title": "500 \"attempt to write a readonly database\" error caused by \"PRAGMA schema_version\"", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 9, "created_at": "2023-04-11T23:57:50Z", "updated_at": "2023-04-13T16:35:21Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "I've not been able to replicate this myself yet, but I've seen log files from a user affected by it.\r\n\r\n```\r\nFile \"/usr/local/lib/python3.11/site-packages/datasette/views/base.py\", line 89, in dispatch_request\r\nawait self.ds.refresh_schemas()\r\nFile \"/usr/local/lib/python3.11/site-packages/datasette/app.py\", line 371, in refresh_schemas\r\nawait self._refresh_schemas()\r\nFile \"/usr/local/lib/python3.11/site-packages/datasette/app.py\", line 386, in _refresh_schemas\r\nschema_version = (await db.execute(\"PRAGMA schema_version\")).first()[0]\r\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\r\nFile \"/usr/local/lib/python3.11/site-packages/datasette/database.py\", line 267, in execute\r\nresults = await self.execute_fn(sql_operation_in_thread)\r\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\r\nFile \"/usr/local/lib/python3.11/site-packages/datasette/database.py\", line 213, in execute_fn\r\nreturn await asyncio.get_event_loop().run_in_executor(\r\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\r\nFile \"/usr/local/lib/python3.11/concurrent/futures/thread.py\", line 58, in run\r\nresult = self.fn(*self.args, **self.kwargs)\r\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\r\nFile \"/usr/local/lib/python3.11/site-packages/datasette/database.py\", line 211, in in_thread\r\nreturn fn(conn)\r\n^^^^^^^^\r\nFile \"/usr/local/lib/python3.11/site-packages/datasette/database.py\", line 237, in sql_operation_in_thread\r\ncursor.execute(sql, params if params is not None else {})\r\nsqlite3.OperationalError: attempt to write a readonly database\r\n```\r\nThat's running the official Datasette Docker image on https://fly.io/ - it's causing 500 errors on every page of their site.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/2058/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 463492815, "node_id": "MDU6SXNzdWU0NjM0OTI4MTU=", "number": 534, "title": "500 error on m2m facet detection", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2019-07-03T00:42:42Z", "updated_at": "2020-12-17T05:08:22Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "This may help debug:\r\n```\r\ndiff --git a/datasette/facets.py b/datasette/facets.py\r\nindex 76d73e5..07a4034 100644\r\n--- a/datasette/facets.py\r\n+++ b/datasette/facets.py\r\n@@ -499,11 +499,14 @@ class ManyToManyFacet(Facet):\r\n                 \"outgoing\"\r\n             ]\r\n             if len(other_table_outgoing_foreign_keys) == 2:\r\n-                destination_table = [\r\n-                    t\r\n-                    for t in other_table_outgoing_foreign_keys\r\n-                    if t[\"other_table\"] != self.table\r\n-                ][0][\"other_table\"]\r\n+                try:\r\n+                    destination_table = [\r\n+                        t\r\n+                        for t in other_table_outgoing_foreign_keys\r\n+                        if t[\"other_table\"] != self.table\r\n+                    ][0][\"other_table\"]\r\n+                except IndexError:\r\n+                    import pdb; pdb.pm()\r\n                 # Only suggest if it's not selected already\r\n                 if (\"_facet_m2m\", destination_table) in args:\r\n                     continue\r\n```", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/534/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 1493404423, "node_id": "I_kwDOBm6k_c5ZA4sH", "number": 1948, "title": "500 error on permission debug page when testing actors with _r", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2022-12-13T05:22:03Z", "updated_at": "2022-12-13T05:22:19Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "<img width=\"1272\" alt=\"image\" src=\"https://user-images.githubusercontent.com/9599/207233390-ecb1b81b-c9f4-40a3-8131-5537ab23e9fc.png\">\r\n\r\nThe 500 error is silent unless you are looking at the DevTools network pane.\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1948/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 812704869, "node_id": "MDU6SXNzdWU4MTI3MDQ4Njk=", "number": 1237, "title": "?_pretty=1 option for pretty-printing JSON output", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": {"value": 3268330, "label": "Datasette 1.0"}, "comments": 1, "created_at": "2021-02-20T20:54:40Z", "updated_at": "2021-11-16T18:28:33Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "Suggested by @frankieroberto in https://github.com/simonw/datasette/issues/782#issuecomment-782746755", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1237/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 426722204, "node_id": "MDU6SXNzdWU0MjY3MjIyMDQ=", "number": 423, "title": "?_search_col=X not reflected correctly in the UI", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2019-03-28T21:48:19Z", "updated_at": "2020-11-03T19:01:59Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "e.g. https://latest.datasette.io/fixtures/searchable?_search_text1=barry\r\n\r\n![2019-03-28 at 2 47 PM](https://user-images.githubusercontent.com/9599/55195035-84ebb800-5168-11e9-910b-fc9868bcd93e.png)\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/423/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 792890765, "node_id": "MDU6SXNzdWU3OTI4OTA3NjU=", "number": 1200, "title": "?_size=10 option for the arbitrary query page would be useful", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2021-01-24T20:55:35Z", "updated_at": "2021-02-11T03:13:59Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "https://latest.datasette.io/fixtures?sql=select+*+from+compound_three_primary_keys&_size=10 - `_size=10` does not do anything at the moment. It would be useful if it did.\r\n\r\nWould also be good if it persisted in a hidden form field.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1200/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 1237586379, "node_id": "I_kwDOBm6k_c5JxBHL", "number": 1742, "title": "?_trace=1 fails with datasette-geojson for some reason", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2022-05-16T19:06:05Z", "updated_at": "2022-05-16T19:42:13Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "view-source:https://calands.datasettes.com/calands/CPAD_2020a_SuperUnits.geojson?_sort=id&id__exact=4&_labels=on&_trace=1 is showing me a blank page.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1742/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 657572753, "node_id": "MDU6SXNzdWU2NTc1NzI3NTM=", "number": 894, "title": "?sort=colname~numeric to sort by  by column cast to real", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 21, "created_at": "2020-07-15T18:47:48Z", "updated_at": "2021-08-20T02:07:53Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "If a text column actually contains numbers, being able to \"sort by column, treated as numeric\" would be really useful.\r\n\r\nProbably depends on column actions enabled by #690", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/894/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 1345561209, "node_id": "I_kwDOBm6k_c5QM6J5", "number": 1790, "title": "A better HTML title for canned query pages", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2022-08-21T18:27:46Z", "updated_at": "2022-08-21T18:27:46Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "https://scotrail.datasette.io/scotrail/assemble_sentence?terms=This+train+is+formed+of%2Cbomb+which\r\n\r\nCurrent title is:\r\n\r\n`scotrail: with phrases as (  select    key, value  from    json_each(&#39;[&#34;&#39; || replace(:terms, &#39;,&#39;, &#39;&#34;,&#34;&#39;) || &#39;&#34;]&#39;)),matches as (select  phrases.key,  phrases.value,  (    select File from announcements    where announcements.Transcription like &#39;%&#39; || trim(phrases.value) || &#39;%&#39;    order by length(announcements.Transcription)    limit 1  ) as Filefrom  phrases),results as (  select key, announcements.Transcription, announcements.mp3  from announcements join matches on announcements.File = matches.File  order by key)select  &#39;Combined sentence:&#39; as mp3,  group_concat(Transcription, &#39; &#39;) as Transcription,  -1 as keyfrom results  unionselect  mp3, Transcription, keyfrom resultsorder by key`\r\n\r\nI think a better title would be:\r\n\r\n`scotrail: assemble_sentence, terms = This train is formed of,bomb which`", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1790/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 1071531082, "node_id": "I_kwDOCGYnMM4_3kRK", "number": 349, "title": "A way of creating indexes on newly created tables", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2021-12-05T18:56:12Z", "updated_at": "2021-12-07T01:04:37Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "I'm writing code for https://github.com/simonw/git-history/issues/33 that creates a table inside a loop:\r\n\r\n```python\r\nitem_pk = db[item_table].lookup(\r\n    {\"_item_id\": item_id},\r\n    item_to_insert,\r\n    column_order=(\"_id\", \"_item_id\"),\r\n    pk=\"_id\",\r\n)\r\n```\r\nI need to look things up by `_item_id` on this table, which means I need an index on that column (the table can get very big).\r\n\r\nBut there's no mechanism in SQLite utils to detect if the table was created for the first time and add an index to it. And I don't want to run `CREATE INDEX IF NOT EXISTS` every time through the loop.\r\n\r\nThis should work like the `foreign_keys=` mechanism.\r\n", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/349/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 787098345, "node_id": "MDU6SXNzdWU3ODcwOTgzNDU=", "number": 1191, "title": "Ability for plugins to collaborate when adding extra HTML to blocks in default templates", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": {"value": 3268330, "label": "Datasette 1.0"}, "comments": 12, "created_at": "2021-01-15T18:18:51Z", "updated_at": "2023-09-18T06:55:52Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "Sometimes a plugin may want to add content to an existing default template - for example `datasette-search-all` adds a new search box at the top of `index.html`. I also want `datasette-upload-csvs` to add a CTA on the `database.html` page: https://github.com/simonw/datasette-upload-csvs/issues/18\r\n\r\nCurrently plugins can do this by providing a new version of the `index.html` template - but if multiple plugins try to do that only one of them will succeed.\r\n\r\nIt would be better if there were known areas of those templates which plugins could add additional content to, such that multiple plugins can use the same spot.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1191/reactions\", \"total_count\": 4, \"+1\": 4, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 309047460, "node_id": "MDU6SXNzdWUzMDkwNDc0NjA=", "number": 188, "title": "Ability to bundle metadata and templates inside the SQLite file", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2018-03-27T16:42:07Z", "updated_at": "2020-12-04T17:18:34Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "One of the nicest qualities of SQLite as a data format is that you get a single file which you can then backup or share with other people. \r\n\r\nDatasette breaks this a little once you start including custom metadata.json or template files and CSS.\r\n\r\nIt would be cool if there was an optional mechanism for baking that extra configuration into the SQLite file itself. That way entire datasette mini-applications (including canned queries and custom HTML and CSS) could be constructed as single .db files.\r\n\r\nSince datasette configuration is all file-based, one way to achieve that would be to support a \"datasette_files\" table which, if present is used to search for file contents by path.\r\n\r\nThis is inline with the philosophy described by https://www.sqlite.org/appfileformat.html\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/188/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 275415799, "node_id": "MDU6SXNzdWUyNzU0MTU3OTk=", "number": 137, "title": "Ability to combine multiple SQL queries on a single graph", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2017-11-20T16:26:57Z", "updated_at": "2019-05-13T18:33:51Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "This would make visualizations significantly more powerful. The interesting challenge will be around the URL design. It would be useful to be able to combine either multiple explicit SQL queries or multiple queries based on the filter string parameters passed to one or more table views.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/137/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 626211658, "node_id": "MDU6SXNzdWU2MjYyMTE2NTg=", "number": 778, "title": "Ability to configure keyset pagination for views and queries", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2020-05-28T04:48:56Z", "updated_at": "2020-10-02T02:26:25Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "Currently views offer pagination, but it uses offset/limit - e.g. https://latest.datasette.io/fixtures/paginated_view?_next=100\r\n\r\nThis means pagination will perform poorly on deeper pages.\r\n\r\nIf a view is based on a table that has a primary key it should be possible to configure efficient keyset pagination that works the same way that table pagination works.\r\n\r\nThis may be as simple as configuring a column that can be treated as a \"primary key\" for the purpose of pagination using `metadata.json` - or with a `?_view_pk=colname` querystring argument.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/778/reactions\", \"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 1447465004, "node_id": "I_kwDOBm6k_c5WRpAs", "number": 1889, "title": "Ability to create new tokens via the API", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": {"value": 8755003, "label": "Datasette 1.0a-next"}, "comments": 0, "created_at": "2022-11-14T06:21:36Z", "updated_at": "2022-12-13T05:29:08Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "Refs:\r\n- #1850\r\n\r\nInitially I decided that the API shouldn't be able to create new tokens at all - I don't like the idea of an API token holder creating themselves additional tokens.\r\n\r\nThen I realized that two of the API features are specifically more useful if you can generate fresh tokens via the API:\r\n\r\n- Tokes that expire after a time limit are MUCH more useful if they can be automatically generated\r\n- Likewise, tokens that are restricted to a subset of permissions (see #1855) make more sense to be generated like this, especially in conjunction with expiry times", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1889/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 573578548, "node_id": "MDU6SXNzdWU1NzM1Nzg1NDg=", "number": 89, "title": "Ability to customize columns used by extracts= feature", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2020-03-01T16:54:48Z", "updated_at": "2020-10-16T19:17:50Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "@simonw any thoughts on allow extracts to specify the lookup column name? If I'm understanding the documentation right, `.lookup()` allows you to define the \"value\" column (the documentation uses name), but when you use `extracts` keyword as part of `.insert()`, `.upsert()` etc. the lookup must be done against a column named \"value\". I have an existing lookup table that I've populated with columns \"id\" and \"name\" as opposed to \"id\" and \"value\", and seems I can't use `extracts=`, unless I'm missing something...\r\n\r\nInitial thought on how to do this would be to allow the dictionary value to be a tuple of table name column pair... so:\r\n```\r\ntable = db.table(\"trees\", extracts={\"species_id\": (\"Species\", \"name\"})\r\n```\r\n\r\nI haven't dug too much into the existing code yet, but does this make sense? Worth doing?\r\n\r\n_Originally posted by @chrishas35 in https://github.com/simonw/sqlite-utils/issues/46#issuecomment-592999503_", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/89/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 1754174496, "node_id": "I_kwDOCGYnMM5ojpQg", "number": 558, "title": "Ability to define unique columns when creating a table", "user": {"value": 1910303, "label": "aguinane"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2023-06-13T06:56:19Z", "updated_at": "2023-08-18T01:06:03Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "When creating a new table, it would be good to have an option to set unique columns similar to how not_null is set.\r\n\r\n```python\r\nfrom sqlite_utils import Database\r\n\r\ncolumns = {\"mRID\": str, \"name\": str}\r\ndb = Database(\"example.db\")\r\ndb[\"ExampleTable\"].create(columns, pk=\"mRID\", not_null=[\"mRID\"], if_not_exists=True)\r\ndb[\"ExampleTable\"].create_index([\"mRID\"], unique=True, if_not_exists=True)\r\n```\r\n\r\nSo something like this would add the UNIQUE flag to the table definition. \r\n\r\n```python\r\ndb[\"ExampleTable\"].create(columns, pk=\"mRID\", not_null=[\"mRID\"], unique=[\"mRID\"], if_not_exists=True)\r\n```\r\n\r\n```sql\r\nCREATE TABLE ExampleTable (\r\n    mRID TEXT PRIMARY KEY\r\n              NOT NULL\r\n              UNIQUE,\r\n    name TEXT\r\n);\r\n```", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/558/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 456569067, "node_id": "MDU6SXNzdWU0NTY1NjkwNjc=", "number": 510, "title": "Ability to facet by delimiter (e.g. comma separated fields)", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": {"value": 9599, "label": "simonw"}, "milestone": null, "comments": 1, "created_at": "2019-06-15T19:34:41Z", "updated_at": "2019-07-08T15:44:51Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "E.g. if a field contains \"Tags,With,Commas\" be able to facet them in the same way as `_facet_array=` lets you facet `[\"Tags\", \"With\", \"Commas\"]`", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/510/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 897212458, "node_id": "MDU6SXNzdWU4OTcyMTI0NTg=", "number": 63, "title": "Ability to fetch commits from branches other than the default", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2021-05-20T17:58:08Z", "updated_at": "2021-05-20T17:58:08Z", "closed_at": null, "author_association": "MEMBER", "pull_request": null, "body": "This tool is currently almost entirely ignorant of the concept of branches. One example: you can't retrieve commits from any branch other than the default (usually main).", "repo": {"value": 207052882, "label": "github-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/github-to-sqlite/issues/63/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 1374939463, "node_id": "I_kwDOCGYnMM5R8-lH", "number": 489, "title": "Ability to load JSON records held in a file with a single top level key that is a list of objects", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 9, "created_at": "2022-09-15T18:46:03Z", "updated_at": "2022-09-15T20:56:10Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "It's very common for JSON to look like this:\r\n```json\r\n{\r\n  \"Version\": \"5.5.52.6\",\r\n  \"List\": [\r\n    {\r\n      \"Description\": \"Nonpartisan\",\r\n      \"Id\": 1,\r\n      \"ExternalId\": \"\"\r\n    },\r\n    {\r\n      \"Description\": \"Undeclared\",\r\n      \"Id\": 2,\r\n      \"ExternalId\": \"\"\r\n    }\r\n  ]\r\n}\r\n```\r\nThis example taken from the records downloaded from https://www.elections.alaska.gov/election-results/e/\r\n\r\nRight now you can't import this into `sqlite-utils` - you need to run it through `jq .List` first.\r\n\r\nBut since this is so common, it would be neat if `sqlite-utils` could have a rule of thumb that says \"if it's an object, but it has a single key that is is a list of objects, use that instead\".", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/489/reactions\", \"total_count\": 2, \"+1\": 2, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 1383646615, "node_id": "I_kwDOCGYnMM5SeMWX", "number": 491, "title": "Ability to merge databases and tables", "user": {"value": 8904453, "label": "sgraaf"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 7, "created_at": "2022-09-23T11:10:55Z", "updated_at": "2023-06-14T22:14:24Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "Hi! Let me firstly say that I am a big fan of your work -- I follow your tweets and blog posts with great interest \ud83d\ude04.\r\n\r\nNow onto the matter at hand: I think it would be great if `sqlite-utils` included a `merge` or `combine` command, with the purpose of combining different SQLite databases into a single SQLite database. This way, the newly \"merged\" database would contain all differently named tables contained in the databases to be merged as-is, as well a concatenation of all tables of the same name.\r\n\r\nThis could look something like this:\r\n\r\n```bash\r\nsqlite-utils merge cats.db dogs.db > animals.db\r\n```\r\n\r\nI imagine this is rather straightforward if all databases involved in the merge contain differently named tables (i.e. no chance of conflicts), but things get slightly more complicated if two or more of the databases to be merged contain tables with the same name. Not only do you have to \"do something\" with the primary key(s), but these tables could also simply have different schemas (and therefore be incompatible for concatenation to begin with).\r\n\r\nAnyhow, I would love your thoughts on this, and, if you are open to it, work together on the design and implementation!", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/491/reactions\", \"total_count\": 2, \"+1\": 2, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 842862708, "node_id": "MDU6SXNzdWU4NDI4NjI3MDg=", "number": 1280, "title": "Ability to run CI against multiple SQLite versions", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2021-03-28T23:54:50Z", "updated_at": "2021-05-10T19:07:46Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "Issue #1276 happened because I didn't run tests against a SQLite version prior to 3.16.0 (released 2017-01-02).\r\n\r\nGlitch is a deployment target and runs SQLite 3.11.0 from 2016-02-15.\r\n\r\nIf CI ran against that version of SQLite this bug could have been avoided.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1280/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 977323133, "node_id": "MDU6SXNzdWU5NzczMjMxMzM=", "number": 1445, "title": "Ability to search for text across all columns in a table", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 5, "created_at": "2021-08-23T18:50:48Z", "updated_at": "2021-08-23T19:10:17Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "When I'm working with new data I often find myself wanting to run a search for text embedded in ANY of the columns of a table, without having to even fully understand the schema first.\r\n\r\nI figured out a trick for doing that using a SQL-generated SQL query here: https://til.simonwillison.net/datasette/search-all-columns-trick\r\n\r\nBut maybe this should be a core Datasette feature? Or a plugin?", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1445/reactions\", \"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 1294641696, "node_id": "I_kwDOBm6k_c5NKqog", "number": 1767, "title": "Ability to set a custom favicon", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 9, "created_at": "2022-07-05T18:41:12Z", "updated_at": "2022-07-05T18:56:43Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "If you're running a website on Datasette, like https://www.niche-museums.com/ or https://til.simonwillison.net/ - you should have the ability to easily specify a custom favicon.\r\n\r\nCurrently the `/favicon.ico` view is hard-coded to do this: https://github.com/simonw/datasette/blob/9f1eb0d4eac483b953392157bd9fd6cc4df37de7/datasette/app.py#L179-L188", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1767/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 312395790, "node_id": "MDU6SXNzdWUzMTIzOTU3OTA=", "number": 197, "title": "Ability to sort by more than one column", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2018-04-09T05:13:30Z", "updated_at": "2018-07-10T17:45:37Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "Split off from #189.\r\n\r\nI'd like to support \"sort by X descending, then by Y ascending if there are dupes for X\" as well. Suggested syntax for that:\r\n\r\n    ?_sort_desc=X&_sort=Y\r\n\r\nwe currently only allow one argument to be sent. We should allow as many arguments as there are columns, for example:\r\n\r\n    ?_sort=department&_sort_desc=precinct&_sort=age&_sort_desc=size", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/197/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 312396095, "node_id": "MDU6SXNzdWUzMTIzOTYwOTU=", "number": 198, "title": "Ability to sort with nulls last", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2018-04-09T05:15:40Z", "updated_at": "2018-07-10T17:45:37Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "Split off from #189\r\n\r\nHere's how to do that in SQL: https://fivethirtyeight.datasettes.com/fivethirtyeight-2628db9?sql=select+rowid%2C+*+from+%5Bnfl-wide-receivers%2Fadvanced-historical%5D%0D%0Aorder+by+case+when+career_ranypa+is+null+then+1+else+0+end%2C+career_ranypa%2C+rowid\r\n\r\n    order by case when career_ranypa is null then 1 else 0 end, career_ranypa", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/198/reactions\", \"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 780153562, "node_id": "MDU6SXNzdWU3ODAxNTM1NjI=", "number": 1177, "title": "Ability to stream all rows as newline-delimited JSON", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": {"value": 3268330, "label": "Datasette 1.0"}, "comments": 1, "created_at": "2021-01-06T07:10:48Z", "updated_at": "2022-03-21T15:08:52Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "> Yet another use-case for this: I want to be able to stream newline-delimited JSON in order to better import into Pandas:\r\n> \r\n>     pandas.read_json(\"https://latest.datasette.io/fixtures/compound_three_primary_keys.json?_shape=array&_nl=on\", lines=True)\r\n_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1101#issuecomment-755128038_", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1177/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 1399933513, "node_id": "I_kwDOBm6k_c5TcUpJ", "number": 1833, "title": "Ability to submit long queries by POST", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2022-10-06T16:03:26Z", "updated_at": "2022-10-06T16:18:00Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "Datasette doesn't limit URL lengths but some common web proxies do - the one in front of Google Cloud Run for example limits to 8KB total for incoming request headers: https://cloud.google.com/load-balancing/docs/quotas#https-lb-header-limits\r\n\r\nThis means longer SQL queries can break!\r\n\r\nNeed an optional mechanism for submitting queries by POST instead.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1833/reactions\", \"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 1879214365, "node_id": "I_kwDOCGYnMM5wAokd", "number": 590, "title": "Ability to tell if a Database is an in-memory one", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2023-09-03T19:50:15Z", "updated_at": "2023-09-03T19:50:36Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "Currently the constructor accepts `memory=True` or `memory_name=...` and uses those to create a connection, but does not record what those values were:\r\n\r\nhttps://github.com/simonw/sqlite-utils/blob/1260bdc7bfe31c36c272572c6389125f8de6ef71/sqlite_utils/db.py#L307-L349\r\n\r\nThis makes it hard to tell if a database object is to an in-memory or a file-based database, which is sometimes useful to know.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/590/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 791237799, "node_id": "MDU6SXNzdWU3OTEyMzc3OTk=", "number": 1196, "title": "Access Denied Error in Windows", "user": {"value": 2826376, "label": "QAInsights"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2021-01-21T15:40:40Z", "updated_at": "2021-04-14T19:28:38Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "I am trying to publish a db to vercel. But while issuing the below command throwing `Access Denied` error which is leading to `RecursionError: maximum recursion depth exceeded while calling a Python object`.\r\n\r\nI am using PyCharm and Python 3.9. I have reinstalled both and launched PyCharm as Admin in Windows 10. But still the issue persists.\r\n\r\nIssued command `datasette publish vercel jmeter.db --project jmeter --install datasette-vega`\r\n\r\nPS: localhost is working fine.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1196/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 451585764, "node_id": "MDU6SXNzdWU0NTE1ODU3NjQ=", "number": 499, "title": "Accessibility for non-techie newsies? ", "user": {"value": 7936571, "label": "chrismp"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2019-06-03T16:49:37Z", "updated_at": "2019-06-05T21:22:55Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "Hi again, I'm having fun uploading datasets to Heroku via datasette. I'd like to set up datasette so that it's easy for other newsroom workers, who don't use Linux and aren't programmers, to upload datasets. Does datsette provide this out-of-the-box, or as a plugin? ", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/499/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 751195017, "node_id": "MDU6SXNzdWU3NTExOTUwMTc=", "number": 1111, "title": "Accessing a database's `.json` is slow for very large SQLite files", "user": {"value": 15178711, "label": "asg017"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2020-11-26T00:27:27Z", "updated_at": "2021-01-04T19:57:53Z", "closed_at": null, "author_association": "CONTRIBUTOR", "pull_request": null, "body": "I have a SQLite DB that's pretty large, 23GB and something like 300 million rows. I expect that most queries I run on it will be slow, which is fine, but there are some things that Datasette does that makes working with the DB very slow. Specifically, when I access the `.json` metadata for a table (which I believe it comes from `datasette/views/database.py`, it takes 43 seconds for the request to come in:\r\n\r\n```bash\r\n$ time curl localhost:9999/out.json\r\n{\"database\": \"out\", \"size\": 24291454976, \"tables\": [{\"name\": \"PageviewsHour\", \"columns\": [\"file\", \"code\", \"page\", \"pageviews\"], \"primary_keys\": [], \"count\": null, \"hidden\": false, \"fts_table\": null, \"foreign_keys\": {\"incoming\": [], \"outgoing\": [{\"other_table\": \"PageviewsHourFiles\", \"column\": \"file\", \"other_column\": \"file_id\"}]}, \"private\": false}, {\"name\": \"PageviewsHourFiles\", \"columns\": [\"file_id\", \"filename\", \"sha256\", \"size\", \"day\", \"hour\"], \"primary_keys\": [\"file_id\"], \"count\": null, \"hidden\": false, \"fts_table\": null, \"foreign_keys\": {\"incoming\": [{\"other_table\": \"PageviewsHour\", \"column\": \"file_id\", \"other_column\": \"file\"}], \"outgoing\": []}, \"private\": false}, {\"name\": \"sqlite_sequence\", \"columns\": [\"name\", \"seq\"], \"primary_keys\": [], \"count\": 1, \"hidden\": false, \"fts_table\": null, \"foreign_keys\": {\"incoming\": [], \"outgoing\": []}, \"private\": false}], \"hidden_count\": 0, \"views\": [], \"queries\": [], \"private\": false, \"allow_execute_sql\": true, \"query_ms\": 43340.23213386536}\r\nreal\t0m43.417s\r\nuser\t0m0.006s\r\nsys\t0m0.016s\r\n```\r\n\r\nI suspect this is because a `COUNT(*)` is happening under the hood, which, when I run it through sqlite directly, does take around the same time:\r\n\r\n```bash\r\n$ time sqlite3 out.db < <(echo \"select count(*) from PageviewsHour;\")\r\n362794272\r\n\r\nreal\t0m44.523s\r\nuser\t0m2.497s\r\nsys\t0m6.703s\r\n```\r\n\r\nI'm using the `.json` request in the [Observable Datasette Client](https://observablehq.com/@asg017/datasette-client) to 1) verify that a link passed in is a reachable Datasette instance, and 2) a quick way to look at metadata for a db. A few different solutions I can think of:\r\n\r\n1. Have some other endpoint, like `/-/datasette.json` that the Observable Datasette client can fetch from to verify that the passed in URL is a valid Datasette (doesnt solve the slow problem, feel free to split this issue into 2)\r\n2. Have a way to turn off table counts when accessing a database's `.json` view, like `?no_count=1` or something\r\n3. Maybe have a timeout on the `table_counts()` function if it takes too long. which is odd, because it seems like it already does that (I think?), I can debug a little more if that's the case\r\n\r\nMore than happy to debug further, or send a PR if you like one of the proposals above!", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1111/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 1674322631, "node_id": "PR_kwDOBm6k_c5OpEz_", "number": 2061, "title": "Add \"Packaging a plugin using Poetry\" section in docs", "user": {"value": 1238873, "label": "rclement"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2023-04-19T07:23:28Z", "updated_at": "2023-04-19T07:27:18Z", "closed_at": null, "author_association": "FIRST_TIME_CONTRIBUTOR", "pull_request": "simonw/datasette/pulls/2061", "body": "This PR adds a new section about packaging a plugin using `poetry` within the \"Writing plugins\" page of the documentation.\r\n\r\n<!-- readthedocs-preview datasette start -->\r\n----\n:books: Documentation preview :books:: https://datasette--2061.org.readthedocs.build/en/2061/\n\r\n<!-- readthedocs-preview datasette end -->", "repo": {"value": 107914493, "label": "datasette"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/2061/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 975161924, "node_id": "MDExOlB1bGxSZXF1ZXN0NzE2MzU3OTgy", "number": 66, "title": "Add --merged-by flag to pull-requests sub command", "user": {"value": 30531572, "label": "sarcasticadmin"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2021-08-20T00:57:55Z", "updated_at": "2021-09-28T21:50:31Z", "closed_at": null, "author_association": "FIRST_TIME_CONTRIBUTOR", "pull_request": "dogsheep/github-to-sqlite/pulls/66", "body": "## Description\r\n\r\nProposing a solution to the API limitation for `merged_by` in pull_requests. Specifically the following called out in the readme:\r\n\r\n```\r\nNote that the merged_by column on the pull_requests table will only be populated for pull requests that are loaded using the --pull-request option - the GitHub API does not return this field for pull requests that are loaded in bulk.\r\n```\r\n\r\nThis approach might cause larger repos to hit rate limits called out in https://github.com/dogsheep/github-to-sqlite/issues/51 but seems to work well in the repos I tested and included below.\r\n\r\n## Old Behavior\r\n- Had to list out the pull-requests individually via multiple `--pull-request` flags\r\n\r\n## New Behavior\r\n\r\n- `--merged-by` flag for getting 'merge_by' information out of pull-requests without having to specify individual PR numbers.\r\n\r\n# Testing\r\n\r\nPicking some repo that has more than one merger (datasette only has 1 \ud83d\ude09 )\r\n\r\n```\r\n$ github-to-sqlite pull-requests ./github.db opnsense/tools --merged-by\r\n$ echo \"select id, url, merged_by from pull_requests;\" | sqlite3 ./github.db \r\n83533612|https://github.com/opnsense/tools/pull/39|1915288\r\n102632885|https://github.com/opnsense/tools/pull/43|1915288\r\n149114810|https://github.com/opnsense/tools/pull/57|1915288\r\n160394495|https://github.com/opnsense/tools/pull/64|1915288\r\n163308408|https://github.com/opnsense/tools/pull/67|1915288\r\n169723264|https://github.com/opnsense/tools/pull/69|1915288\r\n171381422|https://github.com/opnsense/tools/pull/72|1915288\r\n179938195|https://github.com/opnsense/tools/pull/77|1915288\r\n196233824|https://github.com/opnsense/tools/pull/82|1915288\r\n215289964|https://github.com/opnsense/tools/pull/93|\r\n219696100|https://github.com/opnsense/tools/pull/97|1915288\r\n223664843|https://github.com/opnsense/tools/pull/99|\r\n228446172|https://github.com/opnsense/tools/pull/103|1915288\r\n238930434|https://github.com/opnsense/tools/pull/110|1915288\r\n255507110|https://github.com/opnsense/tools/pull/119|1915288\r\n255980675|https://github.com/opnsense/tools/pull/120|1915288\r\n261906770|https://github.com/opnsense/tools/pull/125|\r\n263800503|https://github.com/opnsense/tools/pull/127|1915288\r\n264038685|https://github.com/opnsense/tools/pull/128|1915288\r\n264696704|https://github.com/opnsense/tools/pull/129|1915288\r\n266660547|https://github.com/opnsense/tools/pull/130|1915288\r\n273120409|https://github.com/opnsense/tools/pull/133|1915288\r\n274370803|https://github.com/opnsense/tools/pull/135|\r\n276600629|https://github.com/opnsense/tools/pull/139|\r\n277303655|https://github.com/opnsense/tools/pull/141|1915288\r\n293033714|https://github.com/opnsense/tools/pull/145|\r\n294827649|https://github.com/opnsense/tools/pull/146|\r\n295140008|https://github.com/opnsense/tools/pull/147|1915288\r\n305690829|https://github.com/opnsense/tools/pull/150|9783985\r\n307077931|https://github.com/opnsense/tools/pull/152|1915288\r\n321782100|https://github.com/opnsense/tools/pull/155|\r\n337265672|https://github.com/opnsense/tools/pull/160|\r\n337267484|https://github.com/opnsense/tools/pull/161|1915288\r\n368251763|https://github.com/opnsense/tools/pull/169|\r\n428262505|https://github.com/opnsense/tools/pull/181|\r\n437557011|https://github.com/opnsense/tools/pull/182|1915288\r\n447079893|https://github.com/opnsense/tools/pull/185|\r\n461822092|https://github.com/opnsense/tools/pull/191|\r\n463290142|https://github.com/opnsense/tools/pull/193|1915288\r\n470112962|https://github.com/opnsense/tools/pull/194|1915288\r\n472644649|https://github.com/opnsense/tools/pull/195|1915288\r\n488696898|https://github.com/opnsense/tools/pull/198|\r\n513289902|https://github.com/opnsense/tools/pull/201|\r\n522530265|https://github.com/opnsense/tools/pull/203|\r\n564443347|https://github.com/opnsense/tools/pull/213|\r\n597579516|https://github.com/opnsense/tools/pull/220|1915288\r\n602860357|https://github.com/opnsense/tools/pull/221|1915288\r\n608744738|https://github.com/opnsense/tools/pull/222|1915288\r\n623279673|https://github.com/opnsense/tools/pull/228|1915288\r\n664656182|https://github.com/opnsense/tools/pull/233|\r\n664781786|https://github.com/opnsense/tools/pull/234|1915288\r\n670683636|https://github.com/opnsense/tools/pull/235|1915288\r\n683150764|https://github.com/opnsense/tools/pull/237|\r\n685016233|https://github.com/opnsense/tools/pull/238|\r\n687099825|https://github.com/opnsense/tools/pull/239|1915288\r\n715705652|https://github.com/opnsense/tools/pull/244|1915288\r\n715721248|https://github.com/opnsense/tools/pull/245|1915288\r\n```\r\n`userid` are now present for those PRs that were merged.\r\n\r\nWithout the flag the `merged_by` behavior remains missing as expected when get PRs bulk:\r\n\r\n```\r\n$ github-to-sqlite pull-requests ./github.db opnsense/tools\r\n$ echo \"select id, url, merged_by from pull_requests;\" | sqlite3 ./github.db \r\n83533612|https://github.com/opnsense/tools/pull/39|\r\n102632885|https://github.com/opnsense/tools/pull/43|\r\n149114810|https://github.com/opnsense/tools/pull/57|\r\n160394495|https://github.com/opnsense/tools/pull/64|\r\n163308408|https://github.com/opnsense/tools/pull/67|\r\n169723264|https://github.com/opnsense/tools/pull/69|\r\n171381422|https://github.com/opnsense/tools/pull/72|\r\n179938195|https://github.com/opnsense/tools/pull/77|\r\n196233824|https://github.com/opnsense/tools/pull/82|\r\n215289964|https://github.com/opnsense/tools/pull/93|\r\n219696100|https://github.com/opnsense/tools/pull/97|\r\n223664843|https://github.com/opnsense/tools/pull/99|\r\n228446172|https://github.com/opnsense/tools/pull/103|\r\n238930434|https://github.com/opnsense/tools/pull/110|\r\n255507110|https://github.com/opnsense/tools/pull/119|\r\n255980675|https://github.com/opnsense/tools/pull/120|\r\n261906770|https://github.com/opnsense/tools/pull/125|\r\n263800503|https://github.com/opnsense/tools/pull/127|\r\n264038685|https://github.com/opnsense/tools/pull/128|\r\n264696704|https://github.com/opnsense/tools/pull/129|\r\n266660547|https://github.com/opnsense/tools/pull/130|\r\n273120409|https://github.com/opnsense/tools/pull/133|\r\n274370803|https://github.com/opnsense/tools/pull/135|\r\n276600629|https://github.com/opnsense/tools/pull/139|\r\n277303655|https://github.com/opnsense/tools/pull/141|\r\n293033714|https://github.com/opnsense/tools/pull/145|\r\n294827649|https://github.com/opnsense/tools/pull/146|\r\n295140008|https://github.com/opnsense/tools/pull/147|\r\n305690829|https://github.com/opnsense/tools/pull/150|\r\n307077931|https://github.com/opnsense/tools/pull/152|\r\n321782100|https://github.com/opnsense/tools/pull/155|\r\n337265672|https://github.com/opnsense/tools/pull/160|\r\n337267484|https://github.com/opnsense/tools/pull/161|\r\n368251763|https://github.com/opnsense/tools/pull/169|\r\n428262505|https://github.com/opnsense/tools/pull/181|\r\n437557011|https://github.com/opnsense/tools/pull/182|\r\n447079893|https://github.com/opnsense/tools/pull/185|\r\n461822092|https://github.com/opnsense/tools/pull/191|\r\n463290142|https://github.com/opnsense/tools/pull/193|\r\n470112962|https://github.com/opnsense/tools/pull/194|\r\n472644649|https://github.com/opnsense/tools/pull/195|\r\n488696898|https://github.com/opnsense/tools/pull/198|\r\n513289902|https://github.com/opnsense/tools/pull/201|\r\n522530265|https://github.com/opnsense/tools/pull/203|\r\n564443347|https://github.com/opnsense/tools/pull/213|\r\n597579516|https://github.com/opnsense/tools/pull/220|\r\n602860357|https://github.com/opnsense/tools/pull/221|\r\n608744738|https://github.com/opnsense/tools/pull/222|\r\n623279673|https://github.com/opnsense/tools/pull/228|\r\n664656182|https://github.com/opnsense/tools/pull/233|\r\n664781786|https://github.com/opnsense/tools/pull/234|\r\n670683636|https://github.com/opnsense/tools/pull/235|\r\n683150764|https://github.com/opnsense/tools/pull/237|\r\n685016233|https://github.com/opnsense/tools/pull/238|\r\n687099825|https://github.com/opnsense/tools/pull/239|\r\n715705652|https://github.com/opnsense/tools/pull/244|\r\n715721248|https://github.com/opnsense/tools/pull/245|\r\n```\r\n\r\nIndividual PRs passed via `--pull-request` flag behaves as expected (unchanged):\r\n\r\n```\r\n$ github-to-sqlite pull-requests ./github.db opnsense/tools --pull-request 39 --pull-request 237\r\n$ echo \"select id, url, merged_by from pull_requests;\" | sqlite3 ./github.db\r\n83533612|https://github.com/opnsense/tools/pull/39|1915288\r\n683150764|https://github.com/opnsense/tools/pull/237|\r\n```\r\n> Picking 1 PR that has a merged_by (39) and one that does not (237)", "repo": {"value": 207052882, "label": "github-to-sqlite"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/github-to-sqlite/issues/66/reactions\", \"total_count\": 3, \"+1\": 2, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 1, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 465019882, "node_id": "MDU6SXNzdWU0NjUwMTk4ODI=", "number": 552, "title": "Add --plugin-secret support to \"datasette package\"", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2019-07-08T01:46:47Z", "updated_at": "2019-07-08T01:47:30Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "Split out from #544.\r\n\r\nI think I should combine this with #347 (renaming `datasette package` to `datasette publish docker`).", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/552/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 323658641, "node_id": "MDU6SXNzdWUzMjM2NTg2NDE=", "number": 262, "title": "Add ?_extra= mechanism for requesting extra properties in JSON", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": {"value": 3268330, "label": "Datasette 1.0"}, "comments": 27, "created_at": "2018-05-16T14:55:42Z", "updated_at": "2023-03-29T06:22:22Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "Datasette views currently work by creating a set of data that should be returned as JSON, then defining an additional, optional `template_data()` function which is called if the view is being rendered as HTML.\r\n\r\nThis `template_data()` function calculates extra template context variables which are necessary for the HTML view but should not be included in the JSON.\r\n\r\nExample of how that is used today: https://github.com/simonw/datasette/blob/2b79f2bdeb1efa86e0756e741292d625f91cb93d/datasette/views/table.py#L672-L704\r\n\r\nWith features like Facets in #255 I'm beginning to want to move more items into the `template_data()` - in the case of facets it's the `suggested_facets` array. This saves that feature from being calculated (involving several SQL queries) for the JSON case where it is unlikely to be used.\r\n\r\nBut... as an API user, I want to still optionally be able to access that information.\r\n\r\nSolution: Add a `?_extra=suggested_facets&_extra=table_metadata` argument which can be used to optionally request additional blocks to be added to the JSON API.\r\n\r\nThen redefine as many of the current `template_data()` features as extra arguments instead, and teach Datasette to return certain extras by default when rendering templates.\r\n\r\nThis could allow the JSON representation to be slimmed down further (removing e.g. the `table_definition` and `view_definition` keys) while still making that information available to API users who need it.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/262/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 954546309, "node_id": "MDExOlB1bGxSZXF1ZXN0Njk4NDIzNjY3", "number": 8, "title": "Add Gmail takeout mbox import (v2)", "user": {"value": 28565, "label": "maxhawkins"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 7, "created_at": "2021-07-28T07:05:32Z", "updated_at": "2023-09-08T01:22:49Z", "closed_at": null, "author_association": "FIRST_TIME_CONTRIBUTOR", "pull_request": "dogsheep/google-takeout-to-sqlite/pulls/8", "body": "WIP\r\n\r\nThis PR builds on #5 to continue implementing gmail import support.\r\n\r\nBuilding on @UtahDave's work, these commits add a few performance and bug fixes:\r\n\r\n* Decreased memory overhead for import by manually parsing mbox headers.\r\n* Fixed error where some messages in the mbox would yield a row with NULL in all columns.\r\n\r\nI will send more commits to fix any errors I encounter as I run the importer on my personal takeout data.", "repo": {"value": 206649770, "label": "google-takeout-to-sqlite"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/8/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 285168503, "node_id": "MDU6SXNzdWUyODUxNjg1MDM=", "number": 176, "title": "Add GraphQL endpoint", "user": {"value": 173848, "label": "yozlet"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 8, "created_at": "2017-12-29T23:21:01Z", "updated_at": "2020-04-21T14:16:24Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "Would make it much easier to build React & similar frontends. Maybe with https://github.com/graphql-python/sanic-graphql ?", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/176/reactions\", \"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 1823428714, "node_id": "I_kwDOBm6k_c5sr1Bq", "number": 2120, "title": "Add __all__ to datasette/__init__.py", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2023-07-27T01:07:10Z", "updated_at": "2023-07-27T01:07:10Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "Currently looks like this: https://github.com/simonw/datasette/blob/08181823990a71ffa5a1b57b37259198eaa43e06/datasette/__init__.py#L1-L6\r\n\r\nAdding `__all__ = [\"Permission\", \"Forbidden\"...]` would let me get rid of those `# noqa` comments.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/2120/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 694136490, "node_id": "MDU6SXNzdWU2OTQxMzY0OTA=", "number": 15, "title": "Add a bunch of config examples", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2020-09-05T17:58:43Z", "updated_at": "2020-09-18T23:17:39Z", "closed_at": null, "author_association": "MEMBER", "pull_request": null, "body": "I can bring these over from my personal Dogsheep.", "repo": {"value": 197431109, "label": "dogsheep-beta"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/dogsheep-beta/issues/15/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 830901133, "node_id": "MDExOlB1bGxSZXF1ZXN0NTkyMzY0MjU1", "number": 16, "title": "Add a fallback ID, print if no ID found", "user": {"value": 1234956, "label": "n8henrie"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2021-03-13T13:38:29Z", "updated_at": "2021-03-13T14:44:04Z", "closed_at": null, "author_association": "FIRST_TIME_CONTRIBUTOR", "pull_request": "dogsheep/healthkit-to-sqlite/pulls/16", "body": "Fixes https://github.com/dogsheep/healthkit-to-sqlite/issues/14\n", "repo": {"value": 197882382, "label": "healthkit-to-sqlite"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/16/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 534629631, "node_id": "MDU6SXNzdWU1MzQ2Mjk2MzE=", "number": 650, "title": "Add a glossary to the documentation", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2019-12-09T00:23:45Z", "updated_at": "2022-01-13T22:04:56Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "Call it `glossary.rst` - it can use a definition list something like this:\r\n```rst\r\n.. _glossary:\r\n\r\nGlossary\r\n========\r\n\r\nTerm\r\n  A definition of the term.\r\n\r\nAnother term\r\n  Another definition.\r\n```", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/650/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 961008507, "node_id": "MDU6SXNzdWU5NjEwMDg1MDc=", "number": 308, "title": "Add an interactive tutorial as a Jupyter notebook", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2021-08-04T20:34:22Z", "updated_at": "2021-08-04T21:30:59Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "Can show people how to open this up in Binder.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/308/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 1353074021, "node_id": "I_kwDOCGYnMM5QpkVl", "number": 474, "title": "Add an option for specifying column names when inserting CSV data", "user": {"value": 14294, "label": "hubgit"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2022-08-27T15:29:59Z", "updated_at": "2022-08-31T03:42:36Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "https://sqlite-utils.datasette.io/en/stable/cli.html#csv-files-without-a-header-row\r\n\r\n> The first row of any CSV or TSV file is expected to contain the names of the columns in that file.\r\n\r\n> If your file does not include this row, you can use the `--no-headers` option to specify that the tool should not use that fist row as headers.\r\n\r\n> If you do this, the table will be created with column names called `untitled_1` and `untitled_2` and so on. You can then rename them using the `sqlite-utils transform ... --rename` command.\r\n\r\nIt would be nice to be able to specify the column names when importing CSV/TSV without a header row, via an extra command line option.\r\n\r\n(renaming a column of a large table can take a long time, which makes it an inconvenient workaround)", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/474/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 1102966378, "node_id": "I_kwDOBm6k_c5Bve5q", "number": 1599, "title": "Add architecture documentation", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2022-01-14T04:55:38Z", "updated_at": "2022-01-14T04:56:03Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "Inspired by https://matklad.github.io/2021/02/06/ARCHITECTURE.md.html\r\n\r\nGood example: https://github.com/rust-analyzer/rust-analyzer/blob/d7c99931d05e3723d878bea5dc26766791fa4e69/docs/dev/architecture.md", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1599/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 1581218043, "node_id": "PR_kwDOBm6k_c5JyqPy", "number": 2025, "title": "Add database metadata to index.html template context", "user": {"value": 9993, "label": "palewire"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2023-02-12T11:16:58Z", "updated_at": "2023-02-12T11:17:14Z", "closed_at": null, "author_association": "FIRST_TIME_CONTRIBUTOR", "pull_request": "simonw/datasette/pulls/2025", "body": "Fixes #2016 \r\n\r\n<!-- readthedocs-preview datasette start -->\r\n----\n:books: Documentation preview :books:: https://datasette--2025.org.readthedocs.build/en/2025/\n\r\n<!-- readthedocs-preview datasette end -->", "repo": {"value": 107914493, "label": "datasette"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/2025/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 476852861, "node_id": "MDU6SXNzdWU0NzY4NTI4NjE=", "number": 568, "title": "Add database_color as a configurable option", "user": {"value": 50906992, "label": "LBHELewis"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2019-08-05T13:14:45Z", "updated_at": "2023-08-11T05:19:42Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "This would be really useful as it would allow us to tie in with colour schemes.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/568/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 836064851, "node_id": "MDExOlB1bGxSZXF1ZXN0NTk2NjI3Nzgw", "number": 18, "title": "Add datetime parsing", "user": {"value": 1234956, "label": "n8henrie"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2021-03-19T14:34:22Z", "updated_at": "2021-03-19T14:34:22Z", "closed_at": null, "author_association": "FIRST_TIME_CONTRIBUTOR", "pull_request": "dogsheep/healthkit-to-sqlite/pulls/18", "body": "Parses the datetime columns so they are subsequently properly recognized as\ndatetime.\n\nFixes https://github.com/dogsheep/healthkit-to-sqlite/issues/17\n", "repo": {"value": 197882382, "label": "healthkit-to-sqlite"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/18/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 1396977994, "node_id": "I_kwDOBm6k_c5TRDFK", "number": 1830, "title": "Add documentation for writing tests with signed actor cookies", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2022-10-04T23:51:26Z", "updated_at": "2022-10-04T23:51:26Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "I use this pattirn in a lot of plugin tests, e.g. https://github.com/simonw/datasette-edit-templates/blob/087f6a6cabc20020f2b0524f11aa3a7836320848/tests/test_edit_templates.py#L55-L58\r\n```python\r\n    actor = ds.sign({\"a\": {\"id\": \"root\"}}, \"actor\")\r\n    response1 = await ds.client.get(\r\n        \"/-/edit-templates/_footer.html\", cookies={\"ds_actor\": actor}\r\n    )\r\n```\r\nI should add this to the documentation on this page: https://docs.datasette.io/en/latest/testing_plugins.html", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1830/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 723499985, "node_id": "MDExOlB1bGxSZXF1ZXN0NTA1MDc2NDE4", "number": 5, "title": "Add fitbit-to-sqlite", "user": {"value": 4632208, "label": "mrphil007"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2020-10-16T20:04:05Z", "updated_at": "2020-10-16T20:04:05Z", "closed_at": null, "author_association": "FIRST_TIME_CONTRIBUTOR", "pull_request": "dogsheep/dogsheep.github.io/pulls/5", "body": "", "repo": {"value": 214746582, "label": "dogsheep.github.io"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/dogsheep.github.io/issues/5/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 975166271, "node_id": "MDU6SXNzdWU5NzUxNjYyNzE=", "number": 20, "title": "Add index on workout_points.date", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2021-08-20T01:08:04Z", "updated_at": "2021-08-20T01:12:48Z", "closed_at": null, "author_association": "MEMBER", "pull_request": null, "body": "Sorting that by date makes sense for seeing most recent points, and my DB has 2.5m points in so it's an expensive sort!", "repo": {"value": 197882382, "label": "healthkit-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/20/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 925384329, "node_id": "MDExOlB1bGxSZXF1ZXN0NjczODcyOTc0", "number": 7, "title": "Add instagram-to-sqlite", "user": {"value": 36654812, "label": "gavindsouza"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2021-06-19T12:26:16Z", "updated_at": "2021-07-28T07:58:59Z", "closed_at": null, "author_association": "FIRST_TIME_CONTRIBUTOR", "pull_request": "dogsheep/dogsheep.github.io/pulls/7", "body": "The tool covers only chat imports at the time of opening this PR but I'm planning to import everything else that I feel inquisitive about\r\n\r\nref: https://github.com/gavindsouza/instagram-to-sqlite", "repo": {"value": 214746582, "label": "dogsheep.github.io"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/dogsheep.github.io/issues/7/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 718395987, "node_id": "MDExOlB1bGxSZXF1ZXN0NTAwNzk4MDkx", "number": 1008, "title": "Add json_loads and json_dumps jinja2 filters", "user": {"value": 649467, "label": "mhalle"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2020-10-09T20:11:34Z", "updated_at": "2020-12-15T02:30:28Z", "closed_at": null, "author_association": "FIRST_TIME_CONTRIBUTOR", "pull_request": "simonw/datasette/pulls/1008", "body": "", "repo": {"value": 107914493, "label": "datasette"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1008/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 316621102, "node_id": "MDU6SXNzdWUzMTY2MjExMDI=", "number": 235, "title": "Add limit on the size in KB of data returned from a single query", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2018-04-22T23:01:15Z", "updated_at": "2018-04-24T00:30:02Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "Datasette limits the number of rows returned to 1,000 and limits the time spent executing a SQL query to 1000ms - and both of these limits can be customized.\r\n\r\nIt does not have a limit on the size of the response returned. It's possible to compose maliciously large SQL responses in a small number of rows using mechanisms like the `group_concat()` aggregate function. It would be good to avoid malicious SQL creating 100MB+ responses and potentially crashing the server.\r\n\r\nI think the easiest place to implement that is here:\r\n\r\nhttps://github.com/simonw/datasette/blob/f3f42957128c1e7ece584d45d9167f2ac003a3b8/datasette/app.py#L175-L190\r\n\r\nCurrently we use `cursor.fetchmany()` to fetch up to 1,001 rows at once. Instead, we could switch to iterating through `cursor.fetchone()` (or just using `for row in cursor`) and keeping a running tally of the size of the response as we go - maybe just using `rough_response_size += len(str(row))`. If that goes above a certain threshold we can terminate the response with an error, like we do with timelimits.\r\n\r\nThe bigger challenge here is understanding how well this approach works and what impact it will have on overall Datasette performance. I think I need #33 for this.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/235/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 541274681, "node_id": "MDU6SXNzdWU1NDEyNzQ2ODE=", "number": 2, "title": "Add linkedin-to-sqlite", "user": {"value": 881925, "label": "mnp"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2019-12-21T03:13:40Z", "updated_at": "2019-12-21T03:13:40Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "There is an API available. https://developer.linkedin.com/docs/rest-api#\r\n\r\nAt the minimum, I would think contact list and messages would be of interest.", "repo": {"value": 214746582, "label": "dogsheep.github.io"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/dogsheep.github.io/issues/2/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 504720731, "node_id": "MDU6SXNzdWU1MDQ3MjA3MzE=", "number": 1, "title": "Add more details on how to request data from google takeout correctly.", "user": {"value": 1055831, "label": "dazzag24"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2019-10-09T15:17:34Z", "updated_at": "2019-10-09T15:17:34Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "The default is to download everything.  This can result in an enormous amount of data when you only really need 2 types of data for now:\r\n\r\n- My Activity\r\n- Location History\r\n\r\nIn addition unless you specify that \"My Activity\" is downloaded in JSON format the default is HTML.  This then causes the \r\n\r\n`google-takeout-to-sqlite my-activity takeout.db takeout.zip`\r\n\r\ncommand to fail as it only contains html files not json files.\r\n\r\nThanks", "repo": {"value": 206649770, "label": "google-takeout-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/1/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 697162939, "node_id": "MDU6SXNzdWU2OTcxNjI5Mzk=", "number": 20, "title": "Add more tags so people can find your project.", "user": {"value": 7902810, "label": "ran88dom99"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2020-09-09T21:14:09Z", "updated_at": "2020-09-09T21:14:09Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "quantified-self habit-tracking google-fit time-tracking wearables quantifiedself \r\nfor example", "repo": {"value": 197431109, "label": "dogsheep-beta"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/dogsheep-beta/issues/20/reactions\", \"total_count\": 1, \"+1\": 0, \"-1\": 1, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 1060631257, "node_id": "I_kwDOBm6k_c4_N_LZ", "number": 1528, "title": "Add new `\"sql_file\"` key to Canned Queries in metadata?", "user": {"value": 15178711, "label": "asg017"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2021-11-22T21:58:01Z", "updated_at": "2022-06-10T03:23:08Z", "closed_at": null, "author_association": "CONTRIBUTOR", "pull_request": null, "body": "Currently for canned queries, you have to inline SQL in your `metadata.yaml` like so:\r\n\r\n```yaml\r\ndatabases:\r\n  fixtures:\r\n    queries:\r\n      neighborhood_search:\r\n        sql: |-\r\n          select neighborhood, facet_cities.name, state\r\n          from facetable\r\n            join facet_cities on facetable.city_id = facet_cities.id\r\n          where neighborhood like '%' || :text || '%'\r\n          order by neighborhood\r\n        title: Search neighborhoods\r\n```\r\n\r\nThis works fine, but for a few reasons, I usually have my canned queries already written in separate `.sql` files. I'd like to instead re-use those instead of re-writing it. \r\n\r\nSo, I'd like to see a new `\"sql_file\"` key that works like so:\r\n\r\n`metadata.yaml`:\r\n\r\n```yaml\r\ndatabases:\r\n  fixtures:\r\n    queries:\r\n      neighborhood_search:\r\n        sql_file: neighborhood_search.sql\r\n        title: Search neighborhoods\r\n```\r\n`neighborhood_search.sql`:\r\n```sql\r\nselect neighborhood, facet_cities.name, state\r\nfrom facetable\r\njoin facet_cities on facetable.city_id = facet_cities.id\r\nwhere neighborhood like '%' || :text || '%'\r\norder by neighborhood\r\n```\r\n\r\nBoth of these would work in the exact same way, where Datasette would instead open + include `neighborhood_search.sql` on startup. \r\n\r\n\r\nA few reasons why I'd like to keep my canned queries SQL separate from metadata.yaml:\r\n\r\n- Keeping SQL in standalone SQL files means syntax highlighting and other text editor integrations in my code\r\n- Multiline strings in yaml, while functional, are a tad cumbersome and are hard to edit\r\n- Works well with other tools (can pipe `.sql` files into the `sqlite3` CLI, or use with other SQLite clients easier)\r\n- Typically my canned queries are quite long compared to everything else in my metadata.yaml, so I'd love to separate it where possible\r\n\r\nLet me know if this is a feature you'd like to see, I can try to send up a PR if this sounds right!", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1528/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 352768017, "node_id": "MDU6SXNzdWUzNTI3NjgwMTc=", "number": 362, "title": "Add option to include/exclude columns in search filters", "user": {"value": 78156, "label": "annapowellsmith"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2018-08-22T01:32:08Z", "updated_at": "2020-11-03T19:01:59Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "I have a dataset with many columns, of which only some are likely to be of interest for searching.\r\n\r\nIt would be great for usability if the search filters in the UI could be configured to include/exclude columns.\r\n\r\nSee also: https://github.com/simonw/datasette/issues/292", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/362/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 1453134846, "node_id": "I_kwDOCGYnMM5WnRP-", "number": 513, "title": "Add or document streamlined workflow for importing Datasette csv / json exports", "user": {"value": 19328961, "label": "henry501"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2022-11-17T10:54:47Z", "updated_at": "2022-11-17T10:54:47Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "I'm working on some small front-end enhancements to the laion-aesthetic-datasette project, and I wanted to partially populate a database directly using exports from the existing Datasette instance instead of downloading the parquet files and creating my own multi-GB database.\r\n\r\nThere have been a number of small issues that are certainly related to my relative lack of familiarity with the toolkit, but that are still surprising. \r\n\r\nFor example: a CSV export of the images table (http://laion-aesthetic.datasette.io/laion-aesthetic-6pls.csv?sql=select+rowid%2C+url%2C+text%2C+domain_id%2C+width%2C+height%2C+similarity%2C+punsafe%2C+pwatermark%2C+aesthetic%2C+hash%2C+__index_level_0__+from+images+order+by+random%28%29+limit+100) has nested single quotes, double quotes, and commas that aren't handled by rows_from_file. Similarly, the json output has to be manually transformed to add the column names and remove extraneous information before sqlite_utils can import it.\r\n\r\nI was able to work through these issues, but as an enhancement it would be really helpful to create or document a clear workflow that avoids the friction of this data transformation.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/513/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 1363280254, "node_id": "PR_kwDODFdgUs4-cIa_", "number": 76, "title": "Add organization support to repos command", "user": {"value": 2757699, "label": "OverkillGuy"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2022-09-06T13:21:42Z", "updated_at": "2022-09-06T13:59:08Z", "closed_at": null, "author_association": "FIRST_TIME_CONTRIBUTOR", "pull_request": "dogsheep/github-to-sqlite/pulls/76", "body": "New --organization flag to signify all given \"usernames\" are private\r\norgs. Adapts API URL to the organization path instead.\r\n\r\nNot the best implementation, but a first draft to talk around\r\n\r\nFixes #75 (badly, no tests, overly vague, untested)", "repo": {"value": 207052882, "label": "github-to-sqlite"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/github-to-sqlite/issues/76/reactions\", \"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 1641117021, "node_id": "PR_kwDODtX3eM5M66op", "number": 6, "title": "Add permalink virtual field to items table", "user": {"value": 1231935, "label": "xavdid"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2023-03-26T22:22:38Z", "updated_at": "2023-03-29T18:38:52Z", "closed_at": null, "author_association": "FIRST_TIME_CONTRIBUTOR", "pull_request": "dogsheep/hacker-news-to-sqlite/pulls/6", "body": "I added a virtual column (no storage overhead) to the output that easily links back to the source. It works nicely out of the box with datasette:\r\n\r\n![](https://cdn.zappy.app/faf43661d539ee0fee02c0421de22d65.png)\r\n\r\nI got bit a bit by https://github.com/simonw/sqlite-utils/issues/411, so I went with a manual `table_xinfo` and creating the table via execute. Happy to adjust if that issue moves, but this seems like it works.\r\n\r\nI also added my best-guess instructions for local development on this package. I'm shooting in the dark, so feel free to replace with how you work on it locally.", "repo": {"value": 248903544, "label": "hacker-news-to-sqlite"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/hacker-news-to-sqlite/issues/6/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 724759588, "node_id": "MDU6SXNzdWU3MjQ3NTk1ODg=", "number": 29, "title": "Add search highlighting snippets", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 5, "created_at": "2020-10-19T16:00:48Z", "updated_at": "2021-08-26T20:23:11Z", "closed_at": null, "author_association": "MEMBER", "pull_request": null, "body": "Like on https://til.simonwillison.net/til/search?q=Snippet", "repo": {"value": 197431109, "label": "dogsheep-beta"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/dogsheep-beta/issues/29/reactions\", \"total_count\": 1, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 1, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 915488244, "node_id": "MDU6SXNzdWU5MTU0ODgyNDQ=", "number": 1372, "title": "Add section to \"writing plugins\" about security, e.g. avoiding XSS", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2021-06-08T20:49:33Z", "updated_at": "2021-06-08T20:49:46Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "https://docs.datasette.io/en/stable/writing_plugins.html should have tips on writing secure plugins.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1372/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 1513237712, "node_id": "PR_kwDODEm0Qs5GUoG_", "number": 67, "title": "Add support for app-only bearer tokens", "user": {"value": 26161409, "label": "sometimes-i-send-pull-requests"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2022-12-28T23:31:20Z", "updated_at": "2022-12-28T23:31:20Z", "closed_at": null, "author_association": "FIRST_TIME_CONTRIBUTOR", "pull_request": "dogsheep/twitter-to-sqlite/pulls/67", "body": "Previously, twitter-to-sqlite only supported OAuth1 authentication, and the token must be on behalf of a user.  However, Twitter also supports application-only bearer tokens, documented here:\r\nhttps://developer.twitter.com/en/docs/authentication/oauth-2-0/bearer-tokens This PR adds support to twitter-to-sqlite for using application-only bearer tokens.  To use, the auth.json file just needs to contain a \"bearer_token\" key instead of \"api_key\", \"api_secret_key\", etc.", "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/67/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 1013506559, "node_id": "PR_kwDODFdgUs4skaNS", "number": 68, "title": "Add support for retrieving teams / members", "user": {"value": 68329, "label": "philwills"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2021-10-01T15:55:02Z", "updated_at": "2021-10-01T15:59:53Z", "closed_at": null, "author_association": "FIRST_TIME_CONTRIBUTOR", "pull_request": "dogsheep/github-to-sqlite/pulls/68", "body": "Adds a method for retrieving all the teams within an organisation and all the members in those teams. The latter is stored as a join table `team_members` beteween `teams` and `users`.", "repo": {"value": 207052882, "label": "github-to-sqlite"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/github-to-sqlite/issues/68/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 1822918995, "node_id": "I_kwDOCGYnMM5sp4lT", "number": 580, "title": "Add way to export to a csv file using the Python library", "user": {"value": 44324811, "label": "kevinlinxc"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2023-07-26T18:09:26Z", "updated_at": "2023-07-26T18:09:26Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "According to the documentation, we can make a csv output using the CLI tool, but not the Python library. Could we have the latter?", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/580/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 274615452, "node_id": "MDU6SXNzdWUyNzQ2MTU0NTI=", "number": 111, "title": "Add \u201cupdated\u201d to metadata", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 12, "created_at": "2017-11-16T18:22:20Z", "updated_at": "2021-09-21T22:48:27Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "To give an indication as to when the data was last updated.\r\n\r\nThis should be a field in the metadata that is then shown on the index page and in the footer, if it is set.\r\n\r\nAlso support setting it using an option to \u201cdatasette publish\u201d and \u201cdatasette package\u201d - which can either be a string or can be the magic string \u201ctoday\u201d to set it to today\u2019s date:\r\n\r\n    datasette publish file.db --updated=today", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/111/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 777140799, "node_id": "MDU6SXNzdWU3NzcxNDA3OTk=", "number": 1166, "title": "Adopt Prettier for JavaScript code formatting", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 10, "created_at": "2020-12-31T21:25:27Z", "updated_at": "2022-01-13T22:22:18Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "https://prettier.io/ - I'm going to go with 2 spaces.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1166/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 1781005740, "node_id": "I_kwDOBm6k_c5qJ_2s", "number": 2090, "title": "Adopt ruff for linting", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2023-06-29T14:56:43Z", "updated_at": "2023-06-29T15:05:04Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "https://beta.ruff.rs/docs/", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/2090/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 736365306, "node_id": "MDU6SXNzdWU3MzYzNjUzMDY=", "number": 1083, "title": "Advanced CSV export for arbitrary queries", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2020-11-04T19:23:05Z", "updated_at": "2021-06-17T18:12:31Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "There's no link to download the CSV file - the table page has that as an advanced export option, but this is missing from the query page.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1083/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 1125297737, "node_id": "I_kwDOCGYnMM5DEq5J", "number": 402, "title": "Advanced class-based `conversions=` mechanism", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 14, "created_at": "2022-02-06T19:47:41Z", "updated_at": "2022-02-16T10:18:55Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "The `conversions=` parameter works like this at the moment: https://sqlite-utils.datasette.io/en/3.23/python-api.html#converting-column-values-using-sql-functions\r\n\r\n```python\r\ndb[\"places\"].insert(\r\n    {\"name\": \"Wales\", \"geometry\": wkt},\r\n    conversions={\"geometry\": \"GeomFromText(?, 4326)\"},\r\n)\r\n```\r\nThis proposal is to support values in that dictionary that are objects, not strings, which can represent more complex conversions - spun out from #399.\r\n\r\nNew proposed mechanism:\r\n```python\r\nfrom sqlite_utils.utils import LongitudeLatitude\r\n\r\ndb[\"places\"].insert(\r\n    {\r\n        \"name\": \"London\",\r\n        \"point\": (-0.118092, 51.509865)\r\n    },\r\n    conversions={\"point\": LongitudeLatitude},\r\n)\r\n```\r\nHere `LongitudeLatitude` is a magical value which does TWO things: it sets up the `GeomFromText(?, 4326)` SQL function, and it handles converting the `(51.509865, -0.118092)` tuple into a `POINT({} {})` string.\r\n\r\nThis would involve a change to the `conversions=` contract - where it usually expects a SQL string fragment, but it can also take an object which combines that SQL string fragment with a Python conversion function.\r\n\r\nBest of all... this resolves the `lat, lon` v.s. `lon, lat` dilemma because you can use `from sqlite_utils.utils import LongitudeLatitude` OR `from sqlite_utils.utils import LatitudeLongitude` depending on which you prefer!\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/399#issuecomment-1030739566_", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/402/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 1244082183, "node_id": "PR_kwDODEm0Qs44PPLy", "number": 66, "title": "Ageinfo workaround", "user": {"value": 11887, "label": "ashanan"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2022-05-21T21:08:29Z", "updated_at": "2022-05-21T21:09:16Z", "closed_at": null, "author_association": "FIRST_TIME_CONTRIBUTOR", "pull_request": "dogsheep/twitter-to-sqlite/pulls/66", "body": "I'm not sure if this is due to a new format or just because my ageinfo file is blank, but trying to import an archive would crash when it got to that file.  This PR adds a guard clause in the `ageinfo` transformer and sets a default value that doesn't throw an exception.  Seems likely to be the same issue mentioned by danp in https://github.com/dogsheep/twitter-to-sqlite/issues/54, my ageinfo file looks the same.  Added that same ageinfo file to the test archive as well to help confirm my workaround didn't break anything.\r\n\r\nLet me know if you want any changes!", "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/66/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 1251700382, "node_id": "I_kwDOBm6k_c5Km26e", "number": 1750, "title": "Allow `label_column` to specify array of columns", "user": {"value": 408765, "label": "knutwannheden"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2022-05-28T18:45:48Z", "updated_at": "2022-05-28T18:45:48Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "I think it would be great if the Datasette metadata would allow the `label_column` table key to list multiple columns. Something like:\r\n```json\r\n    \"tables\": {\r\n        \"person\": {\r\n            \"label_column\": [\"first_name\", \"last_name\"]\r\n        },\r\n```\r\nIt would even be interesting with a \"label expression\" similar to a Python f-string. E.g. `{row.last_name}, {row.first_name}`.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1750/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 828858421, "node_id": "MDU6SXNzdWU4Mjg4NTg0MjE=", "number": 1258, "title": "Allow canned query params to specify default values", "user": {"value": 1385831, "label": "wdccdw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 5, "created_at": "2021-03-11T07:19:02Z", "updated_at": "2023-02-20T23:39:58Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "If I call a canned query that includes named parameters, without passing any parameters, datasette runs the query anyway, resulting in an HTTP status code 400, and a visible error in the browser, with only a link back to home.  This means that one of the default links on https://site/database/ will lead to a broken page with no apparent way out.\r\n\r\n![image](https://user-images.githubusercontent.com/1385831/110748683-13e72300-820e-11eb-855c-32e03dfef5bf.png)\r\n\r\nIs there any way to skip performing the query when parameters aren't supplied, but otherwise render the usual canned query page?  Alternatively, can I supply default values for my parameters, either when defining my canned queries or when linking to the canned query page from the default database template.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1258/reactions\", \"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 548591089, "node_id": "MDU6SXNzdWU1NDg1OTEwODk=", "number": 657, "title": "Allow creation of virtual tables at startup", "user": {"value": 1055831, "label": "dazzag24"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2020-01-12T16:10:55Z", "updated_at": "2021-01-15T20:24:35Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "Hi,\r\n \r\nI've been experimenting with SQLite reading from huge datasets using this excellent Parquet extension from @cldellow.\r\nhttps://cldellow.com/2018/06/22/sqlite-parquet-vtable.html\r\nhttps://github.com/cldellow/sqlite-parquet-vtable\r\n\r\nThis works really well, but I was keen to see if I could combine datasette with this. Having previously experimented with the spatialite extension I knew that datasette supports loading extensions in the underlying sqlite instance.  However I hit a blocker as the current design only allows SELECT statements to be executed and so I am unable to execute the crucial \r\n\r\nCREATE VIRTUAL TABLE .........\r\n\r\ncommand that is required to load the data from the parquet file into the table.\r\n\r\nIt seems like this would be a simple-ish change, but I don't know enough about the architecture of datasette to start implementing this myself?  Could this be done as a datasette plugin?  or would this require more fundamental changes at initialisation time?\r\n\r\nMy thoughts are that something at init time could detect that the user was loading a *.parquet file and then switch to a mode were it loads that via the \"CREATE VIRTUAL TABLE...\" rather than loading the *.db file in the default case??\r\n\r\nI'm happy to contribute code and testing, I just need some pointers on the best approach.\r\n\r\nThanks\r\nDarren", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/657/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 1227571375, "node_id": "I_kwDOCGYnMM5JK0Cv", "number": 431, "title": "Allow making m2m relation of a table to itself", "user": {"value": 738408, "label": "rafguns"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2022-05-06T08:30:43Z", "updated_at": "2022-06-23T14:12:51Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "I am building a database, in which one of the tables has a many-to-many relationship to itself. As far as I can see, this is not (yet) possible using `.m2m()` in sqlite-utils. This may be a bit of a niche use case, so feel free to close this issue if you feel it would introduce too much complexity compared to the benefits.\r\n\r\nExample: suppose I have a table of people, and I want to store the information that John and Mary have two children, Michael and Suzy. It would be neat if I could do something like this:\r\n\r\n```python\r\nfrom sqlite_utils import Database\r\n\r\ndb = Database(memory=True)\r\ndb[\"people\"].insert({\"name\": \"John\"}, pk=\"name\").m2m(\r\n    \"people\", [{\"name\": \"Michael\"}, {\"name\": \"Suzy\"}], m2m_table=\"parent_child\", pk=\"name\"\r\n)\r\ndb[\"people\"].insert({\"name\": \"Mary\"}, pk=\"name\").m2m(\r\n    \"people\", [{\"name\": \"Michael\"}, {\"name\": \"Suzy\"}], m2m_table=\"parent_child\", pk=\"name\"\r\n)\r\n```\r\n\r\nBut if I do that, the many-to-many table `parent_child` has only one column:\r\n```\r\nCREATE TABLE [parent_child] (\r\n   [people_id] TEXT REFERENCES [people]([name]),\r\n   PRIMARY KEY ([people_id], [people_id])\r\n)\r\n```\r\n\r\nThis could be solved by adding one or two keyword_arguments to `.m2m()`, e.g. `.m2m(..., left_name=None, right_name=None)` or `.m2m(..., names=(None, None))`.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/431/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 1033678984, "node_id": "PR_kwDOBm6k_c4tjgJ8", "number": 1495, "title": "Allow routes to have extra options", "user": {"value": 536941, "label": "fgregg"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 5, "created_at": "2021-10-22T15:00:45Z", "updated_at": "2021-11-19T15:36:27Z", "closed_at": null, "author_association": "CONTRIBUTOR", "pull_request": "simonw/datasette/pulls/1495", "body": "Right now, datasette routes can only be a 2-tuple of `(regex, view_fn)`. \r\n\r\nIf it was possible for datasette to handle extra options, like [standard Django does](https://docs.djangoproject.com/en/3.2/topics/http/urls/#passing-extra-options-to-view-functions), it would add flexibility for plugin authors.\r\n\r\nFor example, if extra options were enabled, then it would be easy to make a single table the home page (#1284). This plugin would accomplish it.\r\n\r\n```python\r\nfrom datasette import hookimpl\r\nfrom datasette.views.table import TableView\r\n\r\n@hookimpl\r\ndef register_routes(datasette):\r\n        return [\r\n        (r\"^/$\", TableView.as_view(datasette), {'db_name': 'DB_NAME',\r\n                                                'table': 'TABLE_NAME'})\r\n    ]\r\n```\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1495/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 1504352503, "node_id": "I_kwDOBm6k_c5Zqpj3", "number": 1968, "title": "Allow to hide some queries in metadata.yml", "user": {"value": 562352, "label": "CharlesNepote"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2022-12-20T10:45:41Z", "updated_at": "2022-12-20T10:45:41Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "By default all queries are displayed.\r\n\r\nBut there are many cases where it would be interesting to hide the queries by default:\r\n* the website is targeting non-tech people\r\n* the query is veeeeeery long ([eg.](https://mirabelle.openfoodfacts.org/products/energy_calculator))\r\n* reading the query is not important for the users, they only want to see the result\r\n\r\nOf course, the user still could have the option to see the query.\r\n\r\nIt could be an option in the metadata file:\r\n```yml\r\ndatabases:\r\n  awesome_db:\r\n    tables:\r\n      products:\r\n        hide_sql: true\r\n    queries:\r\n      great_query:\r\n        hide_sql: true\r\n        sql: select * from products where code = :barcode\r\n```\r\n\r\nThe priority could be:\r\n* no option in the metadata and nothing in the URL: query displayed\r\n* hide_sql in the metadata and nothing in the URL: query displayed as asked in the metadata\r\n* hide_sql in the metadata and &_hide_sql= in the URL: query as asked in the URL\r\n\r\nSee also: #1824\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1968/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 607888367, "node_id": "MDU6SXNzdWU2MDc4ODgzNjc=", "number": 13, "title": "Also upload movie files", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2020-04-27T22:11:25Z", "updated_at": "2020-04-28T00:39:45Z", "closed_at": null, "author_association": "MEMBER", "pull_request": null, "body": "The `upload` command currently only handles static images:\r\n\r\nhttps://github.com/dogsheep/photos-to-sqlite/blob/d939455af00e07866686457ee2fcb9b2d1b7194e/photos_to_sqlite/utils.py#L26-L33\r\n\r\nNeed to cover movies taken by my phone and DSLR too.", "repo": {"value": 256834907, "label": "dogsheep-photos"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/dogsheep-photos/issues/13/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 1500636982, "node_id": "I_kwDOBm6k_c5Zcec2", "number": 1962, "title": "Alternative, async-friendly pattern for `make_app_client()` and similar - fully retire `TestClient`", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2022-12-16T17:56:51Z", "updated_at": "2022-12-16T21:55:29Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "In this issue I replaced a whole bunch of places that used the non-async `app_client` fixture with an async `ds_client` fixture instead:\r\n- #1959\r\n\r\nBut I didn't get everything, and a lot of tests are still using the old `TestClient` mechanism as a result.\r\n\r\nThe main work here is replacing all of the `app_client_...` fixtures which use variants on the default client - and changing the tests that call `make_app_client()` to do something else instead.\r\n\r\nThis requires some careful thought. I need to come up with a really nice pattern for creating variants on the `ds_client` default fixture - and do so in a way that minimizes the number of open files, refs:\r\n\r\n- #1843", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1962/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 608512747, "node_id": "MDU6SXNzdWU2MDg1MTI3NDc=", "number": 14, "title": "Annotate photos using the Google Cloud Vision API", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 5, "created_at": "2020-04-28T18:09:03Z", "updated_at": "2020-04-28T18:19:06Z", "closed_at": null, "author_association": "MEMBER", "pull_request": null, "body": "It can detect faces, run OCR, do image labeling (it knows what a lemur is!) and do object localization where it identifies objects and returns bounding polygons for them.", "repo": {"value": 256834907, "label": "dogsheep-photos"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/dogsheep-photos/issues/14/reactions\", \"total_count\": 3, \"+1\": 2, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 1, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 1564769997, "node_id": "I_kwDOBm6k_c5dRH7N", "number": 2011, "title": "Applied facet did not result in an \"x\" icon to dismiss it", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2023-01-31T17:57:44Z", "updated_at": "2023-01-31T17:58:54Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "![CleanShot 2023-01-31 at 09 55 56@2x](https://user-images.githubusercontent.com/9599/215843684-1761a230-d490-4f87-be6d-186319366794.png)\r\n\r\nThat's against this data https://data.sfgov.org/City-Management-and-Ethics/Supplier-Contracts/cqi5-hm2d imported using https://datasette.io/plugins/datasette-socrata\r\n\r\nIt's for `Contract Type` of `Non-Purchasing Contract (Rents, etc.)` - so possible that some of the spaces or punctuation in either the name of the value tripped up the code that decides if the X icon should be displayed.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/2011/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 779088071, "node_id": "MDU6SXNzdWU3NzkwODgwNzE=", "number": 54, "title": "Archive import appears to be broken on recent exports", "user": {"value": 21148, "label": "jacobian"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 5, "created_at": "2021-01-05T14:18:01Z", "updated_at": "2023-01-04T11:06:55Z", "closed_at": null, "author_association": "CONTRIBUTOR", "pull_request": null, "body": "I requested a Twitter export yesterday, and unfortunately they seem to have changed it such that `twitter-to-sqlite import` can't handle it anymore \ud83d\ude22 \r\n\r\nSo far I've ran into two issues. The first was easy to work around, but the second will take more investigation. If I can find the time I'll keep working on it and update this issue accordingly.\r\n\r\nThe issues (so far):\r\n\r\n### 1. Data seems to have moved to a `data/` subdirectory\r\n\r\nRunning `twitter-to-sqlite import` on the raw zip file reports a bunch of \"not yet implemented\" errors, and then exits without actually importing anything:\r\n\r\n```\r\n\u276f twitter-to-sqlite import tarchive.db twitter.zip\r\n...\r\ndata/manifest: not yet implemented\r\ndata/account-creation-ip: not yet implemented\r\ndata/account-suspension: not yet implemented\r\n... (dozens of more lines like this, including critical stuff like data/tweets) ...\r\n```\r\n\r\n(`tarchive.db` now exists, but is empty)\r\n\r\nWorkaround: unpack the zip file, and run `twitter-to-sqlite import tarchive.db path/to/archive/data`\r\n\r\nThat gets further, but:\r\n\r\n### 2. Some schema(s?) have changed\r\n\r\nAt least, the `blocks` schema seems different now:\r\n\r\n```\r\n\u276f twitter-to-sqlite import tarchive.db archive/data\r\ndirect-messages-group: not yet implemented\r\nbranch-links: not yet implemented\r\nperiscope-expired-broadcasts: not yet implemented\r\ndirect-messages: not yet implemented\r\nmute: not yet implemented\r\nTraceback (most recent call last):\r\n  File \"/Users/jacob/Library/Caches/pypoetry/virtualenvs/jacobian-dogsheep-4AXaN4tu-py3.8/bin/twitter-to-sqlite\", line 8, in <module>\r\n    sys.exit(cli())\r\n  File \"/Users/jacob/Library/Caches/pypoetry/virtualenvs/jacobian-dogsheep-4AXaN4tu-py3.8/lib/python3.8/site-packages/click/core.py\", line 829, in __call__\r\n    return self.main(*args, **kwargs)\r\n  File \"/Users/jacob/Library/Caches/pypoetry/virtualenvs/jacobian-dogsheep-4AXaN4tu-py3.8/lib/python3.8/site-packages/click/core.py\", line 782, in main\r\n    rv = self.invoke(ctx)\r\n  File \"/Users/jacob/Library/Caches/pypoetry/virtualenvs/jacobian-dogsheep-4AXaN4tu-py3.8/lib/python3.8/site-packages/click/core.py\", line 1259, in invoke\r\n    return _process_result(sub_ctx.command.invoke(sub_ctx))\r\n  File \"/Users/jacob/Library/Caches/pypoetry/virtualenvs/jacobian-dogsheep-4AXaN4tu-py3.8/lib/python3.8/site-packages/click/core.py\", line 1066, in invoke\r\n    return ctx.invoke(self.callback, **ctx.params)\r\n  File \"/Users/jacob/Library/Caches/pypoetry/virtualenvs/jacobian-dogsheep-4AXaN4tu-py3.8/lib/python3.8/site-packages/click/core.py\", line 610, in invoke\r\n    return callback(*args, **kwargs)\r\n  File \"/Users/jacob/Library/Caches/pypoetry/virtualenvs/jacobian-dogsheep-4AXaN4tu-py3.8/lib/python3.8/site-packages/twitter_to_sqlite/cli.py\", line 772, in import_\r\n    archive.import_from_file(db, filepath.name, open(filepath, \"rb\").read())\r\n  File \"/Users/jacob/Library/Caches/pypoetry/virtualenvs/jacobian-dogsheep-4AXaN4tu-py3.8/lib/python3.8/site-packages/twitter_to_sqlite/archive.py\", line 215, in import_from_file\r\n    to_insert = transformer(data)\r\n  File \"/Users/jacob/Library/Caches/pypoetry/virtualenvs/jacobian-dogsheep-4AXaN4tu-py3.8/lib/python3.8/site-packages/twitter_to_sqlite/archive.py\", line 115, in lists_member\r\n    return {\"lists-member\": _list_from_common(data)}\r\n  File \"/Users/jacob/Library/Caches/pypoetry/virtualenvs/jacobian-dogsheep-4AXaN4tu-py3.8/lib/python3.8/site-packages/twitter_to_sqlite/archive.py\", line 200, in _list_from_common\r\n    for url in block[\"userListInfo\"][\"urls\"]:\r\nKeyError: 'urls'\r\n```\r\n\r\nThat's as far as I got before I needed to work on something else. I'll report back if I get further!", "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/54/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 1513238455, "node_id": "PR_kwDODEm0Qs5GUoPm", "number": 71, "title": "Archive: Fix \"ni devices\" typo in importer", "user": {"value": 26161409, "label": "sometimes-i-send-pull-requests"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2022-12-28T23:33:31Z", "updated_at": "2022-12-28T23:33:31Z", "closed_at": null, "author_association": "FIRST_TIME_CONTRIBUTOR", "pull_request": "dogsheep/twitter-to-sqlite/pulls/71", "body": null, "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/71/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 1513238314, "node_id": "PR_kwDODEm0Qs5GUoN6", "number": 70, "title": "Archive: Import Twitter Circle data", "user": {"value": 26161409, "label": "sometimes-i-send-pull-requests"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2022-12-28T23:33:09Z", "updated_at": "2022-12-28T23:33:09Z", "closed_at": null, "author_association": "FIRST_TIME_CONTRIBUTOR", "pull_request": "dogsheep/twitter-to-sqlite/pulls/70", "body": null, "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/70/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 1513237982, "node_id": "PR_kwDODEm0Qs5GUoKL", "number": 68, "title": "Archive: Import mute table", "user": {"value": 26161409, "label": "sometimes-i-send-pull-requests"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2022-12-28T23:32:06Z", "updated_at": "2022-12-28T23:32:06Z", "closed_at": null, "author_association": "FIRST_TIME_CONTRIBUTOR", "pull_request": "dogsheep/twitter-to-sqlite/pulls/68", "body": null, "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/68/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 1513238152, "node_id": "PR_kwDODEm0Qs5GUoMM", "number": 69, "title": "Archive: Import new tweets table name", "user": {"value": 26161409, "label": "sometimes-i-send-pull-requests"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2022-12-28T23:32:44Z", "updated_at": "2022-12-28T23:32:44Z", "closed_at": null, "author_association": "FIRST_TIME_CONTRIBUTOR", "pull_request": "dogsheep/twitter-to-sqlite/pulls/69", "body": "Given the code here, it seems like in the past this file was named \"tweet.js\".  In recent exports, it's named \"tweets.js\".  The archive importer needs to be modified to take this into account.  Existing logic is reused for importing this table.  (However, the resulting table name will be different, matching the different file name -- archive_tweets, rather than archive_tweet).", "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/69/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 817989436, "node_id": "MDU6SXNzdWU4MTc5ODk0MzY=", "number": 242, "title": "Async support", "user": {"value": 25778, "label": "eyeseast"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 13, "created_at": "2021-02-27T18:29:38Z", "updated_at": "2021-10-28T14:37:56Z", "closed_at": null, "author_association": "CONTRIBUTOR", "pull_request": null, "body": "Following our conversation last week, want to note this here before I forget.\r\n\r\nI've had a couple situations where I'd like to do a bunch of updates in an async event loop, but I run into SQLite's issues with concurrent writes. This feels like something sqlite-utils could help with.\r\n\r\nPeeWee ORM has a [SQLite write queue](http://docs.peewee-orm.com/en/latest/peewee/playhouse.html#sqliteq) that might be a good model. It's using threads or gevent, but I _think_ that approach would translate well enough to asyncio. \r\n\r\nHappy to help with this, too.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/242/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 1113384383, "node_id": "I_kwDOBm6k_c5CXOW_", "number": 1611, "title": "Avoid ever running count(*) against SpatiaLite KNN table", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2022-01-25T03:32:54Z", "updated_at": "2022-02-02T06:45:47Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "Got this in a trace:\r\n\r\n<img width=\"941\" alt=\"image\" src=\"https://user-images.githubusercontent.com/9599/150906011-5f09dc6d-4def-433a-8546-e4fd94e0edf0.png\">\r\n\r\nLooks like running `count(*)` against KNN took 83s! It ignored the time limit. And still only returned a count of 0.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1611/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 1586980089, "node_id": "PR_kwDOBm6k_c5KF-by", "number": 2026, "title": "Avoid repeating primary key columns if included in _col args", "user": {"value": 8513, "label": "runderwood"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2023-02-16T04:16:25Z", "updated_at": "2023-02-16T04:16:41Z", "closed_at": null, "author_association": "FIRST_TIME_CONTRIBUTOR", "pull_request": "simonw/datasette/pulls/2026", "body": "...while maintaining given order.\r\n\r\nFixes #1975 (if I'm understanding correctly).\r\n\r\n<!-- readthedocs-preview datasette start -->\r\n----\n:books: Documentation preview :books:: https://datasette--2026.org.readthedocs.build/en/2026/\n\r\n<!-- readthedocs-preview datasette end -->", "repo": {"value": 107914493, "label": "datasette"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/2026/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 1452360613, "node_id": "I_kwDOBm6k_c5WkUOl", "number": 1895, "title": "Avoid using host name when building absolute URLs?", "user": {"value": 14294, "label": "hubgit"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2022-11-16T22:21:27Z", "updated_at": "2022-11-16T22:21:27Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "When deploying Datasette to Cloud Run and rewriting certain routes from a Firebase app to the Cloud Run service, some of the URLs in the page start with `https://[service].run.app` rather than the (custom) domain of the Firebase app. \r\n\r\nI guess this is because a) the custom domain of the Firebase app isn't being passed through in the `host` header of the request to the Cloud Run instance and b) the `absolute_url` function in Datasette is using information from the request to build the URL.\r\n\r\nWould it be possible to not use the host name when building the absolute URLs, i.e. only include the path in the URL?", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1895/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 516874735, "node_id": "MDU6SXNzdWU1MTY4NzQ3MzU=", "number": 613, "title": "Basic join support for table view", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2019-11-03T19:12:53Z", "updated_at": "2019-11-03T19:14:01Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "I think it would be possible to support basic foreign key joins on the table page.\r\n\r\nThe user could specify columns that should result in a join (from a set of suggestions similar to how facets work right now) and they could then be passed as `?_join=city_id`  arguments.\r\n\r\nThis feature will make a lot of sense when combined with the ability to show / hide / customize columns, see #292", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/613/reactions\", \"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}