id,node_id,number,title,user,state,locked,assignee,milestone,comments,created_at,updated_at,closed_at,author_association,pull_request,body,repo,type,active_lock_reason,performed_via_github_app,reactions,draft,state_reason 403625674,MDU6SXNzdWU0MDM2MjU2NzQ=,7,.insert_all() should accept a generator and process it efficiently,9599,closed,0,,,3,2019-01-28T02:11:58Z,2019-01-28T06:26:53Z,2019-01-28T06:26:53Z,OWNER,,"Right now you have to load every record into memory before passing the list to `.insert_all()` and friends. If you want to process millions of rows, this is inefficient. Python has generators - we should use them! The only catch here is that part of the magic of `sqlite-utils` is that it guesses the column types and creates the table for you. This code will need to be updated to notice if the table needs creating and, if it does, create it using the first X (where x=1,000 but can be customized) records. If a record outside of those first 1,000 has a rogue column, we can crash with an error. This will free us up to make the `--nl` option added in #6 much more efficient.",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/7/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 742011049,MDU6SXNzdWU3NDIwMTEwNDk=,1091,.json and .csv exports fail to apply base_url,9599,closed,0,,6346396,22,2020-11-12T23:45:16Z,2021-01-24T21:20:24Z,2021-01-09T22:19:29Z,OWNER,,"> Just tested with the latest Docker image, and it works pretty much everywhere! THANK YOU! > > I did notice that if I try to export json or csv, the base is not applied. Not sure if I should reopen this issue or open a new one. > > To see this, go here: https://corpora.tika.apache.org/datasette/corpora-metadata/REF_PARSE_EXCEPTION_TYPES > > Click/hover over json or CSV and you'll see that the 'datasette' base is not included. _Originally posted by @tballison in https://github.com/simonw/datasette/issues/865#issuecomment-726385422_",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1091/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 403617881,MDU6SXNzdWU0MDM2MTc4ODE=,405,.json?_nl=on option for exporting newline-delimited JSON,9599,closed,0,,,2,2019-01-28T01:10:45Z,2019-01-28T01:49:00Z,2019-01-28T01:48:37Z,OWNER,,"The neat thing about newline-delimited JSON is that you don't have to read an entire array (of potentially thousands of objects) into memory in order to parse it - you can parse things a line at a time instead. It will look like this: `https://latest.datasette.io/fixtures/facetable.json?_shape=array&_nl=on` ``` {""pk"": 1, ""planet_int"": 1, ""on_earth"": 1, ""state"": ""CA"", ""city_id"": 1, ""neighborhood"": ""Mission""} {""pk"": 2, ""planet_int"": 1, ""on_earth"": 1, ""state"": ""CA"", ""city_id"": 1, ""neighborhood"": ""Dogpatch""} {""pk"": 3, ""planet_int"": 1, ""on_earth"": 1, ""state"": ""CA"", ""city_id"": 1, ""neighborhood"": ""SOMA""} {""pk"": 4, ""planet_int"": 1, ""on_earth"": 1, ""state"": ""CA"", ""city_id"": 1, ""neighborhood"": ""Tenderloin""} {""pk"": 5, ""planet_int"": 1, ""on_earth"": 1, ""state"": ""CA"", ""city_id"": 1, ""neighborhood"": ""Bernal Heights""} ``` I added this as part of the `sqlite-utils json` CLI command is this commit - I think Datasette should offer it as well: https://github.com/simonw/sqlite-utils/commit/5466c9745dfef858286146ea158ffd5a71391d10 It can be offered alongside `_stream=on` (which currently only works for CSV, but it could work for JSON as well thanks to this trick).",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/405/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 787900412,MDU6SXNzdWU3ODc5MDA0MTI=,222,.m2m() should accept alter=True parameter,9599,closed,0,,,0,2021-01-18T04:15:43Z,2021-01-18T04:26:10Z,2021-01-18T04:26:10Z,OWNER,,Needed by https://github.com/dogsheep/swarm-to-sqlite/issues/11,140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/222/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 738128913,MDU6SXNzdWU3MzgxMjg5MTM=,201,.search(columns=) and sqlite-utils search -c ... bug,9599,closed,0,,6079500,1,2020-11-07T01:27:26Z,2020-11-08T16:54:15Z,2020-11-08T16:54:15Z,OWNER,,"Both `table.search(columns=)` and the `sqlite-utils search -c` option do not work as expected - they always return both the `rowid` and the `rank` columns even if those have not been requested. This should be fixed before the 3.0 non-alpha release.",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/201/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1856075668,I_kwDOCGYnMM5uoXeU,586,.transform() fails to drop column if table is part of a view,9599,open,0,,,3,2023-08-18T05:25:22Z,2023-08-18T06:13:47Z,,OWNER,,"I got this error trying to drop a column from a table that was part of a SQL view: > error in view plugins: no such table: main.pypi_releases Upon further investigation I found that this pattern seemed to fix it: ```python def transform_the_table(conn): # Run this in a transaction: with conn: # We have to read all the views first, because we need to drop and recreate them db = sqlite_utils.Database(conn) views = {v.name: v.schema for v in db.views if table.lower() in v.schema.lower()} for view in views.keys(): db[view].drop() db[table].transform( types=types, rename=rename, drop=drop, column_order=[p[0] for p in order_pairs], ) # Now recreate the views for name, schema in views.items(): db.create_view(name, schema) ``` So grab a copy of any view that might reference this table, start a transaction, drop those views, run the transform, recreate the views again. > I wonder if this should become an option in `sqlite-utils`? Maybe a `recreate_views=True` argument for `table.tranform(...)`? Should it be opt-in or opt-out? _Originally posted by @simonw in https://github.com/simonw/datasette-edit-schema/issues/35#issuecomment-1683370548_ ",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/586/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 925320167,MDU6SXNzdWU5MjUzMjAxNjc=,284,.transform(types=) turns rowid into a concrete column,9599,closed,0,,,5,2021-06-19T05:25:27Z,2021-06-19T15:28:30Z,2021-06-19T15:28:30Z,OWNER,,"Noticed this in the tests for `sqlite-utils memory` in #282 - is it possible to fix this? https://github.com/simonw/sqlite-utils/commit/ec5174ed40fa283cb06f25ee0c0136297ec313ae",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/284/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 561460274,MDU6SXNzdWU1NjE0NjAyNzQ=,84,.upsert() with hash_id throws error,9599,closed,0,,,0,2020-02-07T07:08:19Z,2020-02-07T07:17:11Z,2020-02-07T07:17:11Z,OWNER,,"```python db[table_name].upsert_all(rows, hash_id=""pk"") ``` This throws an error: `PrimaryKeyRequired('upsert() requires a pk')` The problem is, if you try this: ```python db[table_name].upsert_all(rows, hash_id=""pk"", pk=""pk"") ``` You get this error: `AssertionError('Use either pk= or hash_id=')` `hash_id=` should imply that `pk=` that column.",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/84/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 598640234,MDU6SXNzdWU1OTg2NDAyMzQ=,99,.upsert_all() should maybe error if dictionaries passed to it do not have the same keys,9599,closed,0,,,2,2020-04-13T03:02:25Z,2020-04-13T03:05:20Z,2020-04-13T03:05:04Z,OWNER,,"While investigating #98 I stumbled across this: ``` def test_upsert_compound_primary_key(fresh_db): table = fresh_db[""table""] table.upsert_all( [ {""species"": ""dog"", ""id"": 1, ""name"": ""Cleo"", ""age"": 4}, {""species"": ""cat"", ""id"": 1, ""name"": ""Catbag""}, ], pk=(""species"", ""id""), ) table.upsert_all( [ {""species"": ""dog"", ""id"": 1, ""age"": 5}, {""species"": ""dog"", ""id"": 2, ""name"": ""New Dog"", ""age"": 1}, ], pk=(""species"", ""id""), ) > assert [ {""species"": ""dog"", ""id"": 1, ""name"": ""Cleo"", ""age"": 5}, {""species"": ""cat"", ""id"": 1, ""name"": ""Catbag"", ""age"": None}, {""species"": ""dog"", ""id"": 2, ""name"": ""New Dog"", ""age"": 1}, ] == list(table.rows) E AssertionError: assert [{'age': 5, '...cies': 'dog'}] == [{'age': 5, '...cies': 'dog'}] E At index 0 diff: {'species': 'dog', 'id': 1, 'name': 'Cleo', 'age': 5} != {'species': 'dog', 'id': 1, 'name': None, 'age': 5} E Full diff: E - [{'age': 5, 'id': 1, 'name': 'Cleo', 'species': 'dog'}, E ? ^^^ -- E + [{'age': 5, 'id': 1, 'name': None, 'species': 'dog'}, E ? ^^^ E {'age': None, 'id': 1, 'name': 'Catbag', 'species': 'cat'}, E {'age': 1, 'id': 2, 'name': 'New Dog', 'species': 'dog'}] ``` If you run `.upsert_all()` with multiple dictionaries it doesn't quite have the effect you might expect.",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/99/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 315548495,MDU6SXNzdWUzMTU1NDg0OTU=,225,/-/(inspect|metadata|plugins)(.json)? introspection,9599,closed,0,,,0,2018-04-18T16:14:58Z,2018-04-19T05:25:33Z,2018-04-19T05:25:33Z,OWNER,,"3 pages (and accompanying .json endpoints) for viewing: * the metadata.json that datasette was loaded with * the output of ds.inspect() * a list of installed plugins, detected by pluggy",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/225/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 800669347,MDU6SXNzdWU4MDA2NjkzNDc=,1216,"/-/databases should reflect connection order, not alphabetical order",9599,closed,0,,,1,2021-02-03T20:20:23Z,2021-06-02T03:10:19Z,2021-06-02T03:10:19Z,OWNER,,"The order in which databases are attached to Datasette matters - it affects the homepage, and it's beginning to influence how certain plugins work (see https://github.com/simonw/datasette-tiles/issues/8). Two years ago in cccea85be6aaaeadb31f3b588ec7f732628815f5 I made `/-/databases` return things in alphabetical order, to fix a test failure in Python 3.5. Python 3.5 is no longer supported, so this is no longer necessary - and this behaviour should now be treated as a bug.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1216/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 444997937,MDU6SXNzdWU0NDQ5OTc5Mzc=,470,/-/databases showing currently attached database details,9599,closed,0,,4305096,1,2019-05-16T14:45:18Z,2019-05-19T19:28:44Z,2019-05-16T14:50:26Z,OWNER,,"Split from #419. Mainly useful to see what is connected as mutable v.s. immutable. Also helps fill the gap left by `/-/inspect` until #465 ",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/470/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 637253789,MDU6SXNzdWU2MzcyNTM3ODk=,833,/-/metadata and so on should respect view-instance permission,9599,closed,0,,5512395,4,2020-06-11T19:07:21Z,2020-06-11T22:15:32Z,2020-06-11T22:14:59Z,OWNER,,"The only URLs that should be available without authentication at all times are the `/-/static/` prefix, to allow for HTTP caching.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/833/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1122416919,I_kwDOBm6k_c5C5rkX,1623,/-/patterns returns link: alternate JSON header to 404,9599,closed,0,,3268330,2,2022-02-02T21:42:49Z,2022-03-19T04:04:49Z,2022-02-02T21:48:56Z,OWNER,,"Bug from: - #1620 ``` % curl -s -I 'https://latest.datasette.io/-/patterns' | grep link link: https://latest.datasette.io/-/patterns.json; rel=""alternate""; type=""application/json+datasette"" ```",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1623/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 319954545,MDU6SXNzdWUzMTk5NTQ1NDU=,248,/-/plugins should show version of each installed plugin,9599,closed,0,,,2,2018-05-03T14:50:45Z,2018-05-04T18:25:40Z,2018-05-04T18:05:04Z,OWNER,,"Refs #244 https://stackoverflow.com/questions/20180543/how-to-check-version-of-python-modules ``` >>> import pkg_resources >>> pkg_resources.get_distribution('datasette_cluster_map').version '0.4' ```",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/248/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 512218858,MDU6SXNzdWU1MTIyMTg4NTg=,606,/-/plugins shows incorrect name for plugins,9599,closed,0,,,3,2019-10-24T22:53:25Z,2019-11-01T05:41:04Z,2019-11-01T05:40:07Z,OWNER,,"https://fivethirtyeight.datasettes.com/-/plugins ```json [ { ""name"": ""datasette_jellyfish"", ""static"": false, ""templates"": false, ""version"": ""0.3"" }, { ""name"": ""datasette_vega"", ""static"": true, ""templates"": false, ""version"": ""0.6.2"" } ] ``` These should be shown as `datasette-jellyfish` and `datasette-vega` since those are the names on PyPI.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/606/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 778530523,MDU6SXNzdWU3Nzg1MzA1MjM=,1172,/-/static should be excluded from auth and permission checks,9599,open,0,,,0,2021-01-05T02:53:41Z,2021-01-05T02:53:41Z,,OWNER,,"I want to set far future / immutable cache headers on everything served from `/-/static` and `/-/static-plugins` This has security implications since it will be possible to see what plugins are installed by checking for known static URLs. I'm fine with that - performance is more important here.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1172/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 398089089,MDU6SXNzdWUzOTgwODkwODk=,399,/-/versions for official Docker image returns wrong Datasette version,9599,closed,0,,,2,2019-01-11T01:19:58Z,2019-01-13T23:31:59Z,2019-01-13T23:10:45Z,OWNER,,"``` docker run -p 8001:8001 datasetteproject/datasette datasette -p 8001 -h 0.0.0.0 ``` http://0.0.0.0:8001/-/versions returns this: ``` { ""datasette"": { ""version"": ""0+unknown"" }, ... ``` This is because the Docker image is built by copying in the Datasette source code, which confuses versioneer. Maybe the Docker image should install the code using a wheel or similar? ",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/399/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 318738000,MDU6SXNzdWUzMTg3MzgwMDA=,244,/-/versions page,9599,closed,0,,,1,2018-04-29T18:22:15Z,2018-05-03T14:13:49Z,2018-05-03T14:09:53Z,OWNER,,"Displays the current version of: * datasette * Python * SQLite * Spatialite (if available) Installed plugin versions should be shown on /-/plugins",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/244/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 321624016,MDU6SXNzdWUzMjE2MjQwMTY=,252,/-/versions should report the FTS version supported by SQLite,9599,closed,0,,,0,2018-05-09T15:43:47Z,2018-05-11T13:19:52Z,2018-05-11T13:19:52Z,OWNER,,I can copy this function from `csvs-to-sqlite`: https://github.com/simonw/csvs-to-sqlite/blob/dccbf65b37bc9eed50e9edb80a42f257e93edb1f/csvs_to_sqlite/utils.py#L283-L293,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/252/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 663317875,MDU6SXNzdWU2NjMzMTc4NzU=,905,/database.db download should include content-length header,9599,closed,0,,,2,2020-07-21T21:23:48Z,2020-07-22T04:59:46Z,2020-07-22T04:52:45Z,OWNER,,I can do this by modifying this function: https://github.com/simonw/datasette/blob/02dc6298bdbfb1d63e0d2a39ff597b5fcc60e06b/datasette/utils/asgi.py#L248-L270,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/905/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 462117311,MDU6SXNzdWU0NjIxMTczMTE=,531,/database/-/inspect,9599,open,0,,,1,2019-06-28T16:33:41Z,2019-07-08T15:43:57Z,,OWNER,,"Build `/database/-/inspect` which shows tables, columns, column types and foreign keys It won't show table counts. Or maybe it will include them optionally but only for `-i` databases, in a special area of the JSON reserved for immutable-only inspect details. _Originally posted by @simonw in https://github.com/simonw/datasette/issues/465#issuecomment-506797086_",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/531/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 267886865,MDU6SXNzdWUyNjc4ODY4NjU=,28,/database?sql= should redirect correctly,9599,closed,0,,2857392,0,2017-10-24T03:38:44Z,2017-10-24T23:54:30Z,2017-10-24T23:54:30Z,OWNER,,Needs to redirect to the location with the hash while retaining the query string. This should also work with the .json extension.,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/28/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1483320357,I_kwDOBm6k_c5Yaawl,1937,/db/-/create API should require insert-rows permission to use row: or rows: option,9599,closed,0,,8711695,2,2022-12-08T01:33:09Z,2022-12-14T20:21:26Z,2022-12-14T20:21:26Z,OWNER,,Otherwise someone with `create-table` but no` insert-rows` permission could abuse it to insert data.,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1937/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1426080014,I_kwDOBm6k_c5VAEEO,1867,/db/table/-/rename API (also allows atomic replace),9599,open,0,,8755003,1,2022-10-27T18:13:23Z,2023-01-09T15:34:12Z,,OWNER,,"> There's one catch with batched inserts: if your CLI tool fails half way through you could end up with a partially populated table - since a bunch of batches will have succeeded first. > > ... > > If people care about that kind of thing they could always push all of their inserts to a table called `_tablename` and then atomically rename that once they've uploaded all of the data (assuming I provide an atomic-rename-this-table mechanism). _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1866#issuecomment-1293893789_ ",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1867/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 1432013704,I_kwDOBm6k_c5VWsuI,1878,/db/table/-/upsert API,9599,closed,0,,8711695,8,2022-11-01T20:01:18Z,2022-12-08T01:12:18Z,2022-12-08T01:12:17Z,OWNER,,Equivalent to `sqlite-utils upsert`: https://sqlite-utils.datasette.io/en/stable/python-api.html#upserting-data,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1878/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 647095808,MDU6SXNzdWU2NDcwOTU4MDg=,874,/favicon.ico 500 error,9599,closed,0,,5533512,0,2020-06-29T04:04:22Z,2020-06-29T04:27:18Z,2020-06-29T04:27:18Z,OWNER,,"``` Traceback (most recent call last): File ""...datasette/datasette/app.py"", line 969, in route_path response = await view(request, send) TypeError: favicon() missing 1 required positional argument: 'send' ```",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/874/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1501713288,I_kwDOBm6k_c5ZglOI,1963,0.63.3 bugfix release,9599,closed,0,,,2,2022-12-18T02:48:15Z,2022-12-18T03:26:55Z,2022-12-18T03:26:55Z,OWNER,,"I'm going to ship a release which back-ports these two fixes: - https://github.com/simonw/datasette/issues/1958 - https://github.com/simonw/datasette/issues/1955",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1963/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1450303205,I_kwDOBm6k_c5Wcd7l,1891,1.0a0 release notes,9599,closed,0,,8658075,4,2022-11-15T19:58:20Z,2022-11-29T19:23:41Z,2022-11-29T19:23:41Z,OWNER,,"This release will mainly help preview the new Datasette write API: - #1850",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1891/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 735650864,MDU6SXNzdWU3MzU2NTA4NjQ=,194,3.0 release with some minor breaking changes,9599,closed,0,,6079500,3,2020-11-03T21:36:31Z,2020-11-08T17:19:35Z,2020-11-08T17:19:34Z,OWNER,,"While working on search (#192) I've spotted a few small changes I would like to make that would break backwards compatibility in minor ways, hence requiring a 3.x release. `db[table].search()` - I would like this to default to sorting by rank Also I'd like to free up the `-c` and `-f` options for other purposes from the standard output formats here: https://github.com/simonw/sqlite-utils/blob/43eae8b193d362f2b292df73e087ed6f10838144/sqlite_utils/cli.py#L48-L58 I'd like `-f` to be used to indicate a full-text search column during an insert and `-c` to indicate a column (so you can specify which columns you want to output).",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/194/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 802583450,MDU6SXNzdWU4MDI1ODM0NTA=,226,3.4 release is broken - includes a rogue line,9599,closed,0,,,0,2021-02-06T02:08:01Z,2021-02-06T02:10:26Z,2021-02-06T02:10:26Z,OWNER,,"I started seeing weird errors, caused by this line: https://github.com/simonw/sqlite-utils/blob/f8010ca78fed8c5fca6cde19658ec09fdd468420/sqlite_utils/cli.py#L1-L3 That was added by accident in 1b666f9315d4ea6bb332b2e75e48480c26100199 I'm surprised the tests didn't catch this!",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/226/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 435819321,MDU6SXNzdWU0MzU4MTkzMjE=,436,400 Error when trying to register new user via https://publish.datasettes.com/,317694,closed,0,,,1,2019-04-22T17:55:00Z,2021-01-04T20:15:42Z,2021-01-04T20:15:41Z,NONE,,"Behavior: When registering a new user via Zeit - confirmation is sent and screen acknowledges registered user... When clicking grant access the next screen is a white 400 error message. Replicated: Chrome and Firefox; 2 different email accounts",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/436/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 951817328,MDU6SXNzdWU5NTE4MTczMjg=,12,403 when getting token,285352,open,0,,,1,2021-07-23T18:43:26Z,2021-10-12T18:31:57Z,,NONE,,"I tried to use https://your-foursquare-oauth-token.glitch.me/ to get my Swarm auth token and got a 403 after I clicked the Allow button: ![image](https://user-images.githubusercontent.com/285352/126826478-60e53614-263d-40bb-9f1d-c1a676644eb0.png) I'm not sure if this is the right repo to report this in",205429375,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/12/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 1663399821,I_kwDOBm6k_c5jJXeN,2058,"500 ""attempt to write a readonly database"" error caused by ""PRAGMA schema_version""",9599,open,0,,,9,2023-04-11T23:57:50Z,2023-04-13T16:35:21Z,,OWNER,,"I've not been able to replicate this myself yet, but I've seen log files from a user affected by it. ``` File ""/usr/local/lib/python3.11/site-packages/datasette/views/base.py"", line 89, in dispatch_request await self.ds.refresh_schemas() File ""/usr/local/lib/python3.11/site-packages/datasette/app.py"", line 371, in refresh_schemas await self._refresh_schemas() File ""/usr/local/lib/python3.11/site-packages/datasette/app.py"", line 386, in _refresh_schemas schema_version = (await db.execute(""PRAGMA schema_version"")).first()[0] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File ""/usr/local/lib/python3.11/site-packages/datasette/database.py"", line 267, in execute results = await self.execute_fn(sql_operation_in_thread) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File ""/usr/local/lib/python3.11/site-packages/datasette/database.py"", line 213, in execute_fn return await asyncio.get_event_loop().run_in_executor( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File ""/usr/local/lib/python3.11/concurrent/futures/thread.py"", line 58, in run result = self.fn(*self.args, **self.kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File ""/usr/local/lib/python3.11/site-packages/datasette/database.py"", line 211, in in_thread return fn(conn) ^^^^^^^^ File ""/usr/local/lib/python3.11/site-packages/datasette/database.py"", line 237, in sql_operation_in_thread cursor.execute(sql, params if params is not None else {}) sqlite3.OperationalError: attempt to write a readonly database ``` That's running the official Datasette Docker image on https://fly.io/ - it's causing 500 errors on every page of their site.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2058/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 810397025,MDU6SXNzdWU4MTAzOTcwMjU=,1228,500 error caused by faceting if a column called `n` exists,7107523,closed,0,,,5,2021-02-17T17:41:20Z,2022-03-19T06:44:40Z,2022-03-19T01:38:04Z,NONE,,"I recently discovered `datasette` thanks to your great talk at FOSDEM and would like to use it for some projects. However, when trying to use it on databases created from some csv ot tsv files, I am sometimes getting this issue when going to http://127.0.0.1:8001/databasetest/databasetest and I don't exactly understand what it refers to. So far, I couldn't find anything relevant when reviewing the raw text files that could explain this issue, nor could I find something obvious between the files that generate this issue and those that don't. Does the error ring a bell and, if so, could you please point me to the right direction? ``` $ datasette databasetest.db INFO: Started server process [1408482] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://127.0.0.1:8001 (Press CTRL+C to quit) INFO: 127.0.0.1:56394 - ""GET / HTTP/1.1"" 200 OK INFO: 127.0.0.1:56394 - ""GET /-/static/app.css?4e362c HTTP/1.1"" 200 OK INFO: 127.0.0.1:56396 - ""GET /-/static-plugins/datasette_vega/main.2acbb312.css HTTP/1.1"" 200 OK INFO: 127.0.0.1:56398 - ""GET /-/static-plugins/datasette_vega/main.08f5d3d8.js HTTP/1.1"" 200 OK Traceback (most recent call last): File ""/home/kabouik/.local/lib/python3.7/site-packages/datasette/app.py"", line 1099, in route_path response = await view(request, send) File ""/home/kabouik/.local/lib/python3.7/site-packages/datasette/views/base.py"", line 147, in view request, **request.scope[""url_route""][""kwargs""] File ""/home/kabouik/.local/lib/python3.7/site-packages/datasette/views/base.py"", line 121, in dispatch_request return await handler(request, *args, **kwargs) File ""/home/kabouik/.local/lib/python3.7/site-packages/datasette/views/base.py"", line 260, in get request, database, hash, correct_hash_provided, **kwargs File ""/home/kabouik/.local/lib/python3.7/site-packages/datasette/views/base.py"", line 434, in view_get request, database, hash, **kwargs File ""/home/kabouik/.local/lib/python3.7/site-packages/datasette/views/table.py"", line 782, in data suggested_facets.extend(await facet.suggest()) File ""/home/kabouik/.local/lib/python3.7/site-packages/datasette/facets.py"", line 168, in suggest and any(r[""n""] > 1 for r in distinct_values) File ""/home/kabouik/.local/lib/python3.7/site-packages/datasette/facets.py"", line 168, in and any(r[""n""] > 1 for r in distinct_values) TypeError: '>' not supported between instances of 'str' and 'int' INFO: 127.0.0.1:56402 - ""GET /databasetest/databasetest HTTP/1.1"" 500 Internal Server Error INFO: 127.0.0.1:56402 - ""GET /-/static/app.css?4e362c HTTP/1.1"" 200 OK INFO: 127.0.0.1:56404 - ""GET / HTTP/1.1"" 200 OK INFO: 127.0.0.1:56404 - ""GET /-/static/app.css?4e362c HTTP/1.1"" 200 OK INFO: 127.0.0.1:56406 - ""GET /-/static-plugins/datasette_vega/main.2acbb312.css HTTP/1.1"" 200 OK INFO: 127.0.0.1:56408 - ""GET /-/static-plugins/datasette_vega/main.08f5d3d8.js HTTP/1.1"" 200 OK INFO: 127.0.0.1:56408 - ""GET /databasetest HTTP/1.1"" 200 OK INFO: 127.0.0.1:56408 - ""GET /-/static/app.css?4e362c HTTP/1.1"" 200 OK INFO: 127.0.0.1:56404 - ""GET /-/static-plugins/datasette_vega/main.2acbb312.css HTTP/1.1"" 200 OK INFO: 127.0.0.1:56406 - ""GET /-/static/codemirror-5.57.0.min.css HTTP/1.1"" 200 OK INFO: 127.0.0.1:56410 - ""GET /-/static-plugins/datasette_vega/main.08f5d3d8.js HTTP/1.1"" 200 OK INFO: 127.0.0.1:56414 - ""GET /-/static/codemirror-5.57.0-sql.min.js HTTP/1.1"" 200 OK INFO: 127.0.0.1:56412 - ""GET /-/static/codemirror-5.57.0.min.js HTTP/1.1"" 200 OK INFO: 127.0.0.1:56408 - ""GET /-/static/sql-formatter-2.3.3.min.js HTTP/1.1"" 200 OK INFO: 127.0.0.1:56408 - ""GET /databasetest?sql=select+*+from+databasetest HTTP/1.1"" 200 OK INFO: 127.0.0.1:56410 - ""GET /-/static/app.css?4e362c HTTP/1.1"" 200 OK INFO: 127.0.0.1:56408 - ""GET /-/static-plugins/datasette_vega/main.2acbb312.css HTTP/1.1"" 200 OK INFO: 127.0.0.1:56412 - ""GET /-/static/codemirror-5.57.0.min.css HTTP/1.1"" 200 OK INFO: 127.0.0.1:56404 - ""GET /-/static/sql-formatter-2.3.3.min.js HTTP/1.1"" 200 OK INFO: 127.0.0.1:56406 - ""GET /-/static/codemirror-5.57.0.min.js HTTP/1.1"" 200 OK INFO: 127.0.0.1:56414 - ""GET /-/static-plugins/datasette_vega/main.08f5d3d8.js HTTP/1.1"" 200 OK INFO: 127.0.0.1:56408 - ""GET /-/static/codemirror-5.57.0-sql.min.js HTTP/1.1"" 200 OK INFO: 127.0.0.1:56410 - ""GET /databasetest.json?sql=select+*+from+databasetest&_shape=array&_shape=array HTTP/1.1"" 200 OK ^CINFO: Shutting down INFO: Waiting for application shutdown. INFO: Application shutdown complete. INFO: Finished server process [1408482] ``` Note that there is no error if I go to http://127.0.0.1:8001/databasetest and then click on `Run SQL`.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1228/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1318907685,I_kwDOBm6k_c5OnO8l,1773,500 error if sorted by a column not in the ?_col= list,9599,closed,0,,8303187,4,2022-07-27T01:20:27Z,2022-08-14T16:06:25Z,2022-08-14T15:44:05Z,OWNER,,"For example: https://latest.datasette.io/fixtures/sortable?_sort_desc=sortable&_col=sortable_with_nulls That's `?_sort_desc=sortable&_col=sortable_with_nulls` ",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1773/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1308461063,I_kwDODFdgUs5N_YgH,74,500 error in github-to-sqlite demo,9599,closed,0,,,5,2022-07-18T19:39:32Z,2022-07-18T21:16:18Z,2022-07-18T21:14:22Z,MEMBER,,"https://github-to-sqlite.dogsheep.net/github/issue_comments throws a 500: > `cannot import name 'etree' from 'markdown.util' (/usr/local/lib/python3.8/site-packages/markdown/util.py)` https://console.cloud.google.com/run/detail/us-central1/github-to-sqlite/metrics?project=datasette-222320 suggests this started happening 3 days ago.",207052882,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/74/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 463492815,MDU6SXNzdWU0NjM0OTI4MTU=,534,500 error on m2m facet detection,9599,open,0,,,1,2019-07-03T00:42:42Z,2020-12-17T05:08:22Z,,OWNER,,"This may help debug: ``` diff --git a/datasette/facets.py b/datasette/facets.py index 76d73e5..07a4034 100644 --- a/datasette/facets.py +++ b/datasette/facets.py @@ -499,11 +499,14 @@ class ManyToManyFacet(Facet): ""outgoing"" ] if len(other_table_outgoing_foreign_keys) == 2: - destination_table = [ - t - for t in other_table_outgoing_foreign_keys - if t[""other_table""] != self.table - ][0][""other_table""] + try: + destination_table = [ + t + for t in other_table_outgoing_foreign_keys + if t[""other_table""] != self.table + ][0][""other_table""] + except IndexError: + import pdb; pdb.pm() # Only suggest if it's not selected already if (""_facet_m2m"", destination_table) in args: continue ```",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/534/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 1493404423,I_kwDOBm6k_c5ZA4sH,1948,500 error on permission debug page when testing actors with _r,9599,open,0,,,1,2022-12-13T05:22:03Z,2022-12-13T05:22:19Z,,OWNER,," The 500 error is silent unless you are looking at the DevTools network pane. ",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1948/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 752789159,MDU6SXNzdWU3NTI3ODkxNTk=,1113,500 error on row page if query against foreign keys hits time limit,9599,closed,0,,,1,2020-11-28T23:20:08Z,2020-11-29T02:40:01Z,2020-11-28T23:23:31Z,OWNER,,"This page exhibited the following error: https://data.catalyst.coop/ferc1/f1_respondent_id/145 `(OperationalError('interrupted'), 'select (select count(*) from f1_acb_epda where respondent_id=:id), (select count(*) from f1_accumdepr_prvsn where respondent_id=:id), (select count(*) from f1_accumdfrrdtaxcr where respondent_id=:id), (select count(*) from f1_adit_190_detail where respondent_id=:id), (select count(*) from f1_adit_190_notes where respondent_id=:id), (select count(*) from f1_adit_amrt_prop where respondent_id=:id), (select count(*) from f1_adit_other where respondent_id=:id), (select count(*) from f1_adit_other_prop where respondent_id=:id), (select count(*) from f1_allowances where respondent_id=:id), (select count(*) from f1_bal_sheet_cr where respondent_id=:id), (select count(*) from f1_capital_stock where respondent_id=:id), (select count(*) from f1_cash_flow where respondent_id=:id), (select count(*) from f1_cmmn_utlty_p_e where respondent_id=:id), (select count(*) from f1_comp_balance_db where respondent_id=:id), (select count(*) from f1_construction where respondent_id=:id), (select count(*) from f1_control_respdnt where respondent_id=:id), (select count(*) from f1_co_directors where respondent_id=:id), (select count(*) from f1_cptl_stk_expns where respondent_id=:id), (select count(*) from f1_csscslc_pcsircs where respondent_id=:id), (select count(*) from f1_dacs_epda where respondent_id=:id), (select count(*) from f1_dscnt_cptl_stk where respondent_id=:id), (select count(*) from f1_edcfu_epda where respondent_id=:id), (select count(*) from f1_elctrc_erg_acct where respondent_id=:id), (select count(*) from f1_elctrc_oper_rev where respondent_id=:id), (select count(*) from f1_elc_oper_rev_nb where respondent_id=:id), (select count(*) from f1_elc_op_mnt_expn where respondent_id=:id), (select count(*) from f1_electric where respondent_id=:id), (select count(*) from f1_envrnmntl_expns where respondent_id=:id), (select count(*) from f1_envrnmntl_fclty where respondent_id=:id), (select count(*) from f1_fuel where respondent_id=:id), (select count(*) from f1_general_info where respondent_id=:id), (select count(*) from f1_gnrt_plant where respondent_id=:id), (select count(*) from f1_important_chg where respondent_id=:id), (select count(*) from f1_incm_stmnt_2 where respondent_id=:id), (select count(*) from f1_income_stmnt where respondent_id=:id), (select count(*) from f1_miscgen_expnelc where respondent_id=:id), (select count(*) from f1_misc_dfrrd_dr where respondent_id=:id), (select count(*) from f1_mthly_peak_otpt where respondent_id=:id), (select count(*) from f1_mtrl_spply where respondent_id=:id), (select count(*) from f1_nbr_elc_deptemp where respondent_id=:id), (select count(*) from f1_nonutility_prop where respondent_id=:id), (select count(*) from f1_nuclear_fuel where respondent_id=:id), (select count(*) from f1_officers_co where respondent_id=:id), (select count(*) from f1_othr_dfrrd_cr where respondent_id=:id), (select count(*) from f1_othr_pd_in_cptl where respondent_id=:id), (select count(*) from f1_othr_reg_assets where respondent_id=:id), (select count(*) from f1_othr_reg_liab where respondent_id=:id), (select count(*) from f1_overhead where respondent_id=:id), (select count(*) from f1_pccidica where respondent_id=:id), (select count(*) from f1_plant_in_srvce where respondent_id=:id), (select count(*) from f1_pumped_storage where respondent_id=:id), (select count(*) from f1_purchased_pwr where respondent_id=:id), (select count(*) from f1_reconrpt_netinc where respondent_id=:id), (select count(*) from f1_reg_comm_expn where respondent_id=:id), (select count(*) from f1_respdnt_control where respondent_id=:id), (select count(*) from f1_retained_erng where respondent_id=:id), (select count(*) from f1_r_d_demo_actvty where respondent_id=:id), (select count(*) from f1_sales_by_sched where respondent_id=:id), (select count(*) from f1_sale_for_resale where respondent_id=:id), (select count(*) from f1_sbsdry_totals where respondent_id=:id), (select count(*) from f1_schedules_list where respondent_id=:id), (select count(*) from f1_security_holder where respondent_id=:id), (select count(*) from f1_slry_wg_dstrbtn where respondent_id=:id), (select count(*) from f1_substations where respondent_id=:id), (select count(*) from f1_taxacc_ppchrgyr where respondent_id=:id), (select count(*) from f1_unrcvrd_cost where respondent_id=:id), (select count(*) from f1_utltyplnt_smmry where respondent_id=:id), (select count(*) from f1_work where respondent_id=:id), (select count(*) from f1_xmssn_adds where respondent_id=:id), (select count(*) from f1_xmssn_elc_bothr where respondent_id=:id), (select count(*) from f1_xmssn_elc_fothr where respondent_id=:id), (select count(*) from f1_xmssn_line where respondent_id=:id), (select count(*) from f1_xtraordnry_loss where respondent_id=:id), (select count(*) from f1_audit_log where respondent_id=:id), (select count(*) from f1_load_file_names where respondent_id=:id), (select count(*) from f1_privilege where respondent_id=:id), (select count(*) from f1_sys_error_log where respondent_id=:id), (select count(*) from f1_hydro where respondent_id=:id), (select count(*) from f1_ident_attsttn where respondent_id=:id), (select count(*) from f1_steam where respondent_id=:id), (select count(*) from f1_leased where respondent_id=:id), (select count(*) from f1_sbsdry_detail where respondent_id=:id), (select count(*) from f1_plant where respondent_id=:id), (select count(*) from f1_long_term_debt where respondent_id=:id), (select count(*) from f1_106_2009 where respondent_id=:id), (select count(*) from f1_106a_2009 where respondent_id=:id), (select count(*) from f1_106b_2009 where respondent_id=:id), (select count(*) from f1_208_elc_dep where respondent_id=:id), (select count(*) from f1_231_trn_stdycst where respondent_id=:id), (select count(*) from f1_324_elc_expns where respondent_id=:id), (select count(*) from f1_325_elc_cust where respondent_id=:id), (select count(*) from f1_331_transiso where respondent_id=:id), (select count(*) from f1_338_dep_depl where respondent_id=:id), (select count(*) from f1_397_isorto_stl where respondent_id=:id), (select count(*) from f1_398_ancl_ps where respondent_id=:id), (select count(*) from f1_399_mth_peak where respondent_id=:id), (select count(*) from f1_400_sys_peak where respondent_id=:id), (select count(*) from f1_400a_iso_peak where respondent_id=:id), (select count(*) from f1_429_trans_aff where respondent_id=:id), (select count(*) from f1_allowances_nox where respondent_id=:id), (select count(*) from f1_cmpinc_hedge_a where respondent_id=:id), (select count(*) from f1_cmpinc_hedge where respondent_id=:id), (select count(*) from f1_email where respondent_id=:id), (select count(*) from f1_rg_trn_srv_rev where respondent_id=:id), (select count(*) from f1_s0_filing_log where respondent_id=:id), (select count(*) from f1_security where respondent_id=:id)', {'id': '145'})`",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1113/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 292011379,MDU6SXNzdWUyOTIwMTEzNzk=,184,500 from missing table name,222245,closed,0,,,4,2018-01-26T19:46:45Z,2019-05-21T16:17:29Z,2018-04-13T18:18:59Z,NONE,,"https://github.com/simonw/datasette/blob/56623e48da5412b25fb39cc26b9c743b684dd968/datasette/app.py#L517-L519 throws an error if it gets an empty list back. Simplest solution is to write a helper func that just says ```python result = list(await self.execute(name, sql, params) if result: return result[0][0] ``` and use it anywhere `[0][0]` is now.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/184/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 517451234,MDU6SXNzdWU1MTc0NTEyMzQ=,615,?_col= and ?_nocol= support for toggling columns on table view,9599,closed,0,,,16,2019-11-04T22:55:41Z,2021-05-27T04:26:10Z,2021-05-27T04:17:44Z,OWNER,,Split off from #292 (I guess this is a re-opening of #312).,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/615/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 893890496,MDU6SXNzdWU4OTM4OTA0OTY=,1332,?_facet_size=X to increase number of facets results on the page,192568,closed,0,,,5,2021-05-18T02:40:16Z,2021-05-27T16:13:07Z,2021-05-23T00:34:37Z,CONTRIBUTOR,,"Is there a way to add a parameter to the URL to modify default_facet_size? LIkewise, a way to produce a link on the three dots to expand to all items (or match previous number of items, or add x more)? ",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1332/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 269731374,MDU6SXNzdWUyNjk3MzEzNzQ=,44,?_group_count=country - return counts by specific column(s),9599,closed,0,,,7,2017-10-30T19:50:32Z,2018-04-26T15:09:58Z,2018-04-26T15:09:58Z,OWNER,,"Imagine if this: https://stateless-datasets-jykibytogk.now.sh/flights-07d1283/airports.jsono?country__contains=gu&_group_count=country Turned into this: https://stateless-datasets-jykibytogk.now.sh/flights-07d1283?sql=select%20country,%20count(*)%20as%20group_count_country%20from%20airports%20where%20country%20like%20%27%gu%%27%20group%20by%20country%20order%20by%20group_count_country%20desc This would involve introducing a new precedent of query string arguments that start with an _ having special meanings. While we're at it, could try adding _fields=x,y,z Tasks: - [x] Get initial version working - [ ] Refactor code to not just ""pretend to be a view"" - [ ] Get foreign key relationships expanded",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/44/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 445003029,MDU6SXNzdWU0NDUwMDMwMjk=,471,?_hash=1 and --config hash_urls:1 should only work for immutable databases,9599,closed,0,,4305096,1,2019-05-16T14:54:25Z,2019-05-16T15:11:03Z,2019-05-16T15:11:03Z,OWNER,,Split from #419.,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/471/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 275089535,MDU6SXNzdWUyNzUwODk1MzU=,121,?_json=foo&_json=bar query string argument ,9599,closed,0,,,4,2017-11-18T16:09:55Z,2018-05-31T13:48:12Z,2018-05-28T18:11:51Z,OWNER,,"Causes the specified columns in the output to be treated as JSON, and returned deserialized in the .json or .jsono response. This will be particularly powerful when combined with https://sqlite.org/json1.html",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/121/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 904071938,MDU6SXNzdWU5MDQwNzE5Mzg=,1345,?_nocol= does not interact well with default facets,9599,closed,0,,,7,2021-05-27T18:39:55Z,2021-05-31T02:40:44Z,2021-05-31T02:31:21Z,OWNER,,"Clicking ""Hide this column"" on `fips` on https://covid-19.datasettes.com/covid/ny_times_us_counties shows this error: https://covid-19.datasettes.com/covid/ny_times_us_counties?_nocol=fips > ## Invalid SQL > no such column: fips The reason is that https://covid-19.datasettes.com/-/metadata sets up the following: ```json ""ny_times_us_counties"": { ""sort_desc"": ""date"", ""facets"": [ ""state"", ""county"", ""fips"" ], ``` It's setting `fips` as a default facet, which breaks if you attempt to remove the column using `?_nocol`.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1345/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 908446997,MDU6SXNzdWU5MDg0NDY5OTc=,1353,?_nocount=1 for opting out of table counts,9599,closed,0,,,2,2021-06-01T15:53:27Z,2021-06-01T16:18:54Z,2021-06-01T16:17:04Z,OWNER,,"Running a trace against a CSV streaming export with the new `_trace=1` feature from #1351 shows that the following code is executing a `select count(*) from table` for every page of results returned: https://github.com/simonw/datasette/blob/d1d06ace49606da790a765689b4fbffa4c6deecb/datasette/views/table.py#L700-L705 This is inefficient - a new `?_nocount=1` option would let us disable this count in the same way as #1349: https://github.com/simonw/datasette/blob/d1d06ace49606da790a765689b4fbffa4c6deecb/datasette/views/base.py#L264-L276 ",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1353/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 906977719,MDU6SXNzdWU5MDY5Nzc3MTk=,1350,?_nofacets=1 query string argument for disabling facets and suggested facets,9599,closed,0,,,2,2021-05-31T02:22:29Z,2021-06-01T16:19:38Z,2021-05-31T02:39:18Z,OWNER,,"This is needed as an internal option for #1349. `datasette-graphql` can benefit from this too - maybe can even use it so that if you pass `?_shape=array` it gets automatically added, fixing #263.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1350/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 812704869,MDU6SXNzdWU4MTI3MDQ4Njk=,1237,?_pretty=1 option for pretty-printing JSON output,9599,open,0,,3268330,1,2021-02-20T20:54:40Z,2021-11-16T18:28:33Z,,OWNER,,Suggested by @frankieroberto in https://github.com/simonw/datasette/issues/782#issuecomment-782746755,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1237/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 275179724,MDU6SXNzdWUyNzUxNzk3MjQ=,135,?_search=x should work if used directly against a FTS virtual table,9599,closed,0,,2949431,3,2017-11-19T18:17:53Z,2017-12-07T04:54:41Z,2017-12-07T04:54:41Z,OWNER,,e.g. https://sf-trees.now.sh/sf-trees-ebc2ad9/Street_Tree_List_fts?_search=grove should work,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/135/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 426722204,MDU6SXNzdWU0MjY3MjIyMDQ=,423,?_search_col=X not reflected correctly in the UI,9599,open,0,,,0,2019-03-28T21:48:19Z,2020-11-03T19:01:59Z,,OWNER,,"e.g. https://latest.datasette.io/fixtures/searchable?_search_text1=barry ![2019-03-28 at 2 47 PM](https://user-images.githubusercontent.com/9599/55195035-84ebb800-5168-11e9-910b-fc9868bcd93e.png) ",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/423/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 568091133,MDU6SXNzdWU1NjgwOTExMzM=,676,?_searchmode=raw option for running FTS searches without escaping characters,58088336,closed,0,,,9,2020-02-20T06:56:57Z,2020-02-25T05:57:24Z,2020-02-25T05:56:04Z,NONE,,"After the version 0.34. I am not able to use the wildchar in the _search option( or the full text search). It will not return any result unless I specify the whole word for text search. If I use 'match :search || ""*"" ' in the sql statement then it will work as expected.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/676/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 610342575,MDU6SXNzdWU2MTAzNDI1NzU=,748,?_searchmode=raw should be documented on full-text search page,9599,closed,0,,,0,2020-04-30T19:50:06Z,2020-04-30T21:06:12Z,2020-04-30T21:06:12Z,OWNER,,"It's currently documented here: https://datasette.readthedocs.io/en/stable/json_api.html#special-table-arguments But it should also be described here: https://datasette.readthedocs.io/en/stable/full_text_search.html#the-table-view-api",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/748/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 319358200,MDU6SXNzdWUzMTkzNTgyMDA=,245,?_shape=array option,9599,closed,0,,,1,2018-05-01T23:11:07Z,2018-05-03T14:14:33Z,2018-05-02T00:12:20Z,OWNER,,"Some tools (`pandas.DataFrame(...)` for example) are happiest when you give them a raw array of JSON objects. `?_shape=array` should do just that While I'm at it, rename the default `?_shape=lists` to instead be called `?shape=arrays` And validate that `_shape` is a valid option And have `?_shape=object` return the object at the root level rather than nested in `.rows` to better match the behavior of `?_shape=array`",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/245/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 326617744,MDU6SXNzdWUzMjY2MTc3NDQ=,287,?_shape=arrayfirst,9599,closed,0,,,1,2018-05-25T18:11:03Z,2018-05-27T00:32:53Z,2018-05-27T00:32:29Z,OWNER,,Return an array of single items (the first item in each row returned from the SQL query).,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/287/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 788447787,MDU6SXNzdWU3ODg0NDc3ODc=,1194,?_size= argument is not persisted by hidden form fields in the table filters,9599,closed,0,,6346396,3,2021-01-18T17:41:52Z,2021-01-25T03:10:23Z,2021-01-25T03:10:23Z,OWNER,,"Click ""Apply"" on https://covid-19.datasettes.com/covid/ny_times_us_counties?_size=1000&county__exact=San+Francisco&state__exact=California&_sort_desc=date#g.mark=line&g.x_column=date&g.x_type=temporal&g.y_column=cases&g.y_type=quantitative and the `?_size=1000` parameter from the URL will no longer apply on the reloaded page.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1194/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 792890765,MDU6SXNzdWU3OTI4OTA3NjU=,1200,?_size=10 option for the arbitrary query page would be useful,9599,open,0,,,2,2021-01-24T20:55:35Z,2021-02-11T03:13:59Z,,OWNER,,"https://latest.datasette.io/fixtures?sql=select+*+from+compound_three_primary_keys&_size=10 - `_size=10` does not do anything at the moment. It would be useful if it did. Would also be good if it persisted in a hidden form field.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1200/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 320090329,MDU6SXNzdWUzMjAwOTAzMjk=,249,?_size=max argument ,9599,closed,0,,,1,2018-05-03T21:42:04Z,2018-05-04T18:26:30Z,2018-05-04T18:05:04Z,OWNER,,"For plugins that want to load the most data allowable, having `?_size=max` would be useful.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/249/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 995098231,MDU6SXNzdWU5OTUwOTgyMzE=,1470,?_sort=rowid with _next= returns error,19851673,closed,0,,,4,2021-09-13T16:36:15Z,2021-10-18T19:30:15Z,2021-10-10T01:15:03Z,NONE,,"For example: - Go to https://cryptics.eigenfoo.xyz/clues/clues?_next=100 (this is the second page of results in a Datasette site) - Search anything using the FTS search bar. For example, searching for `hello` will take you to https://cryptics.eigenfoo.xyz/clues/clues?_search=hello&_sort=rowid&_next=100 - A `500 Error: list index out of range` is raised. This is because the search URL includes the `&_next=100` UTM parameter, carried over from where the FTS search was run. However, there isn't a second page in the search results, so a `list index out of range` error is raised. You can confirm that removing this UTM parameter from the URL returns the appropriate search results. The FTS search request should strip any `_next` UTM parameter. --- ```bash datasette, version 0.58.1 sqlite-utils, version 3.17 ```",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1470/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1216619276,I_kwDOBm6k_c5IhCMM,1724,?_trace=1 doesn't work on Global Power Plants demo,9599,closed,0,,,3,2022-04-27T00:15:02Z,2022-04-27T06:15:14Z,2022-04-27T00:18:30Z,OWNER,,"https://global-power-plants.datasettes.com/global-power-plants/global-power-plants?_trace=1 is not showing the trace JSON at the bottom of the page. Confirmed that `trace_debug` is `true` on https://global-power-plants.datasettes.com/-/settings Possibly related: - https://github.com/simonw/datasette-total-page-time/issues/1",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1724/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1237586379,I_kwDOBm6k_c5JxBHL,1742,?_trace=1 fails with datasette-geojson for some reason,9599,open,0,,,4,2022-05-16T19:06:05Z,2022-05-16T19:42:13Z,,OWNER,,view-source:https://calands.datasettes.com/calands/CPAD_2020a_SuperUnits.geojson?_sort=id&id__exact=4&_labels=on&_trace=1 is showing me a blank page.,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1742/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 326768188,MDU6SXNzdWUzMjY3NjgxODg=,289,?_ttl= parameter to control caching,9599,closed,0,,,3,2018-05-26T21:22:55Z,2018-05-26T22:22:47Z,2018-05-26T22:17:48Z,OWNER,,"This would allow clients to specify the max-age caching header that should be returned with the query. Most important this will allow caching to be completely urned off for specific queries using `?_ttl=0`. Sending 0 should cause a `Cache-Control: no-cache` header to be returned.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/289/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 432636432,MDU6SXNzdWU0MzI2MzY0MzI=,429,?_where=sql-fragment parameter for table views,9599,closed,0,,,7,2019-04-12T15:58:51Z,2019-04-15T10:48:01Z,2019-04-13T01:37:25Z,OWNER,,"Only available if arbitrary SQL is enabled (the default). `?_where=id in (1,2,3)&_where=id in (select tag_id from tags)` Allows any table (or view) page to have arbitrary additional `extra_where` clauses defined using the URL! This would be extremely useful for building JavaScript applications against the Datasette API that only need on extra tiny bit of SQL but still want to benefit from other table view features like faceting. Would be nice if this could take `:named` parameters and have them filled in via querystring as well.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/429/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 433297989,MDU6SXNzdWU0MzMyOTc5ODk=,433,"?column__in=value1,value2,value3 filter",9599,closed,0,,,0,2019-04-15T13:58:24Z,2019-04-15T23:00:20Z,2019-04-15T23:00:20Z,OWNER,,"Support for the SQL `where column in (...)` construct, inspired by the new design for facet configuration in #427 `?column__in=value1,value2,value3` will map to `where column in (""value1"", ""value2"", ""value3"")` If comma separation won't work (because the values themselves contain commas) you can do this instead: `?column__in=[""value1"",""value2"",""value3,with-comma""]` See also #288",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/433/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 657572753,MDU6SXNzdWU2NTc1NzI3NTM=,894,?sort=colname~numeric to sort by by column cast to real,9599,open,0,,,21,2020-07-15T18:47:48Z,2021-08-20T02:07:53Z,,OWNER,,"If a text column actually contains numbers, being able to ""sort by column, treated as numeric"" would be really useful. Probably depends on column actions enabled by #690",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/894/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 737855731,MDU6SXNzdWU3Mzc4NTU3MzE=,199,"@db.register_function(..., replace=False) to avoid double-registering custom functions",9599,closed,0,,,1,2020-11-06T15:39:21Z,2020-11-06T18:30:44Z,2020-11-06T18:30:44Z,OWNER,,"I'd like a mechanism to optionally avoid registering a custom function if it has already been registered. SQLite doesn't seem to offer a way to introspect registered custom functions so I'll need to track what has already been registered in `sqlite-utils` instead. > Should I register the custom `rank_bm25` SQLite function for every connection, or should I register it against the connection just the first time the user attempts an FTS4 search? I think I'd rather register it only if it is needed. _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/198#issuecomment-723145383_",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/199/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1345561209,I_kwDOBm6k_c5QM6J5,1790,A better HTML title for canned query pages,9599,open,0,,,0,2022-08-21T18:27:46Z,2022-08-21T18:27:46Z,,OWNER,,"https://scotrail.datasette.io/scotrail/assemble_sentence?terms=This+train+is+formed+of%2Cbomb+which Current title is: `scotrail: with phrases as ( select key, value from json_each('["' || replace(:terms, ',', '","') || '"]')),matches as (select phrases.key, phrases.value, ( select File from announcements where announcements.Transcription like '%' || trim(phrases.value) || '%' order by length(announcements.Transcription) limit 1 ) as Filefrom phrases),results as ( select key, announcements.Transcription, announcements.mp3 from announcements join matches on announcements.File = matches.File order by key)select 'Combined sentence:' as mp3, group_concat(Transcription, ' ') as Transcription, -1 as keyfrom results unionselect mp3, Transcription, keyfrom resultsorder by key` I think a better title would be: `scotrail: assemble_sentence, terms = This train is formed of,bomb which`",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1790/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 705995722,MDU6SXNzdWU3MDU5OTU3MjI=,162,A decorator for registering custom SQL functions,9599,closed,0,,,2,2020-09-22T00:18:32Z,2020-09-22T00:40:44Z,2020-09-22T00:32:17Z,OWNER,,"Syntactic sugar for `db.conn.create_function` - it would work something like this: ```python db = sqlite_utils.Database(""mydb.db"") @db.register_function def scramble(text): chars = list(text) random.shuffle(chars) return """".join(chars) ``` The decorator would inspect the function to find its name and arity (number of arguments). Having run the above you could then do: ```python db.execute(""select scramble('hello')"").fetchall() ```",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/162/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 794554881,MDU6SXNzdWU3OTQ1NTQ4ODE=,1208,A lot of open(file) functions are used without a context manager thus producing ResourceWarning: unclosed file <_io.TextIOWrapper,4488943,closed,0,,,2,2021-01-26T20:56:28Z,2021-03-11T16:15:49Z,2021-03-11T16:15:49Z,CONTRIBUTOR,,"Your code is full of open files that are never closed, especially when you deal with reading/writing json/yaml files. If you run python with warnings enabled this problem becomes evident. This probably contributes to some memory leaks in long running datasettes if the GC will not 'collect' those resources properly. This is easily fixed by using a context manager instead of just using open: ```python with open('some_file', 'w') as opened_file: opened_file.write('string') ``` In some newer parts of the code you use Path objects 'read_text' and 'write_text' functions which close the file properly and are prefered in some cases. If you want I can create a PR for all places i found this pattern in. Bellow is a fraction of places where i found a ResourceWarning: ```python update-docs-help.py: 20 actual = actual.replace(""Usage: cli "", ""Usage: datasette "") 21: open(docs_path / filename, ""w"").write(actual) 22 datasette\app.py: 210 ): 211: inspect_data = json.load((config_dir / ""inspect-data.json"").open()) 212 if immutables is None: 266 if config_dir and (config_dir / ""settings.json"").exists() and not config: 267: config = json.load((config_dir / ""settings.json"").open()) 268 self._settings = dict(DEFAULT_SETTINGS, **(config or {})) 445 self._app_css_hash = hashlib.sha1( 446: open(os.path.join(str(app_root), ""datasette/static/app.css"")) 447 .read() datasette\cli.py: 130 else: 131: out = open(inspect_file, ""w"") 132 loop = asyncio.get_event_loop() 459 if inspect_file: 460: inspect_data = json.load(open(inspect_file)) 461 ``` ",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1208/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 728895233,MDU6SXNzdWU3Mjg4OTUyMzM=,1047,A new section in the docs about how Datasette handles BLOB columns,9599,closed,0,,6026070,1,2020-10-24T23:01:02Z,2020-10-31T22:11:25Z,2020-10-31T21:38:05Z,OWNER,,"Split from #1040, refs #1036.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1047/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 277589569,MDU6SXNzdWUyNzc1ODk1Njk=,155,A primary key column that has foreign key restriction associated won't rendering label column,388154,closed,0,,2949431,4,2017-11-29T00:40:02Z,2017-12-07T05:39:53Z,2017-12-07T05:39:53Z,NONE,,,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/155/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1108235694,I_kwDOBm6k_c5CDlWu,1603,A proper favicon,9599,closed,0,,3268330,19,2022-01-19T15:24:55Z,2022-03-19T04:04:49Z,2022-01-20T06:07:31Z,OWNER,,"Tips here: https://adamj.eu/tech/2022/01/18/how-to-add-a-favicon-to-your-django-site/ - I think a PNG served at `/favicon.ico` is the best option, since safari doesn't support SVG yet. Relevant code: https://github.com/simonw/datasette/blob/cb29119db9115b1f40de2fb45263ed77e3bfbb3e/datasette/app.py#L182-L183 I can reuse the icon for https://datasette.io/desktop",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1603/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1071531082,I_kwDOCGYnMM4_3kRK,349,A way of creating indexes on newly created tables,9599,open,0,,,3,2021-12-05T18:56:12Z,2021-12-07T01:04:37Z,,OWNER,,"I'm writing code for https://github.com/simonw/git-history/issues/33 that creates a table inside a loop: ```python item_pk = db[item_table].lookup( {""_item_id"": item_id}, item_to_insert, column_order=(""_id"", ""_item_id""), pk=""_id"", ) ``` I need to look things up by `_item_id` on this table, which means I need an index on that column (the table can get very big). But there's no mechanism in SQLite utils to detect if the table was created for the first time and add an index to it. And I don't want to run `CREATE INDEX IF NOT EXISTS` every time through the loop. This should work like the `foreign_keys=` mechanism. ",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/349/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 1469044738,I_kwDOBm6k_c5Xj9gC,1918,API explorer should list mutable databases first,9599,closed,0,,7867486,1,2022-11-30T04:53:33Z,2022-11-30T05:22:07Z,2022-11-30T05:07:56Z,OWNER,,"https://latest.datasette.io/-/api hides `ephemeral` down at the bottom, would be more interesting if it was at the top. Related: - #1915 ",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1918/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1447439985,I_kwDOBm6k_c5WRi5x,1888,API explorer should take immutability into account,9599,closed,0,,8658075,1,2022-11-14T06:00:14Z,2022-11-15T19:59:10Z,2022-11-14T06:04:48Z,OWNER,,"Refs: - #1871 I noticed the API explorer doesn't show any links on https://latest-1-0-dev.datasette.io/-/api because the `fixtures` database is immutable. It should still show read examples there.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1888/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1427293909,I_kwDOBm6k_c5VEsbV,1871,API explorer tool,9599,closed,0,,8658075,24,2022-10-28T13:49:11Z,2022-11-15T19:59:05Z,2022-11-14T04:59:59Z,OWNER,,The API will be much easier to develop if there's a page that helps you execute JSON POSTs against it.,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1871/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1426001541,I_kwDOBm6k_c5U_w6F,1866,API for bulk inserting records into a table,9599,closed,0,,8658075,12,2022-10-27T17:19:25Z,2022-11-15T19:59:34Z,2022-10-30T06:04:07Z,OWNER,,"Similar to https://github.com/simonw/datasette-insert/blob/0.8/README.md#inserting-data-and-creating-tables I expect this to become by far the most common way that data gets into a Datasette instance - more so than the individual row API in: - #1851 ",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1866/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1429030341,I_kwDOBm6k_c5VLUXF,1874,API to drop a table,9599,closed,0,,8658075,4,2022-10-30T21:55:11Z,2022-11-15T19:59:53Z,2022-11-14T05:45:06Z,OWNER,,"`POST /db/table/-/drop` Require `drop-table` permission.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1874/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1421544654,I_kwDOBm6k_c5UuwzO,1851,API to insert a single record into an existing table,9599,closed,0,,8658075,22,2022-10-24T22:24:21Z,2022-11-15T19:59:18Z,2022-10-28T00:59:25Z,OWNER,,Controlled by a new `insert-row` permission.,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1851/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1805076818,I_kwDOBm6k_c5rl0lS,2102,API tokens with view-table but not view-database/view-instance cannot access the table,9599,closed,0,9599,,20,2023-07-14T15:34:27Z,2023-08-29T16:32:36Z,2023-08-29T16:32:35Z,OWNER,,"> Spotted a problem while working on this: if you grant a token access to view table for a specific table but don't also grant view database and view instance permissions, that token is useless. > > This was a deliberate design decision in Datasette - it's documented on https://docs.datasette.io/en/1.0a2/authentication.html#access-permissions-in-metadata > >> If a user cannot access a specific database, they will not be able to access tables, views or queries within that database. If a user cannot access the instance they will not be able to access any of the databases, tables, views or queries. > > I'm now second-guessing if this was a good decision. _Originally posted by @simonw in https://github.com/simonw/datasette-auth-tokens/issues/7#issuecomment-1636031702_ ",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2102/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 582517965,MDU6SXNzdWU1ODI1MTc5NjU=,698,Ability for a canned query to write to the database,9599,closed,0,,5512395,26,2020-03-16T18:31:59Z,2020-06-06T19:43:49Z,2020-06-06T19:43:48Z,OWNER,,"Canned queries are currently read-only: https://datasette.readthedocs.io/en/0.38/sql_queries.html#canned-queries Add a `""write"": true` option to their definition in `metadata.json` which turns them into queries that are submitted via POST and send their queries to the write queue. Then they can be used as a really quick way to define a writable interface and JSON API!",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/698/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 315517578,MDU6SXNzdWUzMTU1MTc1Nzg=,224,Ability for plugins to bundle templates,9599,closed,0,,,1,2018-04-18T14:57:53Z,2018-04-19T05:50:36Z,2018-04-19T05:50:36Z,OWNER,,"Plugins should be able to bundle templates. The Datasette template loader should then consult those plugins first when loading a template. Jinja2 has a `PackageLoader` class that can help with this: http://jinja.pocoo.org/docs/2.10/api/#jinja2.PackageLoader Refs #14",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/224/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 787098345,MDU6SXNzdWU3ODcwOTgzNDU=,1191,Ability for plugins to collaborate when adding extra HTML to blocks in default templates,9599,open,0,,3268330,12,2021-01-15T18:18:51Z,2023-09-18T06:55:52Z,,OWNER,,"Sometimes a plugin may want to add content to an existing default template - for example `datasette-search-all` adds a new search box at the top of `index.html`. I also want `datasette-upload-csvs` to add a CTA on the `database.html` page: https://github.com/simonw/datasette-upload-csvs/issues/18 Currently plugins can do this by providing a new version of the `index.html` template - but if multiple plugins try to do that only one of them will succeed. It would be better if there were known areas of those templates which plugins could add additional content to, such that multiple plugins can use the same spot.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1191/reactions"", ""total_count"": 4, ""+1"": 4, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 314506446,MDU6SXNzdWUzMTQ1MDY0NDY=,214,Ability for plugins to define extra JavaScript and CSS,9599,closed,0,,,6,2018-04-16T05:29:34Z,2020-09-30T20:36:11Z,2018-04-18T03:13:03Z,OWNER,,"This can hook in to the existing `extra_css_urls` and `extra_js_urls` mechanism: https://github.com/simonw/datasette/blob/b2955d9065ea019500c7d072bcd9d49d1967f051/datasette/app.py#L304-L305 The plugins should be able to bundle their own assets though, so it will also have to integrate with the `/static/` static mounts mechanism somehow: https://github.com/simonw/datasette/blob/b2955d9065ea019500c7d072bcd9d49d1967f051/datasette/app.py#L1255-L1257 Refs #14",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/214/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 442330564,MDU6SXNzdWU0NDIzMzA1NjQ=,457,"Ability to ""publish cloudrun"" with no user input",9599,closed,0,,,2,2019-05-09T16:42:51Z,2019-05-09T19:41:31Z,2019-05-09T16:45:08Z,OWNER,,"If you attempt to deploy a new version of a cloudrun deployment, the script currently pauses and asks for user input for the service name like this: ```77d4d7de-3dfc-4acc-9a23-efe16230f318 2019-05-09T15:01:48+00:00 52S gs://datasette-222320_cloudbuild/source/1557414063.1-3a82df8096e9434b93511b0588d8d155.tgz gcr.io/datasette-222320/sf-trees (+1 more) SUCCESS Service name: (sf-trees): USER INPUT REQUIRED HERE Deploying container to Cloud Run service [sf-trees] in project [datasette-222320] region [us-central1] ✓ Deploying... Done. ✓ Creating Revision... ✓ Routing traffic... ✓ Setting IAM Policy... ``` This is incompatible with running under CI.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/457/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 413857257,MDU6SXNzdWU0MTM4NTcyNTc=,15,Ability to add columns to tables,9599,closed,0,,,0,2019-02-24T19:20:51Z,2019-02-24T20:04:40Z,2019-02-24T20:04:40Z,OWNER,,"Makes sense to do this before foreign keys in #2 Python: db[""table""].add_column(""new_column"", int) CLI: $ sqlite-utils add-column table new_column INTEGER ",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/15/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 448189298,MDU6SXNzdWU0NDgxODkyOTg=,486,Ability to add extra routes and related templates,2181410,closed,0,,,2,2019-05-24T14:04:25Z,2019-05-24T14:43:28Z,2019-05-24T14:43:09Z,NONE,,"Hi Simon Thank for an excellent job! Datasette is such an obviously good idea (once you have that idea!) and so well done. The only thing that I miss, is the ability to add extras routes (with associated jinja2-templates). For most of the datasets, that I would like to publish, I would also like at least a page, that describes the data (semantics, provenance, biases...) and a page explaining our cookie- and privacy-policies (which would allows us to use something like Goggle Analytics). ",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/486/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 312620566,MDU6SXNzdWUzMTI2MjA1NjY=,199,Ability to apply sort on mobile in portrait mode,9599,closed,0,,,4,2018-04-09T17:35:04Z,2018-04-10T00:37:53Z,2018-04-10T00:34:38Z,OWNER,,"Missed this in #189... on mobile in portrait mode we hide the column headers, which means you can't click them to sort! You can sort in landscape mode at least. Need to come up with an alternative sort UI for portrait on mobile.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/199/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 413779210,MDU6SXNzdWU0MTM3NzkyMTA=,13,Ability to automatically create IDs from content hash of row,9599,closed,0,,,1,2019-02-24T04:07:08Z,2019-02-24T04:36:48Z,2019-02-24T04:36:48Z,OWNER,,"Sometimes when you are importing data the underlying source provides records without IDs that can be uniquely identified by their contents. A utility mechanism for calculating a sha1 hash of the contents and using that as a unique ID would be useful.",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/13/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 278208011,MDU6SXNzdWUyNzgyMDgwMTE=,160,Ability to bundle and serve additional static files,9599,closed,0,,2949431,8,2017-11-30T17:37:51Z,2019-02-02T00:58:20Z,2017-12-09T18:29:11Z,OWNER,,"Since we now have custom templates, we should support including custom static files with them as well. Maybe something like this: datasette mydb.db --template-dir=templates/ --static-dir=static/ This should also be supported by datasette publish - see also #157 ",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/160/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 309047460,MDU6SXNzdWUzMDkwNDc0NjA=,188,Ability to bundle metadata and templates inside the SQLite file,9599,open,0,,,4,2018-03-27T16:42:07Z,2020-12-04T17:18:34Z,,OWNER,,"One of the nicest qualities of SQLite as a data format is that you get a single file which you can then backup or share with other people. Datasette breaks this a little once you start including custom metadata.json or template files and CSS. It would be cool if there was an optional mechanism for baking that extra configuration into the SQLite file itself. That way entire datasette mini-applications (including canned queries and custom HTML and CSS) could be constructed as single .db files. Since datasette configuration is all file-based, one way to achieve that would be to support a ""datasette_files"" table which, if present is used to search for file contents by path. This is inline with the philosophy described by https://www.sqlite.org/appfileformat.html ",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/188/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 275415799,MDU6SXNzdWUyNzU0MTU3OTk=,137,Ability to combine multiple SQL queries on a single graph,9599,open,0,,,1,2017-11-20T16:26:57Z,2019-05-13T18:33:51Z,,OWNER,,This would make visualizations significantly more powerful. The interesting challenge will be around the URL design. It would be useful to be able to combine either multiple explicit SQL queries or multiple queries based on the filter string parameters passed to one or more table views.,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/137/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 328229224,MDU6SXNzdWUzMjgyMjkyMjQ=,304,Ability to configure SQLite cache_size,9599,closed,0,,,3,2018-05-31T17:28:07Z,2018-06-04T16:13:32Z,2018-06-04T16:03:19Z,OWNER,,"See https://www.sqlite.org/pragma.html#pragma_cache_size Let's call the config setting `cache_size_kb` to emphasize that we're using the negative option. Note this warning: perhaps we should raise an error if you try to use this setting against a SQLite version prior to 3.7.10 > If the argument N is positive then the suggested cache size is set to N. If the argument N is negative, then the number of cache pages is adjusted to use approximately abs(N*1024) bytes of memory. Backwards compatibility note: The behavior of cache_size with a negative N was different in prior to version 3.7.10 (2012-01-16). In version 3.7.9 and earlier, the number of pages in the cache was set to the absolute value of N.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/304/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 626211658,MDU6SXNzdWU2MjYyMTE2NTg=,778,Ability to configure keyset pagination for views and queries,9599,open,0,,,1,2020-05-28T04:48:56Z,2020-10-02T02:26:25Z,,OWNER,,"Currently views offer pagination, but it uses offset/limit - e.g. https://latest.datasette.io/fixtures/paginated_view?_next=100 This means pagination will perform poorly on deeper pages. If a view is based on a table that has a primary key it should be possible to configure efficient keyset pagination that works the same way that table pagination works. This may be as simple as configuring a column that can be treated as a ""primary key"" for the purpose of pagination using `metadata.json` - or with a `?_view_pk=colname` querystring argument.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/778/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 602533352,MDU6SXNzdWU2MDI1MzMzNTI=,2,Ability to convert HEIC images to JPEG,9599,closed,0,,5324096,1,2020-04-18T19:23:43Z,2020-04-28T16:47:21Z,2020-04-28T16:47:21Z,MEMBER,,,256834907,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-photos/issues/2/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1447465004,I_kwDOBm6k_c5WRpAs,1889,Ability to create new tokens via the API,9599,open,0,,8755003,0,2022-11-14T06:21:36Z,2022-12-13T05:29:08Z,,OWNER,,"Refs: - #1850 Initially I decided that the API shouldn't be able to create new tokens at all - I don't like the idea of an API token holder creating themselves additional tokens. Then I realized that two of the API features are specifically more useful if you can generate fresh tokens via the API: - Tokes that expire after a time limit are MUCH more useful if they can be automatically generated - Likewise, tokens that are restricted to a subset of permissions (see #1855) make more sense to be generated like this, especially in conjunction with expiry times",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1889/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 573578548,MDU6SXNzdWU1NzM1Nzg1NDg=,89,Ability to customize columns used by extracts= feature,9599,open,0,,,3,2020-03-01T16:54:48Z,2020-10-16T19:17:50Z,,OWNER,,"@simonw any thoughts on allow extracts to specify the lookup column name? If I'm understanding the documentation right, `.lookup()` allows you to define the ""value"" column (the documentation uses name), but when you use `extracts` keyword as part of `.insert()`, `.upsert()` etc. the lookup must be done against a column named ""value"". I have an existing lookup table that I've populated with columns ""id"" and ""name"" as opposed to ""id"" and ""value"", and seems I can't use `extracts=`, unless I'm missing something... Initial thought on how to do this would be to allow the dictionary value to be a tuple of table name column pair... so: ``` table = db.table(""trees"", extracts={""species_id"": (""Species"", ""name""}) ``` I haven't dug too much into the existing code yet, but does this make sense? Worth doing? _Originally posted by @chrishas35 in https://github.com/simonw/sqlite-utils/issues/46#issuecomment-592999503_",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/89/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 276842536,MDU6SXNzdWUyNzY4NDI1MzY=,153,Ability to customize presentation of specific columns in HTML view,20264,closed,0,,2949431,14,2017-11-26T17:46:11Z,2017-12-10T02:08:45Z,2017-12-07T06:17:33Z,NONE,,"This ties into https://github.com/simonw/datasette/issues/3 in some ways. It would be great to have some adaptability in the HTML views and to specific some columns as displaying in certain ways. - [x] 1. **Auto-parsing URIs into in-browser links.** Why? Lots of public data around cultural commons stuff links to a specific URL. This would be a great utility to turn on at the command line, just parse everything for URLs. Maybe they need to be underlined or represented in a different way than internal URLs. - [x] 2. **Ability to identify a column as plain/preformatted text.** Why? Was trying to import the Enron emails, the body collapses. Hard to read. These fields also tend to screw up the ability to scan a table view. If you knew it was text the system could set an `overflow` property on the relevant CSS, so you could still scan. - [x] 3. **Ability to identify a column as HTML.** Why? I want to spider some stuff and drop sections into SQLite, and just keep them as HTML.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/153/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 634112607,MDU6SXNzdWU2MzQxMTI2MDc=,812,Ability to customize what happens when a view permission fails,9599,closed,0,,5533512,3,2020-06-08T04:26:14Z,2020-07-01T04:17:46Z,2020-07-01T04:17:45Z,OWNER,,"Currently view permission failures raise a `Forbidden` error which is transformed into a 403. It would be good if this page could offer a way forward - maybe just by linking to (or redirecting to) a login screen. This behaviour will vary based on authentication plugins, so a new plugin hook is probably the best way to do this.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/812/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 961367843,MDU6SXNzdWU5NjEzNjc4NDM=,1422,Ability to default to hiding the SQL for a canned query,9599,closed,0,,,4,2021-08-05T02:51:39Z,2021-08-07T05:32:29Z,2021-08-07T05:32:29Z,OWNER,,"I'm working on a project with some HUGE (400+ lines of SQL) canned queries right now. Any time you land on the canned query page you have to scroll down a long distance to get to the results! Would be useful to be able to default to https://latest.datasette.io/fixtures/magic_parameters?_hide_sql=1 without needing the parameter.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1422/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed