{"html_url": "https://github.com/simonw/datasette/issues/1349#issuecomment-850778311", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1349", "id": 850778311, "node_id": "MDEyOklzc3VlQ29tbWVudDg1MDc3ODMxMQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-29T06:12:21Z", "updated_at": "2021-05-29T06:12:21Z", "author_association": "OWNER", "body": "It's not just facets, I think it's trying to execute suggested facets too!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 906385991, "label": "CSV ?_stream=on redundantly calculates facets for every page"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1349#issuecomment-850778273", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1349", "id": 850778273, "node_id": "MDEyOklzc3VlQ29tbWVudDg1MDc3ODI3Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-29T06:11:59Z", "updated_at": "2021-05-29T06:11:59Z", "author_association": "OWNER", "body": "Related issue: https://github.com/simonw/datasette/issues/263 - \"Facets should not execute for ?shape=array|object\"", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 906385991, "label": "CSV ?_stream=on redundantly calculates facets for every page"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1349#issuecomment-850778230", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1349", "id": 850778230, "node_id": "MDEyOklzc3VlQ29tbWVudDg1MDc3ODIzMA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-29T06:11:28Z", "updated_at": "2021-05-29T06:11:28Z", "author_association": "OWNER", "body": "Relevant code: https://github.com/simonw/datasette/blob/0a7621f96f8ad14da17e7172e8a7bce24ef78966/datasette/views/base.py#L263-L315", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 906385991, "label": "CSV ?_stream=on redundantly calculates facets for every page"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/250#issuecomment-850772428", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/250", "id": 850772428, "node_id": "MDEyOklzc3VlQ29tbWVudDg1MDc3MjQyOA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-29T05:16:02Z", "updated_at": "2021-05-29T05:28:57Z", "author_association": "OWNER", "body": "I needed to find some CSV files on my computer with a BOM at the beginning - I figured out this recipe:\r\n```\r\n% rg -U -E none '^(?-u:\\xEF\\xBB\\xBF)' --glob '*.csv' .\r\n```\r\nTIL here: https://til.simonwillison.net/bash/finding-bom-csv-files-with-ripgrep", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 838148087, "label": "Handle byte order marks (BOMs) in CSV files"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/250#issuecomment-850771264", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/250", "id": 850771264, "node_id": "MDEyOklzc3VlQ29tbWVudDg1MDc3MTI2NA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-29T05:06:13Z", "updated_at": "2021-05-29T05:06:13Z", "author_association": "OWNER", "body": "The other option is to check if the file starts with `codecs.BOM_UTF8` - which is `b'\\xef\\xbb\\xbf'`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 838148087, "label": "Handle byte order marks (BOMs) in CSV files"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/250#issuecomment-850771017", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/250", "id": 850771017, "node_id": "MDEyOklzc3VlQ29tbWVudDg1MDc3MTAxNw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-29T05:04:28Z", "updated_at": "2021-05-29T05:04:28Z", "author_association": "OWNER", "body": "https://stackoverflow.com/a/44573867/6083 says:\r\n\r\n> There is no reason to check if a BOM exists or not, utf-8-sig manages that for you and behaves exactly as utf-8 if the BOM does not exist\r\n\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 838148087, "label": "Handle byte order marks (BOMs) in CSV files"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/260#issuecomment-850769067", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/260", "id": 850769067, "node_id": "MDEyOklzc3VlQ29tbWVudDg1MDc2OTA2Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-29T04:48:10Z", "updated_at": "2021-05-29T04:48:10Z", "author_association": "OWNER", "body": "I confirmed and it's possible to have a SQLite column with a hyphen at the start, confirmed using:\r\n```\r\n% sqlite-utils create-table demo.db demo -- id integer name text -blah integer\r\n% sqlite-utils tables --schema demo.db -t\r\ntable schema\r\n------- ---------------------\r\ndemo CREATE TABLE [demo] (\r\n [id] INTEGER,\r\n [name] TEXT,\r\n [-blah] INTEGER\r\n )\r\n```\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 906330187, "label": "Support creating descending order indexes"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/260#issuecomment-850768315", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/260", "id": 850768315, "node_id": "MDEyOklzc3VlQ29tbWVudDg1MDc2ODMxNQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-29T04:39:33Z", "updated_at": "2021-05-29T04:39:33Z", "author_association": "OWNER", "body": "This doesn't work:\r\n```\r\nsqlite-utils create-index mydb.db dogs \"-age\" name\r\n```\r\nBut this does:\r\n```\r\nsqlite-utils create-index mydb.db dogs -- -age name\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 906330187, "label": "Support creating descending order indexes"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/260#issuecomment-850767210", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/260", "id": 850767210, "node_id": "MDEyOklzc3VlQ29tbWVudDg1MDc2NzIxMA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-29T04:26:26Z", "updated_at": "2021-05-29T04:28:31Z", "author_association": "OWNER", "body": "It's weird having to use `Database.DescIndex` - I'm going to put `DescIndex` in `sqlite_utils.db` directly and let people import it.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 906330187, "label": "Support creating descending order indexes"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/260#issuecomment-850766552", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/260", "id": 850766552, "node_id": "MDEyOklzc3VlQ29tbWVudDg1MDc2NjU1Mg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-29T04:20:40Z", "updated_at": "2021-05-29T04:24:01Z", "author_association": "OWNER", "body": "`PRAGMA index_xinfo(table)` DOES return that data:\r\n```\r\n(Pdb) [c[0] for c in fresh_db.execute(\"PRAGMA index_xinfo('idx_dogs_age_name')\").description]\r\n['seqno', 'cid', 'name', 'desc', 'coll', 'key']\r\n(Pdb) fresh_db.execute(\"PRAGMA index_xinfo('idx_dogs_age_name')\").fetchall()\r\n[(0, 2, 'age', 1, 'BINARY', 1), (1, 0, 'name', 0, 'BINARY', 1), (2, -1, None, 0, 'BINARY', 0)]\r\n```\r\nSee https://sqlite.org/pragma.html#pragma_index_xinfo\r\n\r\nExample output: https://covid-19.datasettes.com/covid?sql=select+*+from+pragma_index_xinfo%28%27idx_ny_times_us_counties_date%27%29", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 906330187, "label": "Support creating descending order indexes"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/260#issuecomment-850766335", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/260", "id": 850766335, "node_id": "MDEyOklzc3VlQ29tbWVudDg1MDc2NjMzNQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-29T04:18:19Z", "updated_at": "2021-05-29T04:18:19Z", "author_association": "OWNER", "body": "Annoyingly the `table.indexes` property won't indicate if an index is in regular or reverse order - because the SQLite `PRAGMA index_info(table)` statement doesn't indicate that either. You have to look at the `sqlite_master` index definition to tell if any of the columns are in reverse order:\r\n```\r\n(Pdb) fresh_db.execute(\"select * from sqlite_master where type = 'index'\").fetchall()\r\n[('index', 'idx_dogs_age_name', 'dogs', 3, 'CREATE INDEX [idx_dogs_age_name]\\n ON [dogs] ([age] desc, [name])')]\r\n(Pdb) fresh_db.execute(\"PRAGMA index_info('idx_dogs_age_name')\").fetchall()\r\n[(0, 2, 'age'), (1, 0, 'name')]\r\n(Pdb) fresh_db.execute(\"PRAGMA index_info('idx_dogs_age_name')\").description\r\n(('seqno', None, None, None, None, None, None), ('cid', None, None, None, None, None, None), ('name', None, None, None, None, None, None))\r\n(Pdb) dogs.indexes\r\n[Index(seq=0, name='idx_dogs_age_name', unique=0, origin='c', partial=0, columns=['age', 'name'])]\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 906330187, "label": "Support creating descending order indexes"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/260#issuecomment-850765450", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/260", "id": 850765450, "node_id": "MDEyOklzc3VlQ29tbWVudDg1MDc2NTQ1MA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-29T04:09:13Z", "updated_at": "2021-05-29T04:09:13Z", "author_association": "OWNER", "body": "Decisions: for the Python API I'm going with `db.DescIndex(\"column\")` as the way to do this.\r\n\r\nFor the CLI I'm going to do the \"-age\" thing.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 906330187, "label": "Support creating descending order indexes"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/260#issuecomment-850765291", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/260", "id": 850765291, "node_id": "MDEyOklzc3VlQ29tbWVudDg1MDc2NTI5MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-29T04:07:48Z", "updated_at": "2021-05-29T04:08:21Z", "author_association": "OWNER", "body": "For the CLI version I could say that you can use a `-` prefix to specify reverse direction:\r\n```sh\r\nsqlite-utils create-index mydb.db dogs -age name\r\n```\r\nNo, that doesn't work - it could get confused with a command-line flag. I guess you could do this:\r\n```\r\nsqlite-utils create-index mydb.db dogs \"-age\" name\r\n```\r\nThis does mean that if any of your column names begin with a hyphen you can't use the CLI to add indexes to them. Is that an acceptable limitation? Users can always use `sqlite-utils mydb.db \"create index ...\"` in that case.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 906330187, "label": "Support creating descending order indexes"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/260#issuecomment-850765050", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/260", "id": 850765050, "node_id": "MDEyOklzc3VlQ29tbWVudDg1MDc2NTA1MA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-29T04:05:24Z", "updated_at": "2021-05-29T04:05:40Z", "author_association": "OWNER", "body": "Need to solve this for the CLI tool too. Currently that works like this: https://sqlite-utils.datasette.io/en/stable/cli.html#creating-indexes\r\n```sh\r\nsqlite-utils create-index mydb.db mytable col1 [col2...]\r\n```\r\nEven harder to decide how to add a descending option to this. Maybe like this?\r\n```sh\r\nsqlite-utils create-index mydb.db mytable --direction col1 asc --direction col2 desc\r\n```\r\nIt's a bit gross though! We're saying here that if a single one of the columns you are creating an index for is in reverse direction you have to use `--direction` to specify each end every other index.\r\n\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 906330187, "label": "Support creating descending order indexes"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/260#issuecomment-850764700", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/260", "id": 850764700, "node_id": "MDEyOklzc3VlQ29tbWVudDg1MDc2NDcwMA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-29T04:02:10Z", "updated_at": "2021-05-29T04:02:10Z", "author_association": "OWNER", "body": "I could use `db.desc_index(\"age\")` to match SQLite SQL syntax, which uses `desc` and not `descending`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 906330187, "label": "Support creating descending order indexes"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/260#issuecomment-850764655", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/260", "id": 850764655, "node_id": "MDEyOklzc3VlQ29tbWVudDg1MDc2NDY1NQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-29T04:01:41Z", "updated_at": "2021-05-29T04:01:41Z", "author_association": "OWNER", "body": "Maybe:\r\n```python\r\ndb[\"dogs\"].create_index([db.descending_index(\"age\"), \"name\"])\r\n```\r\nIt's a little verbose but it's for a relatively rare activity and it does make it very clear what is going on.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 906330187, "label": "Support creating descending order indexes"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/260#issuecomment-850764594", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/260", "id": 850764594, "node_id": "MDEyOklzc3VlQ29tbWVudDg1MDc2NDU5NA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-29T04:00:54Z", "updated_at": "2021-05-29T04:00:54Z", "author_association": "OWNER", "body": "A few options:\r\n```python\r\ndb[\"dogs\"].create_index([(\"age\", \"desc\"), \"name\"])\r\n\r\ndb[\"dogs\"].create_index([desc(\"age\"), \"name\"])\r\n\r\ndb[\"dogs\"].create_index([db.desc(\"age\"), \"name\"])\r\n```\r\nThe first option uses an optional tuple. The second two use a `desc()` function - the question is where should that live? \r\n\r\n`sqlite_utils.desc(column)` or `db.desc(column)` are both options.\r\n\r\nI don't like using the term `desc()` for \"descending index\" though - it feels like it should mean something more broad.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 906330187, "label": "Support creating descending order indexes"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/260#issuecomment-850764253", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/260", "id": 850764253, "node_id": "MDEyOklzc3VlQ29tbWVudDg1MDc2NDI1Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-29T03:57:54Z", "updated_at": "2021-05-29T03:57:54Z", "author_association": "OWNER", "body": "The problem here is differentiating between a column with the name `date desc` and wanting to create a descending index on a column called `date`.\r\n\r\nThis won't work:\r\n```python\r\ndb[\"ny_times_us_counties\"].create_index([\"date desc\"], desc=True)\r\n```\r\nBecause we need to be able to create compound indexes with columns with different directions - for example:\r\n```sql\r\ncreate index idx_age_desc_name on dogs (age desc, name)\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 906330187, "label": "Support creating descending order indexes"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1296#issuecomment-850583584", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1296", "id": 850583584, "node_id": "MDEyOklzc3VlQ29tbWVudDg1MDU4MzU4NA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-28T18:06:11Z", "updated_at": "2021-05-28T18:06:11Z", "author_association": "OWNER", "body": "> As a bonus, the Docker image becomes smaller\r\n\r\nThat's a huge surprise to me! And most welcome.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 855446829, "label": "Dockerfile: use Ubuntu 20.10 as base"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1346#issuecomment-850059328", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1346", "id": 850059328, "node_id": "MDEyOklzc3VlQ29tbWVudDg1MDA1OTMyOA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-28T02:08:10Z", "updated_at": "2021-05-28T02:09:25Z", "author_association": "OWNER", "body": "This is the current test failure, but it actually another problem that we don't have tests in place for errors with different formats and shapes:\r\n\r\n```\r\n def test_magic_parameters_cannot_be_used_in_arbitrary_queries(magic_parameters_client):\r\n response = magic_parameters_client.get(\r\n \"/data.json?sql=select+:_header_host&_shape=array\"\r\n )\r\n assert 400 == response.status\r\n> assert \"You did not supply a value for binding 1.\" == response.json[\"error\"]\r\nE TypeError: list indices must be integers or slices, not str\r\n```\r\n\r\nThe test fails because `response.json` here is the empty list `[]`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 904537568, "label": "Re-display user's query with an error message if an error occurs"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/619#issuecomment-850059607", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/619", "id": 850059607, "node_id": "MDEyOklzc3VlQ29tbWVudDg1MDA1OTYwNw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-28T02:08:58Z", "updated_at": "2021-05-28T02:08:58Z", "author_association": "OWNER", "body": "Remaining work will happen in #1346", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 520655983, "label": "\"Invalid SQL\" page should let you edit the SQL"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1346#issuecomment-850058851", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1346", "id": 850058851, "node_id": "MDEyOklzc3VlQ29tbWVudDg1MDA1ODg1MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-28T02:06:46Z", "updated_at": "2021-05-28T02:06:46Z", "author_association": "OWNER", "body": "The one test failure here actually illustrates a larger problem: if the user specifies `.json?_shape=array` but an error occurs, what should we do?\r\n\r\nPrior to this change we return the following JSON:\r\n```json\r\n{\r\n \"ok\": false,\r\n \"error\": \"You did not supply a value for binding 1.\",\r\n \"status\": 500,\r\n \"title\": null\r\n}\r\n```\r\nBut this comes from the `handle_500` higher level code here: https://github.com/simonw/datasette/blob/eae3084b46e2c3931db12cdef79093ad0e644bce/datasette/app.py#L1251-L1263", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 904537568, "label": "Re-display user's query with an error message if an error occurs"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/619#issuecomment-850057694", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/619", "id": 850057694, "node_id": "MDEyOklzc3VlQ29tbWVudDg1MDA1NzY5NA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-28T02:03:05Z", "updated_at": "2021-05-28T02:03:05Z", "author_association": "OWNER", "body": "I nearly got this working, but I ran into one last problem: the code path for when an error is raised but the user specified `?_shape=array`. I'll open a draft PR with where I've got to so far.", "reactions": "{\"total_count\": 1, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 1, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 520655983, "label": "\"Invalid SQL\" page should let you edit the SQL"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/619#issuecomment-850025106", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/619", "id": 850025106, "node_id": "MDEyOklzc3VlQ29tbWVudDg1MDAyNTEwNg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-28T00:18:30Z", "updated_at": "2021-05-28T00:18:30Z", "author_association": "OWNER", "body": "I'm going to return a 400 HTTP status code for \"bad request\", under the assumption that the client sent bad SQL.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 520655983, "label": "\"Invalid SQL\" page should let you edit the SQL"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/619#issuecomment-850019486", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/619", "id": 850019486, "node_id": "MDEyOklzc3VlQ29tbWVudDg1MDAxOTQ4Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-28T00:05:50Z", "updated_at": "2021-05-28T00:05:50Z", "author_association": "OWNER", "body": "Came up on discussions here: https://github.com/simonw/datasette/discussions/1334", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 520655983, "label": "\"Invalid SQL\" page should let you edit the SQL"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1344#issuecomment-849820019", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1344", "id": 849820019, "node_id": "MDEyOklzc3VlQ29tbWVudDg0OTgyMDAxOQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-27T17:44:39Z", "updated_at": "2021-05-27T17:52:58Z", "author_association": "OWNER", "body": "This pattern appears to work, executed at the root of a checkout of `datasette`:\r\n```\r\ndocker run -it -v `pwd`:/mnt --platform linux/amd64 \\\r\n datasetteproject/datasette:0.57a1 bash -c '\r\n pip install \"pytest>=5.2.2,<6.3.0\" \\\r\n \"pytest-xdist>=2.2.1,<2.3\" \\\r\n \"pytest-asyncio>=0.10,<0.16\" \\\r\n \"beautifulsoup4>=4.8.1,<4.10.0\" \\\r\n \"black==21.5b1\" \\\r\n \"pytest-timeout>=1.4.2,<1.5\" \\\r\n \"trustme>=0.7,<0.8\" \\\r\n && cd /mnt && pytest'\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 903986178, "label": "Test Datasette Docker images built for different architectures"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1344#issuecomment-849815361", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1344", "id": 849815361, "node_id": "MDEyOklzc3VlQ29tbWVudDg0OTgxNTM2MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-27T17:36:51Z", "updated_at": "2021-05-27T17:36:51Z", "author_association": "OWNER", "body": "To run the test suite... need to install a few more dependencies: https://github.com/simonw/datasette/blob/7b106e106000713bbee31b34d694b3dadbd4818c/setup.py#L71-L79\r\n\r\nAnd then mount the `tests/` folder into the container and run `pytest`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 903986178, "label": "Test Datasette Docker images built for different architectures"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1344#issuecomment-849806581", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1344", "id": 849806581, "node_id": "MDEyOklzc3VlQ29tbWVudDg0OTgwNjU4MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-27T17:22:37Z", "updated_at": "2021-05-27T17:34:27Z", "author_association": "OWNER", "body": "```\r\n~ % docker run -it --platform linux/arm/v7 -v `pwd`:/mnt -p 8001:8001 datasetteproject/datasette:0.57a1 python -c 'import platform; print(platform.uname())'\r\nUnable to find image 'datasetteproject/datasette:0.57a1' locally\r\n0.57a1: Pulling from datasetteproject/datasette\r\nee0a2cc24f29: Pull complete \r\nca231153300d: Pull complete \r\n6073fc53d406: Pull complete \r\nbb2a6a6421dd: Pull complete \r\n14c15f441034: Pull complete \r\n28d113f72ba5: Pull complete \r\n40516c2c3785: Pull complete \r\nDigest: sha256:8dd469efa0b34b9e946c4e91496fc7bec702efec9c0f5ec548ccd4030f3263b2\r\nStatus: Downloaded newer image for datasetteproject/datasette:0.57a1\r\nuname_result(system='Linux', node='c2e1a05732ed', release='4.19.121-linuxkit', version='#1 SMP Thu Jan 21 15:36:34 UTC 2021', machine='armv7l')\r\n```\r\nShorter version:\r\n```\r\n~ % docker run -it --platform linux/ppc64le datasetteproject/datasette:0.57a1 python -c 'import platform; print(platform.uname())'\r\nUnable to find image 'datasetteproject/datasette:0.57a1' locally\r\n0.57a1: Pulling from datasetteproject/datasette\r\nc840eb5e9aed: Pull complete \r\nfc17d8b577ec: Pull complete \r\n712a0ad995f8: Pull complete \r\n56101149914c: Pull complete \r\ne6241dfe9d38: Pull complete \r\n41ed85039888: Pull complete \r\n26beeabe9213: Pull complete \r\nDigest: sha256:8dd469efa0b34b9e946c4e91496fc7bec702efec9c0f5ec548ccd4030f3263b2\r\nStatus: Downloaded newer image for datasetteproject/datasette:0.57a1\r\nuname_result(system='Linux', node='0fe85887fa30', release='4.19.121-linuxkit', version='#1 SMP Thu Jan 21 15:36:34 UTC 2021', machine='ppc64le')\r\n~ % \r\n```\r\n\r\nOK! Now I need to figure out how to run the full test suite on each one, then I can call this done.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 903986178, "label": "Test Datasette Docker images built for different architectures"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1344#issuecomment-849805778", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1344", "id": 849805778, "node_id": "MDEyOklzc3VlQ29tbWVudDg0OTgwNTc3OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-27T17:21:11Z", "updated_at": "2021-05-27T17:21:22Z", "author_association": "OWNER", "body": "This helps:\r\n```\r\n~ % docker run -it --platform linux/s390x -v `pwd`:/mnt \\\r\n -p 8001:8001 datasetteproject/datasette:0.57a1 \\\r\n python -c 'import platform; print(platform.uname())'\r\n```\r\n`uname_result(system='Linux', node='d14916ca91df', release='4.19.121-linuxkit', version='#1 SMP Thu Jan 21 15:36:34 UTC 2021', machine='s390x')`", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 903986178, "label": "Test Datasette Docker images built for different architectures"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1344#issuecomment-849804684", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1344", "id": 849804684, "node_id": "MDEyOklzc3VlQ29tbWVudDg0OTgwNDY4NA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-27T17:19:16Z", "updated_at": "2021-05-27T17:19:16Z", "author_association": "OWNER", "body": "Having pushed that to Docker Hub the following _seems_ to work on my laptop:\r\n\r\n docker run --platform linux/s390x \\\r\n -v `pwd`:/mnt -p 8001:8001 \\\r\n datasetteproject/datasette:0.57a1 datasette -p 8001 -h 0.0.0.0\r\n\r\nI'd like to confirm that this is correctly emulating the architecture and running the correct published image though.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 903986178, "label": "Test Datasette Docker images built for different architectures"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1344#issuecomment-849801969", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1344", "id": 849801969, "node_id": "MDEyOklzc3VlQ29tbWVudDg0OTgwMTk2OQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-27T17:14:49Z", "updated_at": "2021-05-27T17:14:49Z", "author_association": "OWNER", "body": "Job completed in 13m38s - results are here: https://hub.docker.com/r/datasetteproject/datasette/tags?page=1&ordering=last_updated&name=0.57a1\r\n\r\n\"datasetteproject_datasette_Tags\"\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 903986178, "label": "Test Datasette Docker images built for different architectures"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1344#issuecomment-849797767", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1344", "id": 849797767, "node_id": "MDEyOklzc3VlQ29tbWVudDg0OTc5Nzc2Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-27T17:08:13Z", "updated_at": "2021-05-27T17:08:13Z", "author_association": "OWNER", "body": "Related issue: #1272 - being able to run the test suite inside Docker inside GitHub Actions would be incredibly helpful here.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 903986178, "label": "Test Datasette Docker images built for different architectures"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1344#issuecomment-849793312", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1344", "id": 849793312, "node_id": "MDEyOklzc3VlQ29tbWVudDg0OTc5MzMxMg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-27T17:01:16Z", "updated_at": "2021-05-27T17:01:16Z", "author_association": "OWNER", "body": "https://github.com/simonw/datasette/runs/2687196350 is a run of https://github.com/simonw/datasette/blob/main/.github/workflows/push_docker_tag.yml to publish the new `0.57a1` alpha to Docker Hub.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 903986178, "label": "Test Datasette Docker images built for different architectures"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1344#issuecomment-849788412", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1344", "id": 849788412, "node_id": "MDEyOklzc3VlQ29tbWVudDg0OTc4ODQxMg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-27T16:53:28Z", "updated_at": "2021-05-27T16:53:28Z", "author_association": "OWNER", "body": "(Should also update https://docs.datasette.io/en/stable/contributing.html#release-process with notes on how this works)", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 903986178, "label": "Test Datasette Docker images built for different architectures"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1344#issuecomment-849788159", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1344", "id": 849788159, "node_id": "MDEyOklzc3VlQ29tbWVudDg0OTc4ODE1OQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-27T16:53:04Z", "updated_at": "2021-05-27T16:53:04Z", "author_association": "OWNER", "body": "I'm going to release a `0.57a1` alpha to Docker Hub using the new buildx mechanism.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 903986178, "label": "Test Datasette Docker images built for different architectures"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1319#issuecomment-849785098", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1319", "id": 849785098, "node_id": "MDEyOklzc3VlQ29tbWVudDg0OTc4NTA5OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-27T16:48:21Z", "updated_at": "2021-05-27T16:48:21Z", "author_association": "OWNER", "body": "OK, since I have a mechanism for deploying alpha versions now I'm going to merge this and use it to publish 0.57a1.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 881219362, "label": "Add Docker multi-arch support with Buildx"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1343#issuecomment-849784005", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1343", "id": 849784005, "node_id": "MDEyOklzc3VlQ29tbWVudDg0OTc4NDAwNQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-27T16:46:37Z", "updated_at": "2021-05-27T16:46:37Z", "author_association": "OWNER", "body": "That worked:\r\n```\r\n % docker run -it datasetteproject/datasette:0.57a0\r\nUnable to find image 'datasetteproject/datasette:0.57a0' locally\r\n0.57a0: Pulling from datasetteproject/datasette\r\n75646c2fb410: Pull complete \r\nedf8b70d3cc4: Pull complete \r\n229daa3ebd94: Pull complete \r\n65d21ac50a56: Pull complete \r\ne083ad7cc3ca: Pull complete \r\ned4de1431432: Pull complete \r\n1da5e8b27cca: Pull complete \r\nDigest: sha256:4ab61095c1d879a91d154ed469227387356a305035da9c74542c55bd500c89db\r\nStatus: Downloaded newer image for datasetteproject/datasette:0.57a0\r\nINFO: Started server process [1]\r\nINFO: Waiting for application startup.\r\nINFO: Application startup complete.\r\nINFO: Uvicorn running on http://127.0.0.1:8001 (Press CTRL+C to quit)\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 903978133, "label": "Figure out how to publish alpha/beta releases to Docker Hub"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1343#issuecomment-849783398", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1343", "id": 849783398, "node_id": "MDEyOklzc3VlQ29tbWVudDg0OTc4MzM5OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-27T16:45:41Z", "updated_at": "2021-05-27T16:45:41Z", "author_association": "OWNER", "body": "https://hub.docker.com/r/datasetteproject/datasette/tags?page=1&ordering=last_updated now shows this at the top: https://hub.docker.com/layers/datasetteproject/datasette/0.57a0/images/sha256-4ab61095c1d879a91d154ed469227387356a305035da9c74542c55bd500c89db?context=explore", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 903978133, "label": "Figure out how to publish alpha/beta releases to Docker Hub"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1343#issuecomment-849782098", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1343", "id": 849782098, "node_id": "MDEyOklzc3VlQ29tbWVudDg0OTc4MjA5OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-27T16:43:40Z", "updated_at": "2021-05-27T16:43:55Z", "author_association": "OWNER", "body": "https://github.com/simonw/datasette/actions/workflows/push_docker_tag.yml\r\n\r\n\"Actions_\u00b7_simonw_datasette\"\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 903978133, "label": "Figure out how to publish alpha/beta releases to Docker Hub"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1343#issuecomment-849781747", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1343", "id": 849781747, "node_id": "MDEyOklzc3VlQ29tbWVudDg0OTc4MTc0Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-27T16:43:04Z", "updated_at": "2021-05-27T16:43:04Z", "author_association": "OWNER", "body": "I'm going to try pushing an alpha of https://github.com/simonw/datasette/releases/tag/0.57a0 using this workflow I built a few months ago: https://github.com/simonw/datasette/blob/1a8972f9c012cd22b088c6b70661a9c3d3847853/.github/workflows/push_docker_tag.yml", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 903978133, "label": "Figure out how to publish alpha/beta releases to Docker Hub"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1319#issuecomment-849780481", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1319", "id": 849780481, "node_id": "MDEyOklzc3VlQ29tbWVudDg0OTc4MDQ4MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-27T16:41:03Z", "updated_at": "2021-05-27T16:41:03Z", "author_association": "OWNER", "body": "It looks like all I need to do to ship an alpha version to Docker Hub is NOT point the `latest` tag at it after it goes live: https://github.com/simonw/datasette/blob/1a8972f9c012cd22b088c6b70661a9c3d3847853/.github/workflows/publish.yml#L75-L77", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 881219362, "label": "Add Docker multi-arch support with Buildx"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1319#issuecomment-849778373", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1319", "id": 849778373, "node_id": "MDEyOklzc3VlQ29tbWVudDg0OTc3ODM3Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-27T16:37:50Z", "updated_at": "2021-05-27T16:37:50Z", "author_association": "OWNER", "body": "Started [a conversation](https://twitter.com/simonw/status/1397953338407223303) about this on Twitter. Ideally I'd like to push an alpha release to Docker Hub so people can start testing these out before they make it into an official non-alpha release.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 881219362, "label": "Add Docker multi-arch support with Buildx"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1337#issuecomment-849762735", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1337", "id": 849762735, "node_id": "MDEyOklzc3VlQ29tbWVudDg0OTc2MjczNQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-27T16:14:14Z", "updated_at": "2021-05-27T16:14:14Z", "author_association": "OWNER", "body": "Here's a demo showing that once you get above 1,000 results you still get the \"...\" but it no longer links to anything: https://fivethirtyeight.datasettes.com/fivethirtyeight/births%2FUS_births_2000-2014_SSA?_facet=births&_facet_size=max", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 898904402, "label": "\"More\" link for facets that shows _facet_size=max results"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1332#issuecomment-849761894", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1332", "id": 849761894, "node_id": "MDEyOklzc3VlQ29tbWVudDg0OTc2MTg5NA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-27T16:13:07Z", "updated_at": "2021-05-27T16:13:07Z", "author_association": "OWNER", "body": "Added `?_facet_size=max` in #1337 and made the `...` truncated note link to that - here's a demo: https://fivethirtyeight.datasettes.com/fivethirtyeight/antiquities-act%2Factions_under_antiquities_act?_facet=states&_facet=current_agency&_facet=pres_or_congress", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 893890496, "label": "?_facet_size=X to increase number of facets results on the page"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1337#issuecomment-849761508", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1337", "id": 849761508, "node_id": "MDEyOklzc3VlQ29tbWVudDg0OTc2MTUwOA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-27T16:12:31Z", "updated_at": "2021-05-27T16:12:31Z", "author_association": "OWNER", "body": "More exciting demo: https://fivethirtyeight.datasettes.com/fivethirtyeight/antiquities-act%2Factions_under_antiquities_act?_facet=states&_facet=current_agency&_facet=pres_or_congress", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 898904402, "label": "\"More\" link for facets that shows _facet_size=max results"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1337#issuecomment-849759440", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1337", "id": 849759440, "node_id": "MDEyOklzc3VlQ29tbWVudDg0OTc1OTQ0MA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-27T16:09:34Z", "updated_at": "2021-05-27T16:09:34Z", "author_association": "OWNER", "body": "Demo: https://latest.datasette.io/fixtures/facetable?_facet_size=2&_facet=state&_facet=city_id\r\n\r\n![demo-max](https://user-images.githubusercontent.com/9599/119860507-3f071780-becb-11eb-8698-64c8c6112983.gif)\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 898904402, "label": "\"More\" link for facets that shows _facet_size=max results"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1337#issuecomment-849725291", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1337", "id": 849725291, "node_id": "MDEyOklzc3VlQ29tbWVudDg0OTcyNTI5MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-27T15:23:33Z", "updated_at": "2021-05-27T15:23:33Z", "author_association": "OWNER", "body": "Turns out `path_with_replaced_args()` is already exposed to the `table.html` template context: https://github.com/simonw/datasette/blob/f1c29fd6a184254aa68efadf096bcf21e848f921/datasette/views/table.py#L897-L900", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 898904402, "label": "\"More\" link for facets that shows _facet_size=max results"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1342#issuecomment-849724915", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1342", "id": 849724915, "node_id": "MDEyOklzc3VlQ29tbWVudDg0OTcyNDkxNQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-27T15:23:01Z", "updated_at": "2021-05-27T15:23:01Z", "author_association": "OWNER", "body": "Since they are already exposed (but undocumented) this isn't actually a blocker on #1337 like I thought it was. Should still clean this up in the future though.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 903902495, "label": "Improve `path_with_replaced_args()` and friends and document them"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1342#issuecomment-849724108", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1342", "id": 849724108, "node_id": "MDEyOklzc3VlQ29tbWVudDg0OTcyNDEwOA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-27T15:21:54Z", "updated_at": "2021-05-27T15:21:54Z", "author_association": "OWNER", "body": "Turns out some of these are already being exposed to the template context on an ad-hoc basis.\r\n\r\nIn `QueryView`: https://github.com/simonw/datasette/blob/2bd9d54b2762c991e11950c22c88c0336158d49b/datasette/views/database.py#L295-L304\r\n\r\nIn `TableView`: https://github.com/simonw/datasette/blob/f1c29fd6a184254aa68efadf096bcf21e848f921/datasette/views/table.py#L897-L900", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 903902495, "label": "Improve `path_with_replaced_args()` and friends and document them"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1342#issuecomment-849722218", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1342", "id": 849722218, "node_id": "MDEyOklzc3VlQ29tbWVudDg0OTcyMjIxOA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-27T15:19:23Z", "updated_at": "2021-05-27T15:19:23Z", "author_association": "OWNER", "body": "Current implementations: https://github.com/simonw/datasette/blob/51d788114035458d8f860d9ea6d74078e6c0ea0d/datasette/utils/__init__.py#L214-L272", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 903902495, "label": "Improve `path_with_replaced_args()` and friends and document them"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1337#issuecomment-849721280", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1337", "id": 849721280, "node_id": "MDEyOklzc3VlQ29tbWVudDg0OTcyMTI4MA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-27T15:18:08Z", "updated_at": "2021-05-27T15:18:08Z", "author_association": "OWNER", "body": "In order to cleanly implement this I need to expose the `path_with_replaced_args` utility function to Datasette's template engine. This is the first time this will become an exposed (and hence should-by-documented) API and I don't like its shape much.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 898904402, "label": "\"More\" link for facets that shows _facet_size=max results"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1337#issuecomment-849695117", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1337", "id": 849695117, "node_id": "MDEyOklzc3VlQ29tbWVudDg0OTY5NTExNw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-27T14:44:13Z", "updated_at": "2021-05-27T14:44:13Z", "author_association": "OWNER", "body": "Here's a much easier way to implement this: implement `?_facet_size=max` (sets it to `max_returned_rows`, default 1000) and then link to that from the ellipsis.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 898904402, "label": "\"More\" link for facets that shows _facet_size=max results"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1319#issuecomment-849311884", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1319", "id": 849311884, "node_id": "MDEyOklzc3VlQ29tbWVudDg0OTMxMTg4NA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-27T04:33:51Z", "updated_at": "2021-05-27T04:33:51Z", "author_association": "OWNER", "body": "This is so cool! I wonder how I can best test it? I have a Raspberry Pi I should dig out to try this.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 881219362, "label": "Add Docker multi-arch support with Buildx"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1341#issuecomment-849310294", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1341", "id": 849310294, "node_id": "MDEyOklzc3VlQ29tbWVudDg0OTMxMDI5NA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-27T04:29:04Z", "updated_at": "2021-05-27T04:29:04Z", "author_association": "OWNER", "body": "For implementation simplicity I'm going to show the item even if all of the columns are visible because `?_col=` has selected all of them - selecting it will still reset the view to having no `?_col=` or `?_nocol=` parameters.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 903200328, "label": "\"Show all columns\" cog menu item should show if ?_col= is used"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/615#issuecomment-849309330", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/615", "id": 849309330, "node_id": "MDEyOklzc3VlQ29tbWVudDg0OTMwOTMzMA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-27T04:26:10Z", "updated_at": "2021-05-27T04:26:10Z", "author_association": "OWNER", "body": "Demos:\r\n\r\n- https://latest.datasette.io/fixtures/sortable?_nocol=sortable\r\n- https://latest.datasette.io/fixtures/sortable?_col=sortable&_col=text\r\n\r\nAlso try the column cog menu on that page.\r\n\r\nDocumentation: https://docs.datasette.io/en/latest/json_api.html#special-table-arguments", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 517451234, "label": "?_col= and ?_nocol= support for toggling columns on table view"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/292#issuecomment-849308907", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/292", "id": 849308907, "node_id": "MDEyOklzc3VlQ29tbWVudDg0OTMwODkwNw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-27T04:25:01Z", "updated_at": "2021-05-27T04:25:01Z", "author_association": "OWNER", "body": "Now that `?_col=` and `?_nocol=` are implemented I'm closing this ticket - other customizations can already be handled by defining SQL views or creating canned SQL queries.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 326800219, "label": "Mechanism for customizing the SQL used to select specific columns in the table view"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/670#issuecomment-849022714", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/670", "id": 849022714, "node_id": "MDEyOklzc3VlQ29tbWVudDg0OTAyMjcxNA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-26T18:33:47Z", "updated_at": "2021-05-26T18:33:58Z", "author_association": "OWNER", "body": "Worth mentioning here: I've been doing a tun of research around running Datasette-like functionality against PostgreSQL in my https://github.com/simonw/django-sql-dashboard project - which will definitely inform the Datasette implementation.", "reactions": "{\"total_count\": 3, \"+1\": 3, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 564833696, "label": "Prototoype for Datasette on PostgreSQL"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1340#issuecomment-847987346", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1340", "id": 847987346, "node_id": "MDEyOklzc3VlQ29tbWVudDg0Nzk4NzM0Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-25T15:49:55Z", "updated_at": "2021-05-25T15:49:55Z", "author_association": "OWNER", "body": "Usual problem with this kind of UI: what would it do on mobile where hover isn't a thing? Plenty of other questions: if it opens a menu, what options might I put in there other than \"Show more like this\"?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 901009787, "label": "Research: Cell action menu (like column action but for individual cells)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1327#issuecomment-847240470", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1327", "id": 847240470, "node_id": "MDEyOklzc3VlQ29tbWVudDg0NzI0MDQ3MA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-24T18:21:44Z", "updated_at": "2021-05-24T18:21:44Z", "author_association": "OWNER", "body": "I'm having trouble replicating this - I tried the following `metadata.json` in my local environment and it displayed correctly:\r\n```json\r\n{\r\n \"title\": \"\u1019\u103c\u1014\u103a\u1019\u102c\u1005\u102c (\"\r\n}\r\n```\r\nWhat operating system are you using here? Is there a chance that your `metadata.json` file is saved in a text encoding other than utf-8?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 892457208, "label": "Support Unicode characters in metadata.json"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1320#issuecomment-847237524", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1320", "id": 847237524, "node_id": "MDEyOklzc3VlQ29tbWVudDg0NzIzNzUyNA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-24T18:15:56Z", "updated_at": "2021-05-24T18:15:56Z", "author_association": "OWNER", "body": "Added some new documentation about that here: https://github.com/simonw/datasette/blob/c0a748e5c3f498fa8c139b420d07dd3dea612379/docs/installation.rst#installing-plugins", "reactions": "{\"total_count\": 1, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 1, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 884952179, "label": "Can't use apt-get in Dockerfile when using datasetteproj/datasette as base"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1320#issuecomment-847232547", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1320", "id": 847232547, "node_id": "MDEyOklzc3VlQ29tbWVudDg0NzIzMjU0Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-24T18:06:32Z", "updated_at": "2021-05-24T18:06:32Z", "author_association": "OWNER", "body": "I'm testing tweaks to the Dockerfile by editing it and running:\r\n```\r\ndocker build . -t datasette-057a0 --build-arg VERSION=0.57a0\r\n```\r\nThen running this to inspect the new image:\r\n```\r\ndocker run -it -p 8001:8001 datasette-057a0 bash\r\n```\r\nAfter changing that line to `rm -rf /var/lib/dpkg/info/*` I can run `apt-get` successfully from inside that bash prompt:\r\n```\r\n# apt-get update\r\n# apt-get install ripgrep\r\n# rg --version\r\nripgrep 12.1.1\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 884952179, "label": "Can't use apt-get in Dockerfile when using datasetteproj/datasette as base"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1320#issuecomment-847230866", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1320", "id": 847230866, "node_id": "MDEyOklzc3VlQ29tbWVudDg0NzIzMDg2Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-24T18:03:29Z", "updated_at": "2021-05-24T18:03:29Z", "author_association": "OWNER", "body": "```\r\nroot@33c269900821:/# du -h /var/lib/dpkg\r\n6.3M\t/var/lib/dpkg/info\r\n24K\t/var/lib/dpkg/triggers\r\n4.0K\t/var/lib/dpkg/updates\r\n20K\t/var/lib/dpkg/alternatives\r\n4.0K\t/var/lib/dpkg/parts\r\n6.7M\t/var/lib/dpkg\r\n```\r\nSo `/var/lib/dpkg/info` is most of the space, I'll try just deleting that.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 884952179, "label": "Can't use apt-get in Dockerfile when using datasetteproj/datasette as base"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1320#issuecomment-847223853", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1320", "id": 847223853, "node_id": "MDEyOklzc3VlQ29tbWVudDg0NzIyMzg1Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-24T17:51:12Z", "updated_at": "2021-05-24T17:51:12Z", "author_association": "OWNER", "body": "This is a good catch, thank you.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 884952179, "label": "Can't use apt-get in Dockerfile when using datasetteproj/datasette as base"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/615#issuecomment-846592583", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/615", "id": 846592583, "node_id": "MDEyOklzc3VlQ29tbWVudDg0NjU5MjU4Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-23T16:50:45Z", "updated_at": "2021-05-24T17:15:48Z", "author_association": "OWNER", "body": "Still needed:\r\n\r\n- Unit tests\r\n- Documentation\r\n\r\nAnd maybe a UI mechanism for this? I could at least add a \"remove this column\" item to the cog menu on columns.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 517451234, "label": "?_col= and ?_nocol= support for toggling columns on table view"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/615#issuecomment-846668419", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/615", "id": 846668419, "node_id": "MDEyOklzc3VlQ29tbWVudDg0NjY2ODQxOQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-24T01:14:27Z", "updated_at": "2021-05-24T01:14:27Z", "author_association": "OWNER", "body": "I tried allowing the removal of `rowid` and got this exception:\r\n```\r\n File \"/Users/simon/Dropbox/Development/datasette/datasette/views/table.py\", line 831, in extra_template\r\n display_columns, display_rows = await self.display_columns_and_rows(\r\n File \"/Users/simon/Dropbox/Development/datasette/datasette/views/table.py\", line 163, in display_columns_and_rows\r\n pk_path = path_from_row_pks(row, pks, not pks, False)\r\n File \"/Users/simon/Dropbox/Development/datasette/datasette/utils/__init__.py\", line 82, in path_from_row_pks\r\n bits = [row[\"rowid\"]]\r\nIndexError: No item with that key\r\n```\r\nI'm going to disable the removal of `rowid` - or indeed any of the primary keys, since they are needed to construct the row permalink.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 517451234, "label": "?_col= and ?_nocol= support for toggling columns on table view"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/615#issuecomment-846660103", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/615", "id": 846660103, "node_id": "MDEyOklzc3VlQ29tbWVudDg0NjY2MDEwMw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-24T00:47:00Z", "updated_at": "2021-05-24T00:47:00Z", "author_association": "OWNER", "body": "> Here's a bug: removing the `rowid` column returns an error.\r\n\r\nRemoving the `rowid` column should work. We can continue to show the `Link` column, ensuring users can still navigate to the row page for each row.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 517451234, "label": "?_col= and ?_nocol= support for toggling columns on table view"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/615#issuecomment-846659272", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/615", "id": 846659272, "node_id": "MDEyOklzc3VlQ29tbWVudDg0NjY1OTI3Mg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-24T00:44:47Z", "updated_at": "2021-05-24T00:44:47Z", "author_association": "OWNER", "body": "I can't think of any good reasons to support passing the same column twice `?col=airline&_col=airline` so I'm going to de-duplicate and silently ignore the second one.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 517451234, "label": "?_col= and ?_nocol= support for toggling columns on table view"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/615#issuecomment-846627322", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/615", "id": 846627322, "node_id": "MDEyOklzc3VlQ29tbWVudDg0NjYyNzMyMg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-23T21:31:22Z", "updated_at": "2021-05-23T21:31:22Z", "author_association": "OWNER", "body": "Would it be useful to allow this mechanism to alias columns, for example supporting one of the following:\r\n\r\n- `?_col=airline as name_of_airline`\r\n- `?_col=airline:name_of_airline`\r\n\r\nThis could be handy for renaming columns to match a specific expected JSON output.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 517451234, "label": "?_col= and ?_nocol= support for toggling columns on table view"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/615#issuecomment-846627163", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/615", "id": 846627163, "node_id": "MDEyOklzc3VlQ29tbWVudDg0NjYyNzE2Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-23T21:30:01Z", "updated_at": "2021-05-23T21:30:01Z", "author_association": "OWNER", "body": "Interesting side-effect of this implementation is that you can both control column order and request the same column multiple times:\r\n\r\n`/fivethirtyeight/airline-safety%2Fairline-safety?_col=fatal_accidents_00_14&_col=fatalities_00_14&_col=airline&_col=airline`\r\n\r\n\"fivethirtyeight__airline-safety_airline-safety__56_rows\"\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 517451234, "label": "?_col= and ?_nocol= support for toggling columns on table view"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/615#issuecomment-846626871", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/615", "id": 846626871, "node_id": "MDEyOklzc3VlQ29tbWVudDg0NjYyNjg3MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-23T21:27:36Z", "updated_at": "2021-05-23T21:27:36Z", "author_association": "OWNER", "body": "A better interface for this would be a full list of columns each with a checkbox for making it visible on invisible - this could then be used to apply a bulk change (rather than refreshing the interface after every removed column) and it could also be easily designed to work on narrow mobile screens where the cog icon is not visible.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 517451234, "label": "?_col= and ?_nocol= support for toggling columns on table view"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/615#issuecomment-846626567", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/615", "id": 846626567, "node_id": "MDEyOklzc3VlQ29tbWVudDg0NjYyNjU2Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-23T21:25:19Z", "updated_at": "2021-05-23T21:25:19Z", "author_association": "OWNER", "body": "Here's a bug: removing the `rowid` column returns an error.\r\n\r\n![rowid-bug](https://user-images.githubusercontent.com/9599/119277132-b179ae00-bbd2-11eb-8f2f-e228c7d7c264.gif)\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 517451234, "label": "?_col= and ?_nocol= support for toggling columns on table view"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/615#issuecomment-846599732", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/615", "id": 846599732, "node_id": "MDEyOklzc3VlQ29tbWVudDg0NjU5OTczMg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-23T17:46:45Z", "updated_at": "2021-05-23T17:46:45Z", "author_association": "OWNER", "body": "I've changed my mind about forbidding `?_col=` and `?_nocol=` from being applied at the same time - I'm going to come up with a simple resolution rule instead.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 517451234, "label": "?_col= and ?_nocol= support for toggling columns on table view"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/615#issuecomment-846595473", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/615", "id": 846595473, "node_id": "MDEyOklzc3VlQ29tbWVudDg0NjU5NTQ3Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-23T17:13:09Z", "updated_at": "2021-05-23T17:13:09Z", "author_association": "OWNER", "body": "Natalie suggests a quick way to implement \"undo\" would be to add a \"Show all columns\" item to that menu which only appears when at least one column is hidden.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 517451234, "label": "?_col= and ?_nocol= support for toggling columns on table view"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/615#issuecomment-846595091", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/615", "id": 846595091, "node_id": "MDEyOklzc3VlQ29tbWVudDg0NjU5NTA5MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-23T17:09:59Z", "updated_at": "2021-05-23T17:09:59Z", "author_association": "OWNER", "body": "Here's that prototype of a \"Hide this column\" cog menu button:\r\n\r\n![hide](https://user-images.githubusercontent.com/9599/119269916-effd7180-bbae-11eb-92fa-0ef374bda4b8.gif)\r\n\r\nNeed a way to undo that once you've hidden a column - maybe a list of currently hidden columns that lets you un-hide them.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 517451234, "label": "?_col= and ?_nocol= support for toggling columns on table view"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/615#issuecomment-846592392", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/615", "id": 846592392, "node_id": "MDEyOklzc3VlQ29tbWVudDg0NjU5MjM5Mg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-23T16:49:30Z", "updated_at": "2021-05-23T16:49:44Z", "author_association": "OWNER", "body": "I started looking at this again, inspired by #1326. I have a new diff that works against the latest `main` branch.\r\n\r\n```diff\r\ndiff --git a/datasette/views/table.py b/datasette/views/table.py\r\nindex 4879228..f4b2ee2 100644\r\n--- a/datasette/views/table.py\r\n+++ b/datasette/views/table.py\r\n@@ -64,6 +64,36 @@ class Row:\r\n \r\n \r\n class RowTableShared(DataView):\r\n+ async def columns_to_select(self, db, table, request):\r\n+ table_columns = await db.table_columns(table)\r\n+ if \"_col\" in request.args and \"_nocol\" in request.args:\r\n+ raise DatasetteError(\"Cannot use _col and _nocol at the same time\")\r\n+ if \"_col\" in request.args:\r\n+ new_columns = []\r\n+ for column in request.args.getlist(\"_col\"):\r\n+ if column not in table_columns:\r\n+ raise DatasetteError(\"_col={} is an invalid column\".format(column))\r\n+ new_columns.append(column)\r\n+ return new_columns\r\n+ elif \"_nocol\" in request.args:\r\n+ # Return all columns EXCEPT these\r\n+ bad_columns = [\r\n+ column\r\n+ for column in request.args.getlist(\"_nocol\")\r\n+ if column not in table_columns\r\n+ ]\r\n+ if bad_columns:\r\n+ raise DatasetteError(\r\n+ \"_nocol={} - invalid columns\".format(\", \".join(bad_columns))\r\n+ )\r\n+ return [\r\n+ column\r\n+ for column in table_columns\r\n+ if column not in request.args.getlist(\"_nocol\")\r\n+ ]\r\n+ else:\r\n+ return table_columns\r\n+\r\n async def sortable_columns_for_table(self, database, table, use_rowid):\r\n db = self.ds.databases[database]\r\n table_metadata = self.ds.table_metadata(database, table)\r\n@@ -321,18 +351,16 @@ class TableView(RowTableShared):\r\n )\r\n \r\n pks = await db.primary_keys(table)\r\n- table_column_details = await db.table_column_details(table)\r\n- table_columns = [column.name for column in table_column_details]\r\n-\r\n- select_columns = \", \".join(escape_sqlite(t) for t in table_columns)\r\n+ table_columns = await self.columns_to_select(db, table, request)\r\n+ select_clause = \", \".join(escape_sqlite(t) for t in table_columns)\r\n \r\n use_rowid = not pks and not is_view\r\n if use_rowid:\r\n- select = f\"rowid, {select_columns}\"\r\n+ select = f\"rowid, {select_clause}\"\r\n order_by = \"rowid\"\r\n order_by_pks = \"rowid\"\r\n else:\r\n- select = select_columns\r\n+ select = select_clause\r\n order_by_pks = \", \".join([escape_sqlite(pk) for pk in pks])\r\n order_by = order_by_pks\r\n \r\n@@ -715,6 +743,8 @@ class TableView(RowTableShared):\r\n column = fk[\"column\"]\r\n if column not in columns_to_expand:\r\n continue\r\n+ if column not in columns:\r\n+ continue\r\n expanded_columns.append(column)\r\n # Gather the values\r\n column_index = columns.index(column)\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 517451234, "label": "?_col= and ?_nocol= support for toggling columns on table view"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1326#issuecomment-846493370", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1326", "id": 846493370, "node_id": "MDEyOklzc3VlQ29tbWVudDg0NjQ5MzM3MA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-23T02:55:00Z", "updated_at": "2021-05-23T02:55:00Z", "author_association": "OWNER", "body": "Currently you can do this with a custom SQL query, but as you've noticed this means you don't get automatic pagination: https://latest.datasette.io/fixtures.json?sql=select+pk%2C+created%2C+state+from+facetable+order+by+pk+limit+10&_shape=array\r\n\r\nI have an open issue to allow selecting specific columns on the table page: #615\r\n\r\nA workaround you can use today is to define a SQL view that selects just the tables you want. View pages behave exactly like tables in that they provide pagination by default.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 891969037, "label": "How to limit fields returned from the JSON API?"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1331#issuecomment-846483426", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1331", "id": 846483426, "node_id": "MDEyOklzc3VlQ29tbWVudDg0NjQ4MzQyNg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-23T00:57:30Z", "updated_at": "2021-05-23T00:57:30Z", "author_association": "OWNER", "body": "Here's an alpha release with updated dependencies: https://pypi.org/project/datasette/0.57a0/", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 893537744, "label": "Add support for Jinja2 version 3.0"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1331#issuecomment-846482057", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1331", "id": 846482057, "node_id": "MDEyOklzc3VlQ29tbWVudDg0NjQ4MjA1Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-23T00:39:55Z", "updated_at": "2021-05-23T00:39:55Z", "author_association": "OWNER", "body": "> I'm stuck also because datasette wants itsdangerous~=1.1 instead of allowing itsdangerous-2.0.0\r\n\r\nBumped that dependency in b64d87204612a84663616e075f542499a5d82a03", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 893537744, "label": "Add support for Jinja2 version 3.0"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1331#issuecomment-846481937", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1331", "id": 846481937, "node_id": "MDEyOklzc3VlQ29tbWVudDg0NjQ4MTkzNw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-23T00:38:28Z", "updated_at": "2021-05-23T00:38:28Z", "author_association": "OWNER", "body": "I'm going to push an alpha release with the new dependencies shortly.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 893537744, "label": "Add support for Jinja2 version 3.0"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1337#issuecomment-846481896", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1337", "id": 846481896, "node_id": "MDEyOklzc3VlQ29tbWVudDg0NjQ4MTg5Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-23T00:37:52Z", "updated_at": "2021-05-23T00:37:52Z", "author_association": "OWNER", "body": "The tricky thing about this is that ideally I'd like this to link to a paginated set of results - but arbitrary SQL queries do not yet support pagination, see #856.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 898904402, "label": "\"More\" link for facets that shows _facet_size=max results"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1332#issuecomment-846481801", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1332", "id": 846481801, "node_id": "MDEyOklzc3VlQ29tbWVudDg0NjQ4MTgwMQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-23T00:36:20Z", "updated_at": "2021-05-23T00:36:20Z", "author_association": "OWNER", "body": "Documentation:\r\n\r\n- https://docs.datasette.io/en/latest/facets.html#facets-in-query-strings at the bottom of the section\r\n- https://docs.datasette.io/en/latest/json_api.html#special-table-arguments at the bottom of the list of arguments", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 893890496, "label": "?_facet_size=X to increase number of facets results on the page"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1332#issuecomment-846479730", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1332", "id": 846479730, "node_id": "MDEyOklzc3VlQ29tbWVudDg0NjQ3OTczMA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-23T00:11:31Z", "updated_at": "2021-05-23T00:11:31Z", "author_association": "OWNER", "body": "Here's the code in question: https://github.com/simonw/datasette/blob/5e9672c9bb33e41686472db4aa427168f9e67dbe/datasette/facets.py#L187-L210\r\n\r\nIt has access to the current request object as `self.request`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 893890496, "label": "?_facet_size=X to increase number of facets results on the page"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1332#issuecomment-846479212", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1332", "id": 846479212, "node_id": "MDEyOklzc3VlQ29tbWVudDg0NjQ3OTIxMg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-23T00:07:46Z", "updated_at": "2021-05-23T00:07:46Z", "author_association": "OWNER", "body": "`?_facet_size=50` seems like a very reasonable thing to support - I'll cap it at whatever the `max_returned_rows` setting is (which defaults to 1000).", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 893890496, "label": "?_facet_size=X to increase number of facets results on the page"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1332#issuecomment-846479062", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1332", "id": 846479062, "node_id": "MDEyOklzc3VlQ29tbWVudDg0NjQ3OTA2Mg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-23T00:06:34Z", "updated_at": "2021-05-23T00:06:34Z", "author_association": "OWNER", "body": "A URL parameter to modify that facet size is a really good idea.\r\n\r\nI thought I had an issue open for \"...\" linking to more results but I can't find it now.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 893890496, "label": "?_facet_size=X to increase number of facets results on the page"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/github-to-sqlite/pull/59#issuecomment-844250232", "issue_url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/59", "id": 844250232, "node_id": "MDEyOklzc3VlQ29tbWVudDg0NDI1MDIzMg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-19T16:08:10Z", "updated_at": "2021-05-19T16:08:10Z", "author_association": "MEMBER", "body": "Thanks for catching this.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 771872303, "label": "Remove unneeded exists=True for -a/--auth flag."}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/github-to-sqlite/pull/61#issuecomment-844249385", "issue_url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/61", "id": 844249385, "node_id": "MDEyOklzc3VlQ29tbWVudDg0NDI0OTM4NQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-19T16:07:06Z", "updated_at": "2021-05-19T16:07:06Z", "author_association": "MEMBER", "body": "Thanks!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 797108702, "label": "fixing typo in get cli help text"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/253#issuecomment-843718859", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/253", "id": 843718859, "node_id": "MDEyOklzc3VlQ29tbWVudDg0MzcxODg1OQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-19T03:31:47Z", "updated_at": "2021-05-19T03:31:47Z", "author_association": "OWNER", "body": "Fixed: https://simonwillison.net/2020/Sep/23/sqlite-advanced-alter-table/", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 847423559, "label": "fixtures.db example error in sql-utils blog post"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/253#issuecomment-843716042", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/253", "id": 843716042, "node_id": "MDEyOklzc3VlQ29tbWVudDg0MzcxNjA0Mg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-19T03:29:04Z", "updated_at": "2021-05-19T03:29:04Z", "author_association": "OWNER", "body": "Thanks for pointing this out - the problem here is that these days the `https://latest.datasette.io/fixtures.db` example database includes a test for generated columns, which are a brand new feature in SQLite. Your local SQLite is an older version.\r\n\r\nI'm going to update the blog post to instead use a copy of `fixtures.db` that doesn't use that feature.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 847423559, "label": "fixtures.db example error in sql-utils blog post"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/259#issuecomment-843715502", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/259", "id": 843715502, "node_id": "MDEyOklzc3VlQ29tbWVudDg0MzcxNTUwMg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-19T03:27:33Z", "updated_at": "2021-05-19T03:27:33Z", "author_association": "OWNER", "body": "```\r\n % sqlite-utils insert test-big.db text big.json \r\nError: table text has no column named colun\r\n\r\nTry using --alter to add additional columns\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 894948100, "label": "Suggest the --alter option if a new column cannot be added"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/256#issuecomment-843715308", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/256", "id": 843715308, "node_id": "MDEyOklzc3VlQ29tbWVudDg0MzcxNTMwOA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-19T03:26:54Z", "updated_at": "2021-05-19T03:26:54Z", "author_association": "OWNER", "body": "I decided to treat this as a usability bug: there's a workaround, which is adding `--alter` - but it's not at all obvious that you should do that. I addressed this in #259.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 861622839, "label": "inserting with --nl errors with: sqlite3.OperationalError: table has no column named "}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/256#issuecomment-843708246", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/256", "id": 843708246, "node_id": "MDEyOklzc3VlQ29tbWVudDg0MzcwODI0Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-19T03:06:02Z", "updated_at": "2021-05-19T03:06:02Z", "author_association": "OWNER", "body": "Did you try running this with the `--alter` command-line option? That seems to work when I try it on a sample file here.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 861622839, "label": "inserting with --nl errors with: sqlite3.OperationalError: table
has no column named "}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/244#issuecomment-843705806", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/244", "id": 843705806, "node_id": "MDEyOklzc3VlQ29tbWVudDg0MzcwNTgwNg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-19T02:58:18Z", "updated_at": "2021-05-19T02:58:18Z", "author_association": "OWNER", "body": "Thanks!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 820468864, "label": "Typo in upsert example"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/245#issuecomment-843705721", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/245", "id": 843705721, "node_id": "MDEyOklzc3VlQ29tbWVudDg0MzcwNTcyMQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-19T02:58:01Z", "updated_at": "2021-05-19T02:58:01Z", "author_association": "OWNER", "body": "Thanks very much.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 830803173, "label": "Correct some typos"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/254#issuecomment-843705533", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/254", "id": 843705533, "node_id": "MDEyOklzc3VlQ29tbWVudDg0MzcwNTUzMw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-19T02:57:22Z", "updated_at": "2021-05-19T02:57:22Z", "author_association": "OWNER", "body": "Thanks!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 857280617, "label": "Fix incorrect create-table cli description"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/258#issuecomment-843702392", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/258", "id": 843702392, "node_id": "MDEyOklzc3VlQ29tbWVudDg0MzcwMjM5Mg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-19T02:47:37Z", "updated_at": "2021-05-19T02:47:37Z", "author_association": "OWNER", "body": "I'm going to merge this and add a test - thanks!", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 868191959, "label": "Fixing insert from JSON containing strings with non-ascii characters \u2026"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/856#issuecomment-843291675", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/856", "id": 843291675, "node_id": "MDEyOklzc3VlQ29tbWVudDg0MzI5MTY3NQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-18T15:56:45Z", "updated_at": "2021-05-18T15:56:45Z", "author_association": "OWNER", "body": "Tables and views get \"stream all rows\" at the moment, so one workaround is to define a SQL view for your query - this only works for queries that don't take any parameters though (although you may be able to define a view and then pass it extra fields using the Datasette table interface, like on https://latest.datasette.io/fixtures/paginated_view?content_extra__contains=9)\r\n\r\nI've explored this problem in a bit more detail in https://githu.com/simonw/django-sql-dashboard and I think I have a pattern that could work.\r\n\r\nFor your canned query, you could implement the pattern yourself by setting up two canned queries that look something like this:\r\n\r\nhttps://github-to-sqlite.dogsheep.net/github?sql=select+rowid%2C+sha%2C+author_date+from+commits+order+by+rowid+limit+1000\r\n\r\n```sql\r\nselect rowid, sha, author_date from commits order by rowid limit 1000\r\n```\r\nThat gets you the first set of 1,000 results. The important thing here is to order by a unique column, in this case `rowid` - because then subsequent pages can be loaded by a separate canned query that looks like this:\r\n```sql\r\nselect rowid, sha, author_date from commits where rowid > :after order by rowid limit 1000\r\n```\r\nhttps://github-to-sqlite.dogsheep.net/github?sql=select+rowid%2C+sha%2C+author_date+from+commits+where+rowid+%3E+%3Aafter+order+by+rowid+limit+1000&after=1000\r\n\r\nYou then need to write code which knows how to generate these queries - start with the first query with no `where` clause (or if you are using `rowid` you can just use the second query and pass it `?after=0` for the first call) - then keep calling the query passing in the last rowid you recieved as the `after` parameter.\r\n\r\nBasically this is an implementation of keyset pagination with a smart client. When Datasette grows the ability to do this itself it will work by executing this mechanism inside the Python code, which is how the \"stream all rows\" option for tables works at the moment.", "reactions": "{\"total_count\": 1, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 1, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 642296989, "label": "Consider pagination of canned queries"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1331#issuecomment-842497125", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1331", "id": 842497125, "node_id": "MDEyOklzc3VlQ29tbWVudDg0MjQ5NzEyNQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-17T17:20:15Z", "updated_at": "2021-05-17T17:20:15Z", "author_association": "OWNER", "body": "I wonder if there are any new 3.0 features we should be taking advantage of here that would justify pinning to 3.0 minimum?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 893537744, "label": "Add support for Jinja2 version 3.0"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1331#issuecomment-842495346", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1331", "id": 842495346, "node_id": "MDEyOklzc3VlQ29tbWVudDg0MjQ5NTM0Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-17T17:17:19Z", "updated_at": "2021-05-17T17:17:19Z", "author_association": "OWNER", "body": "The tests are passing on #1324 so this may not require any code changes at all!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 893537744, "label": "Add support for Jinja2 version 3.0"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1324#issuecomment-842494844", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1324", "id": 842494844, "node_id": "MDEyOklzc3VlQ29tbWVudDg0MjQ5NDg0NA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-17T17:16:36Z", "updated_at": "2021-05-17T17:16:36Z", "author_association": "OWNER", "body": "Refs #1331.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 890073940, "label": "Update jinja2 requirement from <2.12.0,>=2.10.3 to >=2.10.3,<3.1.0"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1331#issuecomment-842494332", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1331", "id": 842494332, "node_id": "MDEyOklzc3VlQ29tbWVudDg0MjQ5NDMzMg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-17T17:15:45Z", "updated_at": "2021-05-17T17:16:04Z", "author_association": "OWNER", "body": "Datasette uses async templates which evidently have been improved in Jinja 2 3.0.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 893537744, "label": "Add support for Jinja2 version 3.0"}, "performed_via_github_app": null}