id,node_id,number,title,user,user_label,state,locked,assignee,assignee_label,milestone,milestone_label,comments,created_at,updated_at,closed_at,author_association,pull_request,body,repo,repo_label,type,active_lock_reason,performed_via_github_app,reactions,draft,state_reason
1052247023,I_kwDOBm6k_c4-uAPv,1505,Datasette should have an option to output CSV with semicolons,9599,simonw,open,0,,,,,1,2021-11-12T18:02:21Z,2021-11-16T11:40:52Z,,OWNER,,,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1505/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1102484126,I_kwDOBm6k_c5BtpKe,1595,Release notes for 0.60,9599,simonw,closed,0,,,7571612,Datasette 0.60,4,2022-01-13T22:23:14Z,2022-01-14T01:37:39Z,2022-01-14T01:37:39Z,OWNER,,,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1595/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1306984363,I_kwDOBm6k_c5N5v-r,1771,minor a11y: <select> has no visual indicator when tabbed to,1473102,mustafa0x,closed,0,,,,,5,2022-07-17T04:30:14Z,2022-12-18T06:34:20Z,2022-12-18T06:28:12Z,NONE,,,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1771/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
267516066,MDU6SXNzdWUyNjc1MTYwNjY=,5,Implement sensible query pagination,9599,simonw,closed,0,,,2857392,Ship first public release,3,2017-10-23T01:16:00Z,2017-11-10T20:41:39Z,2017-11-10T20:41:39Z,OWNER,,,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/5/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
267517314,MDU6SXNzdWUyNjc1MTczMTQ=,8,Attempting an INSERT or UPDATE should return a sane error message,9599,simonw,closed,0,,,2857392,Ship first public release,1,2017-10-23T01:28:25Z,2017-10-23T15:28:12Z,2017-10-23T15:28:08Z,OWNER,,,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/8/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
267517348,MDU6SXNzdWUyNjc1MTczNDg=,9,Initial test suite,9599,simonw,closed,0,,,2857392,Ship first public release,2,2017-10-23T01:28:46Z,2017-10-24T05:55:33Z,2017-10-24T05:55:33Z,OWNER,,,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/9/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
267517381,MDU6SXNzdWUyNjc1MTczODE=,10,Set up Travis,9599,simonw,closed,0,,,2859414,v1 stretch goals,1,2017-10-23T01:29:07Z,2017-11-04T23:48:57Z,2017-11-04T23:48:57Z,OWNER,,,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/10/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
268087542,MDU6SXNzdWUyNjgwODc1NDI=,31,Idea: colour scheme based on sha256 of db,9599,simonw,closed,0,,,2859414,v1 stretch goals,1,2017-10-24T15:52:38Z,2018-05-28T18:10:45Z,2017-11-09T14:14:59Z,OWNER,,,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/31/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
273174397,MDU6SXNzdWUyNzMxNzQzOTc=,62,Link to .json and .jsono versions on various pages,9599,simonw,closed,0,,,2857392,Ship first public release,0,2017-11-11T20:37:47Z,2017-11-11T22:41:06Z,2017-11-11T22:41:06Z,OWNER,,,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/62/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
273174447,MDU6SXNzdWUyNzMxNzQ0NDc=,63,Review design of JSON output,9599,simonw,closed,0,,,2857392,Ship first public release,1,2017-11-11T20:38:33Z,2017-11-11T22:20:17Z,2017-11-11T22:20:17Z,OWNER,,,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/63/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
273181020,MDU6SXNzdWUyNzMxODEwMjA=,64,Support for ?field__isnull=1 or similar,9599,simonw,closed,0,,,,,1,2017-11-11T22:26:52Z,2017-11-17T14:38:21Z,2017-11-17T14:38:21Z,OWNER,,,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/64/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
273296684,MDU6SXNzdWUyNzMyOTY2ODQ=,74,Send a 302 redirect to the new hash for hits to old hashes,9599,simonw,closed,0,,,2857392,Ship first public release,1,2017-11-13T03:00:59Z,2017-11-13T18:49:59Z,2017-11-13T18:49:59Z,OWNER,,,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/74/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
273509159,MDU6SXNzdWUyNzM1MDkxNTk=,75,Add --cors argument to serve,9599,simonw,closed,0,,,2857392,Ship first public release,1,2017-11-13T17:16:19Z,2017-11-13T18:17:52Z,2017-11-13T18:17:52Z,OWNER,,,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/75/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
273554949,MDU6SXNzdWUyNzM1NTQ5NDk=,78,Rename after to next and provide a next_url,9599,simonw,closed,0,,,2857392,Ship first public release,0,2017-11-13T19:48:31Z,2017-11-13T20:35:03Z,2017-11-13T20:35:03Z,OWNER,,,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/78/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
273596159,MDU6SXNzdWUyNzM1OTYxNTk=,82,Post a blog entry announcing it to the world,9599,simonw,closed,0,,,2857392,Ship first public release,1,2017-11-13T22:10:35Z,2017-11-14T01:46:10Z,2017-11-14T01:46:10Z,OWNER,,,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/82/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
274023625,MDU6SXNzdWUyNzQwMjM2MjU=,99,Start a change log,9599,simonw,closed,0,,,,,0,2017-11-15T03:33:21Z,2017-11-16T15:12:46Z,2017-11-16T15:12:45Z,OWNER,,,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/99/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
274378301,MDU6SXNzdWUyNzQzNzgzMDE=,109,Set up readthedocs,9599,simonw,closed,0,,,,,1,2017-11-16T02:58:01Z,2017-11-16T16:53:26Z,2017-11-16T16:13:56Z,OWNER,,,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/109/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
274884209,MDU6SXNzdWUyNzQ4ODQyMDk=,116,Add documentation section about SQLite extensions,9599,simonw,closed,0,,,,,1,2017-11-17T14:36:30Z,2018-05-28T17:23:42Z,2018-05-28T17:23:41Z,OWNER,,,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/116/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
275135535,MDU6SXNzdWUyNzUxMzU1MzU=,126,Blog entry announcing foreign key support,9599,simonw,closed,0,,,2919870,Foreign key edition,1,2017-11-19T06:09:06Z,2017-11-30T16:49:24Z,2017-11-30T16:49:24Z,OWNER,,,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/126/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
277589569,MDU6SXNzdWUyNzc1ODk1Njk=,155,A primary key column that has foreign key restriction associated won't rendering label column,388154,wsxiaoys,closed,0,,,2949431,Custom templates edition,4,2017-11-29T00:40:02Z,2017-12-07T05:39:53Z,2017-12-07T05:39:53Z,NONE,,,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/155/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
280023225,MDU6SXNzdWUyODAwMjMyMjU=,166,Documentation for metadata.json and datasette skeleton,9599,simonw,closed,0,,,2949431,Custom templates edition,1,2017-12-07T07:02:52Z,2017-12-07T17:20:35Z,2017-12-07T17:20:25Z,OWNER,,,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/166/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
280745470,MDU6SXNzdWUyODA3NDU0NzA=,170,Custom template for named canned query,9599,simonw,closed,0,,,2949431,Custom templates edition,3,2017-12-09T19:07:51Z,2017-12-09T21:35:30Z,2017-12-09T21:34:52Z,OWNER,,,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/170/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
331343824,MDU6SXNzdWUzMzEzNDM4MjQ=,309,On 404s with a trailing slash redirect to that page without a trailing slash,9599,simonw,closed,0,,,3439337,0.23.1,2,2018-06-11T20:46:49Z,2018-06-21T15:22:02Z,2018-06-21T15:13:15Z,OWNER,,,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/309/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
336464733,MDU6SXNzdWUzMzY0NjQ3MzM=,328,"Installation instructions, including how to use the docker image",9599,simonw,closed,0,,,,,4,2018-06-28T03:59:33Z,2023-09-05T14:10:39Z,2018-06-28T04:02:10Z,OWNER,,,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/328/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
340065374,MDU6SXNzdWUzNDAwNjUzNzQ=,337,Documentation for datasette publish and datasette package,9599,simonw,closed,0,,,,,1,2018-07-11T02:04:06Z,2018-07-11T02:07:32Z,2018-07-11T02:05:56Z,OWNER,,,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/337/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
344654623,MDU6SXNzdWUzNDQ2NTQ2MjM=,347,"Rename ""datasette package"" to ""datasette publish docker""",9599,simonw,open,0,,,,,0,2018-07-26T00:42:46Z,2018-07-26T00:42:46Z,,OWNER,,,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/347/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
464994105,MDU6SXNzdWU0NjQ5OTQxMDU=,548,Add datasette-cors and datasette-auth-github plugins to Ecosystem page,9599,simonw,closed,0,,,4471010,Datasette 0.29,0,2019-07-07T21:14:14Z,2019-07-08T02:02:36Z,2019-07-08T02:02:36Z,OWNER,,,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/548/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
465003070,MDU6SXNzdWU0NjUwMDMwNzA=,551,Ship many-to-many faceting support (and facet-by-delimiter),9599,simonw,open,0,,,,,2,2019-07-07T23:11:45Z,2019-07-08T15:45:23Z,,OWNER,,,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/551/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
471818939,MDU6SXNzdWU0NzE4MTg5Mzk=,48,"Jupyter notebook demo of the library, launchable on Binder",9599,simonw,closed,0,,,,,2,2019-07-23T17:05:05Z,2022-01-26T02:08:46Z,2022-01-26T02:08:39Z,OWNER,,,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/48/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
493599818,MDU6SXNzdWU0OTM1OTk4MTg=,1,Command for fetching starred repos,9599,simonw,closed,0,,,,,0,2019-09-14T08:36:29Z,2019-09-14T21:30:48Z,2019-09-14T21:30:48Z,MEMBER,,,207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/1/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
562911863,MDU6SXNzdWU1NjI5MTE4NjM=,85,Create index doesn't work for columns containing spaces,9599,simonw,closed,0,,,,,1,2020-02-11T00:34:46Z,2020-02-11T05:13:20Z,2020-02-11T05:13:20Z,OWNER,,,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/85/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
574021194,MDU6SXNzdWU1NzQwMjExOTQ=,691,--reload sould reload server if code in --plugins-dir changes,9599,simonw,open,0,,,,,1,2020-03-02T14:42:21Z,2020-06-14T02:35:17Z,,OWNER,,,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/691/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
586454513,MDU6SXNzdWU1ODY0NTQ1MTM=,20,Upgrade to sqlite-utils 2.x,9599,simonw,closed,0,,,5225818,1.0,0,2020-03-23T19:17:58Z,2020-03-23T19:22:52Z,2020-03-23T19:22:52Z,MEMBER,,,207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/20/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
587314002,MDU6SXNzdWU1ODczMTQwMDI=,709,Each plugin hook should link to example plugins built with it,9599,simonw,closed,0,,,5234079,Datasette 0.39,1,2020-03-24T22:18:48Z,2020-03-24T22:30:10Z,2020-03-24T22:29:43Z,OWNER,,,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/709/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
589491711,MDU6SXNzdWU1ODk0OTE3MTE=,7,Upgrade to sqlite-utils 2.x,9599,simonw,closed,0,,,,,0,2020-03-28T02:24:51Z,2020-03-28T02:25:03Z,2020-03-28T02:25:03Z,MEMBER,,,205429375,swarm-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/7/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
597671518,MDU6SXNzdWU1OTc2NzE1MTg=,98,"Only set .last_rowid and .last_pk for single update/inserts, not for .insert_all()/.upsert_all() with multiple records",9599,simonw,closed,0,,,,,7,2020-04-10T03:19:40Z,2021-09-28T04:38:44Z,2020-04-13T03:29:15Z,OWNER,,,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/98/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
602533352,MDU6SXNzdWU2MDI1MzMzNTI=,2,Ability to convert HEIC images to JPEG,9599,simonw,closed,0,,,5324096,Apple Photos online and securely browsable,1,2020-04-18T19:23:43Z,2020-04-28T16:47:21Z,2020-04-28T16:47:21Z,MEMBER,,,256834907,dogsheep-photos,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-photos/issues/2/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
602533481,MDU6SXNzdWU2MDI1MzM0ODE=,3,"Import EXIF data into SQLite - lens used, ISO, aperture etc",9599,simonw,open,0,,,5324096,Apple Photos online and securely browsable,2,2020-04-18T19:24:31Z,2021-10-05T12:38:24Z,,MEMBER,,,256834907,dogsheep-photos,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-photos/issues/3/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
649373451,MDU6SXNzdWU2NDkzNzM0NTE=,885,Blog entry about the release,9599,simonw,closed,0,,,5533512,Datasette 0.45,1,2020-07-01T22:44:37Z,2020-07-01T22:44:48Z,2020-07-01T22:44:47Z,OWNER,,,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/885/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
688389933,MDU6SXNzdWU2ODgzODk5MzM=,143,Move to GitHub Actions CI,9599,simonw,closed,0,,,,,1,2020-08-28T22:34:11Z,2020-08-28T22:41:35Z,2020-08-28T22:41:35Z,OWNER,,,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/143/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
689839399,MDU6SXNzdWU2ODk4MzkzOTk=,4,Optimize the FTS table,9599,simonw,closed,0,,,,,1,2020-09-01T05:58:17Z,2020-09-01T06:10:08Z,2020-09-01T06:10:08Z,MEMBER,,,197431109,dogsheep-beta,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-beta/issues/4/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
812228314,MDU6SXNzdWU4MTIyMjgzMTQ=,1236,Ability to increase size of the SQL editor window,9599,simonw,closed,0,,,,,9,2021-02-19T18:09:27Z,2021-05-18T03:28:25Z,2021-02-22T21:05:21Z,OWNER,,,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1236/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
871304967,MDU6SXNzdWU4NzEzMDQ5Njc=,1315,"settings.json should be picked up by ""datasette publish cloudrun""",9599,simonw,open,0,,,,,0,2021-04-29T18:16:41Z,2021-04-29T18:16:41Z,,OWNER,,,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1315/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
923602693,MDU6SXNzdWU5MjM2MDI2OTM=,276,support small help flag -h,601708,mcint,closed,0,,,,,0,2021-06-17T07:59:31Z,2021-06-18T14:56:59Z,2021-06-18T14:56:59Z,CONTRIBUTOR,,,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/276/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
923697888,MDU6SXNzdWU5MjM2OTc4ODg=,278,"Support db as first parameter before subcommand, or as environment variable",601708,mcint,closed,0,,,,,3,2021-06-17T09:26:29Z,2021-06-20T22:39:57Z,2021-06-18T15:43:19Z,CONTRIBUTOR,,,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/278/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
927385540,MDU6SXNzdWU5MjczODU1NDA=,8,any guidance / experience on imessage-to-sqlite ?,2675621,Casyfill,open,0,,,,,0,2021-06-22T15:46:16Z,2021-06-22T15:46:16Z,,NONE,,,214746582,dogsheep.github.io,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep.github.io/issues/8/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
927766296,MDU6SXNzdWU5Mjc3NjYyOTY=,291,Adopt flake8,9599,simonw,closed,0,,,,,2,2021-06-23T01:19:37Z,2021-06-24T17:50:27Z,2021-06-24T17:50:27Z,OWNER,,,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/291/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
931752773,MDU6SXNzdWU5MzE3NTI3NzM=,294,Add a `sqlite-utils memory` example to the README,9599,simonw,closed,0,,,,,0,2021-06-28T16:35:59Z,2021-08-18T21:40:03Z,2021-08-18T21:40:03Z,OWNER,,,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/294/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
836063389,MDU6SXNzdWU4MzYwNjMzODk=,17,Datetime columns are not properly formatted to be recognizes as datetime,1234956,n8henrie,open,0,,,,,0,2021-03-19T14:33:04Z,2021-03-19T14:33:04Z,,NONE,,"

Currently, the datetimes are formatted in a way that is not recognized by datasette-vega for plotting with a `Date/time` type for the axis. 

For example, if you have datasette running locally with `datasette-vega` installed and have a database that includes resting heart rate:

```
http://localhost:8001/healthkit/rRestingHeartRate#g.mark=line&g.x_column=startDate&g.x_type=temporal&g.y_column=value&g.y_type=quantitative
```

The plot is blank unless you choose `Label` as the type for the date data.

The `startDate` (and `creationDate` and `endDate`) columns appear like: `2019-11-14 18:22:18 -0700`

If instead the format for this column is changed slightly: `2019-11-14T18:22:18-07:00` they are recognized as proper dates and the charting works as expected.

I have a PR that addresses this issue, will submit shortly.",197882382,healthkit-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/17/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
796736607,MDU6SXNzdWU3OTY3MzY2MDc=,56,Not all quoted statuses get fetched?,42315895,gsajko,closed,0,,,,,3,2021-01-29T09:48:44Z,2021-02-03T10:36:36Z,2021-02-03T10:36:36Z,NONE,,"
![image](https://user-images.githubusercontent.com/42315895/106259325-5f75dc80-621f-11eb-8311-db8f2fe2a257.png)

In my database I have 13300 quote tweets, but eta 3600 have `quoted_status` empty.

I fetched some of them using `https://api.twitter.com/1.1/statuses/show.json?id=xx` and they did have ids of quoted tweets.",206156866,twitter-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/56/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1761613778,I_kwDOBm6k_c5pABfS,2084,Support facets for columns that contain timestamps,19492893,devxpy,open,0,,,,,0,2023-06-17T03:33:54Z,2023-06-17T03:33:54Z,,NONE,,"
Django has this very nice filter for datetime fields -

<img width=""176"" alt=""image"" src=""https://github.com/simonw/datasette/assets/19492893/3c66d7c4-1579-4d30-8f08-89d111f4566e"">

It would be nice to have something similar to facet by a field that contains a timestamp in datasette too - Which doesn't seem to do anything with timestamps right now...

<img width=""283"" alt=""image"" src=""https://github.com/simonw/datasette/assets/19492893/069083e4-13f5-4b28-9473-a7b9d48839ea"">
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2084/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
831751367,MDU6SXNzdWU4MzE3NTEzNjc=,246,Escaping FTS search strings,16001974,DeNeutoy,closed,0,,,,,4,2021-03-15T12:15:09Z,2021-08-18T18:57:13Z,2021-08-18T18:43:12Z,CONTRIBUTOR,,"
Thanks for the excellent library, it's very nice to use!

I've been building some in memory search functionality for a data annotation tool i'm making, and I got tripped up a little bit with escaping the full text search queries. First I tried using `db.quote(q)`, which doesn't work, because sqlite FTS has it's own (separate)[ query syntax](https://www2.sqlite.org/fts5.html#full_text_query_syntax). You can see this happening here also:

http://search-24ways.herokuapp.com/24ways-f8f455f/articles?_search=acces%2A

I got around this by aggressively escaping quotes inside the query string like this:

```python
        quoted = q.replace('""', '""""')
        quoted = f'""{quoted}""'
        print(quoted)
        results = db[""data""].search(quoted, columns=[""id""])
        return [x[""id""] for x in results]

```

This works in the sense it doesn't crash, but it also removes access to the search query syntax. Given the well specified definition, it might be possible for sqlite-utils to provide a `db.quote_query(q)` which would intelligently escape a query whilst leaving the syntax intact. This would be very nice!



",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/246/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1943259395,I_kwDOEhK-wc5z08kD,16, time data '2014-11-21T11:44:12.000Z' does not match format '%Y%m%dT%H%M%SZ',3746270,linonetwo,open,0,,,,,0,2023-10-14T13:24:39Z,2023-10-14T13:24:39Z,,NONE,,"
```
evernote-to-sqlite enex evernote.db ./我的笔记.enex
Importing from ENEX  [#####-------------------------------]   14%
Traceback (most recent call last):
  File ""/usr/local/bin/evernote-to-sqlite"", line 8, in <module>
    sys.exit(cli())
             ^^^^^
  File ""/usr/local/lib/python3.11/site-packages/click/core.py"", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ""/usr/local/lib/python3.11/site-packages/click/core.py"", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File ""/usr/local/lib/python3.11/site-packages/click/core.py"", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ""/usr/local/lib/python3.11/site-packages/click/core.py"", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ""/usr/local/lib/python3.11/site-packages/click/core.py"", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ""/usr/local/lib/python3.11/site-packages/evernote_to_sqlite/cli.py"", line 31, in enex
    save_note(db, note)
  File ""/usr/local/lib/python3.11/site-packages/evernote_to_sqlite/utils.py"", line 46, in save_note
    ""created"": convert_datetime(created),
               ^^^^^^^^^^^^^^^^^^^^^^^^^
  File ""/usr/local/lib/python3.11/site-packages/evernote_to_sqlite/utils.py"", line 111, in convert_datetime
    return datetime.datetime.strptime(s, ""%Y%m%dT%H%M%SZ"").isoformat()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ""/usr/local/Cellar/python@3.11/3.11.5/Frameworks/Python.framework/Versions/3.11/lib/python3.11/_strptime.py"", line 568, in _strptime_datetime
    tt, fraction, gmtoff_fraction = _strptime(data_string, format)
                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ""/usr/local/Cellar/python@3.11/3.11.5/Frameworks/Python.framework/Versions/3.11/lib/python3.11/_strptime.py"", line 349, in _strptime
    raise ValueError(""time data %r does not match format %r"" %
ValueError: time data '2014-11-21T11:44:12.000Z' does not match format '%Y%m%dT%H%M%SZ'
```

enex is exported by evernote mac client ",303218369,evernote-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/16/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
273660425,MDU6SXNzdWUyNzM2NjA0MjU=,84,datasette package --metadata does not work with a relative path,9599,simonw,closed,0,,,,,0,2017-11-14T04:00:50Z,2017-11-15T05:18:35Z,2017-11-15T05:18:35Z,OWNER,,"    $ datasette package ~/parlgov-db/parlgov.db --metadata=~/parlgov-db/parlgov.json
    Usage: datasette package [OPTIONS] FILES...

    Error: Invalid value for ""-m"" / ""--metadata"": Could not open file: ~/parlgov-db/parlgov.json: No such file or directory
    
    simonw-07542:~ simonw$ cd ~/parlgov-db/
    simonw-07542:parlgov-db simonw$ datasette package ~/parlgov-db/parlgov.db --metadata=parlgov.json
    Sending build context to Docker daemon   4.46MB
    Step 1/7 : FROM python:3
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/84/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
488833975,MDU6SXNzdWU0ODg4MzM5NzU=,3,Command for running a search and saving tweets for that search,9599,simonw,closed,0,,,,,6,2019-09-03T21:29:56Z,2019-11-04T05:31:56Z,2019-11-04T05:31:16Z,MEMBER,,    $ twitter-to-sqlite search dogsheep,206156866,twitter-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/3/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
267513424,MDU6SXNzdWUyNjc1MTM0MjQ=,1,Addressable pages for every row in a table,9599,simonw,closed,0,,,2857392,Ship first public release,6,2017-10-23T00:44:16Z,2017-10-24T14:11:04Z,2017-10-24T14:11:03Z,OWNER,,"    /database-name-7sha256/table-name/compound-pk
    /database-name-7sha256/table-name/compound-pk.json

Tricky part will be figuring out what the private key is - especially since it could be a compound primary key and it might involve different data types.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
718540751,MDU6SXNzdWU3MTg1NDA3NTE=,1012,For 1.0 update trove classifier in setup.py,9599,simonw,open,0,,,3268330,Datasette 1.0,5,2020-10-10T05:52:08Z,2021-11-16T13:18:36Z,,OWNER,,    Development Status :: 5 - Production/Stable,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1012/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
275939188,MDU6SXNzdWUyNzU5MzkxODg=,143,"Mechanism for ""suggested visualizations""",9599,simonw,closed,0,,,,,1,2017-11-22T04:10:25Z,2018-07-10T17:48:34Z,2018-07-10T17:48:34Z,OWNER,," Each visualization should have a way of deciding if it might be appropriate for the current view of data. 

 We can then offer a ""suggested visualizations"" prompt which shows previews.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/143/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
280745746,MDU6SXNzdWUyODA3NDU3NDY=,171,HTML comments specifying custom templates for page,9599,simonw,closed,0,,,2949431,Custom templates edition,1,2017-12-09T19:11:13Z,2017-12-09T21:50:50Z,2017-12-09T21:48:03Z,OWNER,," This would make the custom templating system self-documenting, and save people from having to figure out the right template names for customizing specific pages.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/171/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
737394470,MDU6SXNzdWU3MzczOTQ0NzA=,1084,Table/database action menu cut off if too short ,9599,simonw,closed,0,,,6055094,Datasette 0.52,4,2020-11-06T01:55:23Z,2020-11-21T23:45:59Z,2020-11-21T23:45:59Z,OWNER,,"![3CC0C181-959E-4B20-BE39-806ED93E833E](https://user-images.githubusercontent.com/9599/98316836-03891800-1f90-11eb-9e52-5266baf33296.jpeg)
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1084/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
710506708,MDU6SXNzdWU3MTA1MDY3MDg=,978,Rendering glitch with column headings on mobile,9599,simonw,closed,0,,,5971510,Datasette 0.50,6,2020-09-28T19:04:45Z,2020-10-08T23:54:40Z,2020-09-28T22:43:01Z,OWNER,,"![6348FED5-1310-47DA-BF54-810E63C96C6F](https://user-images.githubusercontent.com/9599/94474607-bc368d00-0182-11eb-9bb1-9a0e455353f1.jpeg)

https://latest-with-plugins.datasette.io/fixtures?sql=select%0D%0A++dateutil_parse%28%2210+october+2020+3pm%22%29%2C%0D%0A++dateutil_easter%28%222020%22%29%2C%0D%0A++dateutil_parse_fuzzy%28%22This+is+due+10+september%22%29%2C%0D%0A++dateutil_parse%28%221%2F2%2F2020%22%29%2C%0D%0A++dateutil_parse%28%222020-03-04%22%29%2C%0D%0A++dateutil_parse_dayfirst%28%222020-03-04%22%29%2C%0D%0A++dateutil_easter%282020%29",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/978/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
913900374,MDU6SXNzdWU5MTM5MDAzNzQ=,1369,Don't show foreign key IDs twice if no label,9599,simonw,open,0,,,,,1,2021-06-07T19:47:02Z,2021-06-07T19:47:24Z,,OWNER,,"![B5B54D94-A768-4544-A88D-CDCAB417CD3C](https://user-images.githubusercontent.com/9599/121078979-6e9d0600-c78e-11eb-8b70-20e6d29b48b1.jpeg)
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1369/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1469015001,I_kwDOBm6k_c5Xj2PZ,1916,GET requests against POST endpoints should not 500 error,9599,simonw,closed,0,,,7867486,Datasette 1.0a1,1,2022-11-30T04:04:43Z,2022-11-30T05:15:19Z,2022-11-30T05:15:19Z,OWNER,,"![CF37BA4D-0677-4DDD-A339-EAF163BB63B7](https://user-images.githubusercontent.com/9599/204705025-6f88e9f7-757d-45e8-a89c-ab97e84781e8.png)
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1916/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1564769997,I_kwDOBm6k_c5dRH7N,2011,"Applied facet did not result in an ""x"" icon to dismiss it",9599,simonw,open,0,,,,,1,2023-01-31T17:57:44Z,2023-01-31T17:58:54Z,,OWNER,,"![CleanShot 2023-01-31 at 09 55 56@2x](https://user-images.githubusercontent.com/9599/215843684-1761a230-d490-4f87-be6d-186319366794.png)

That's against this data https://data.sfgov.org/City-Management-and-Ethics/Supplier-Contracts/cqi5-hm2d imported using https://datasette.io/plugins/datasette-socrata

It's for `Contract Type` of `Non-Purchasing Contract (Rents, etc.)` - so possible that some of the spaces or punctuation in either the name of the value tripped up the code that decides if the X icon should be displayed.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2011/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1424378012,I_kwDOBm6k_c5U5kic,1860,SQL query field can't begin by a comment,562352,CharlesNepote,closed,0,,,,,12,2022-10-26T16:55:31Z,2022-10-27T18:57:37Z,2022-10-27T04:21:40Z,NONE,,"![image](https://user-images.githubusercontent.com/562352/198085197-f26fcd61-4dac-4ca4-a346-e70f88a30ecc.png)

SQL comments are **very** useful to explain the meaning of the query. It's currently impossible to put it at the beginning of the field as seen on the screen capture: it leads to an error: `Statement must be a SELECT`.

It would be great to make it possible because:
* as the request is the title of the page:
  * it eases the search with search engines
  * it eases the search in the browsers' url field
* it acts as a kind of title: the global meaning of the query is immediately understandable
* some tools, such as Slack, are shortening long URLs and displaying the beginning of the URLs (eg. `https://example.org/products?sql=select+%28length%28data_quality_errors_ta[...]+%21%3D+%22%22+group+by+NB_of_issues+order+by+NB_of_issues+desc+limit+200`)

Beginning a query with a comment is possible with SQLite.
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1860/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1578790070,I_kwDOCGYnMM5eGmy2,527,`Table.convert()` skips falsey values,167893,mcarpenter,closed,0,,,,,5,2023-02-10T00:00:52Z,2023-05-09T21:15:05Z,2023-05-08T21:03:24Z,CONTRIBUTOR,,"# Summary

By design, `Table.convert()` does [not attempt](https://github.com/simonw/sqlite-utils/blob/fc221f9b62ed8624b1d2098e564f525c84497969/sqlite_utils/db.py#L2663) conversion of falsey values (`None`, `""""`, `0`, ...). This is surprising (directly contradicts the docstring) and `convert()` may quietly skip cells where the user assumed a conversion would take place. 

# Example
Increment a column of integers by one

``` python
from sqlite_utils import Database

db = Database(memory=True)
table = db['table']
col = 'x'
table.insert_all([{col: 0}, {col:1}])
print(table.get(1)) # 0
print(table.get(2)) # 1
print()

table.convert(col, lambda x: x+1)
print(table.get(1)) # got 0, expected 1 ⚠⚠⚠
print(table.get(2)) # got 2, expected 2
```

Another example might be, say, transforming cells containing empty string to `NULL`.

# Discussion

This was, I think, a pragmatic choice so that consumers can skip writing guard clauses for these falsey values (particularly from the CLI). But this surprising undocumented behavior can lead to incorrect data. I don't think this is a good trade-off between convenience and correctness.

In the absence of this convenience users will either have to write guard clauses into their conversion expressions (or adapt the called function to do the same), so: 
``` python
    fn(value) if value else value
```
instead of:
``` python
    fn(value)
```
This is more typing and sometimes I will forget, and there will be errors. (But they will be noisy errors, which is a good thing).

Such a change will certainly inconvenience some existing consumers; there will be some breakage. But I think this is worth it to avoid quietly not converting some values by default, which can lead to quietly bad data.

I have a PR that I will attach, please take a look and see what you think.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/527/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
611284481,MDU6SXNzdWU2MTEyODQ0ODE=,38,[Feature Request] Support Repo Name in Search 🥺,5779832,zzeleznick,closed,0,,,,,4,2020-05-02T22:08:51Z,2020-05-03T02:34:32Z,2020-05-02T23:15:11Z,NONE,,"## Description
Per your [v2.2 release tweet](https://twitter.com/simonw/status/1256700238099693568) I played with the demo, but the output did not match my expectations.

## Expected Behavior
Expected a search query for ""twitter"" contained within the `repo` column to return non-zero results.

## Actual Behavior
😭  [0 rows where repo contains ""twitter"" sorted by starred_at descending](https://github-to-sqlite.dogsheep.net/github/stars?repo__contains=twitter&_sort_desc=starred_at) 

## Best Explanation
Per the table schema (see appendix) `repo` is of type `INTEGER` which built from `repo_id` and does not expose the repo name in search.

## Desired Behavior
Given that searching for ""206156866"" is less intuitive than ""twitter"",  it would be great to support this via extending the search capabilities or by adding an additional column.

✅ 104 rows where repo contains ""twitter""
❌  [104 rows where repo contains ""206156866"" sorted by starred_at descending](https://github-to-sqlite.dogsheep.net/github/stars?repo__contains=206156866&_sort_desc=starred_at) 

## Appendix
```
CREATE TABLE [stars] (
   [user] INTEGER REFERENCES [users]([id]),
   [repo] INTEGER REFERENCES [repos]([id]),
   [starred_at] TEXT,
   PRIMARY KEY ([user], [repo])
);
CREATE INDEX [idx_stars_repo]
                ON [stars] ([repo]);
CREATE INDEX [idx_stars_user]
                ON [stars] ([user]);
```",207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/38/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1198822563,I_kwDOBm6k_c5HdJSj,1706,"[feature] immutable mode for a directory, not just individual sqlite file",9020979,hydrosquall,open,0,,,,,4,2022-04-10T00:50:57Z,2022-12-09T19:11:40Z,,CONTRIBUTOR,,"## Motivation

- I have a directory of sqlite databases
- I'd like to use immutable mode when opening them for better performance [docs](https://docs.datasette.io/en/0.54/performance.html#immutable-mode)
- Currently using this flag throws the following error

    IsADirectoryError: [Errno 21] Is a directory: '/name-of-directory'

## Proposal

Immutable flag works for both single files and directories

    datasette -i /folder-of-sqlite-files",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1706/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1182227211,I_kwDOBm6k_c5Gd1sL,1692,[plugins][feature request]: Support additional script tag attributes when loading custom JS,9020979,hydrosquall,open,0,,,,,2,2022-03-27T01:16:03Z,2022-03-30T06:14:51Z,,CONTRIBUTOR,,"## Motivation

- The build system for my new [plugin](https://github.com/hydrosquall/datasette-nteract-data-explorer) has two output JS files, one for browsers that support ES modules, one for browsers that don't. At present, I'm only passing one of them into Datasette.
- I'd like to specify the non-es-module script as a fallback for older browsers. I don't want to load it by default, because browsers will only need one, and it's heavy, so for now I'm only supporting modern browsers. 

To be able to support legacy browsers without slowing down users with modern browsers, I would like to be able to set additional HTML attributes on the tag fallback script, `nomodule` and `defer`. My injected scripts should look something like this:

```html
<script type=""module"" src=""/index.my-es-module-bundle.js""></script>
<script src=""/index.my-legacy-fallback-bundle.js"" nomodule="""" defer></script>
```

## Proposal

To achieve this, I propose additional optional properties to the API accepted by the `extra_js_urls` hook and custom JS field the `metadata.json` [described here](https://docs.datasette.io/en/stable/custom_templates.html#custom-css-and-javascript). 

Under this API, I'd write something like this to get the above HTML rendered in Datasette.

```json
{
    ""extra_js_urls"": [
        {
            ""url"": ""/index.my-es-module-bundle.js"",
            ""module"": true,
        },
        {
            ""url"": ""/index.my-legacy-fallback-bundle.js"",
            ""nomodule"": """",
            ""defer"": true
        }
    ]
}
```

## Resources

- [MDN on the script tag](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/script)
  - There may be other properties that could be added that are potentially valuable, like `async` or `referrerpolicy`, but I don't have an immediate need for those.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1692/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1211283427,I_kwDODFdgUs5IMrfj,72,feature: display progress bar when downloading multi-page responses,9020979,hydrosquall,open,0,,,,,1,2022-04-21T16:37:12Z,2022-04-21T17:29:31Z,,NONE,,"## Motivation

For a long running command (longer than 1 minute) for a big table (like pull requests or commits), it can be tricky to know if the script is still running, or if a rate limit/error was encountered

We know how many pages there are, so it may be possible to indicate how many remain.

https://github.com/dogsheep/github-to-sqlite/blob/a6e237f75a4b86963d91dcb5c9582e3a1b3349d6/github_to_sqlite/utils.py#L367

## Resources

- Using the existing Click API: 
   - https://click.palletsprojects.com/en/5.x/utils/#showing-progress-bars
- Loading spinner: https://github.com/pavdmyt/yaspin
- Progress bar: https://github.com/tqdm/tqdm",207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/72/reactions"", ""total_count"": 3, ""+1"": 3, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1109884720,I_kwDOBm6k_c5CJ38w,1609,"Ensure ""pip install datasette"" still works with Python 3.6",9599,simonw,closed,0,,,,,12,2022-01-21T00:08:10Z,2022-01-24T19:20:09Z,2022-01-21T02:24:13Z,OWNER,,"## Original title: Can I keep ""pip install datasette"" working on Python 3.6?

I dropped support for 3.6 in:
- #1577

I'm getting reports that `pip3 install datasette` throws an error on that Python, even though I haven't made that new release yet - presumably due to lack of pinning of Uvicorn: https://twitter.com/ldodds/status/1484289475195080706

Is it possible to get `pip` on that version of Python to install the highest possible version of the packages that are still known to support Python 3.6?

If so, how?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1609/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1339663518,I_kwDOBm6k_c5P2aSe,1784,"Include ""entrypoint"" option on `--load-extension`?",15178711,asg017,closed,0,,,,,2,2022-08-16T00:22:57Z,2022-08-23T18:34:31Z,2022-08-23T18:34:31Z,CONTRIBUTOR,,"## Problem 

SQLite extensions have the option to define multiple ""entrypoints"" in each loadable extension. For example, the upcoming version of `sqlite-lines` will have 2 entrypoints: the default `sqlite3_lines_init` (which SQLite will automatically guess for) and `sqlite3_lines_noread_init`. The `sqlite3_lines_noread_init` version omits functions that read from the filesystem, which is necessary for security purposes when running untrusted SQL (which Datasette does).

(Similar multiple entrypoints will also be added for sqlite-http).



The `--load-extension` flag, however, doesn't give the option to specify a different entrypoint, so the default one is always used. 

## Proposal

I want there to be a new command line option of the `--load-extension` flag to specify a custom entrypoint like so:
```
datasette my.db \
  --load-extension ./lines0 sqlite3_lines0_noread_init
```

Then, under the hood, this line of code:

https://github.com/simonw/datasette/blob/7af67b54b7d9bca43e948510fc62f6db2b748fa8/datasette/app.py#L562

Would look something like this:

```python
 conn.execute(""SELECT load_extension(?, ?)"", [extension, entrypoint]) 
```

One potential problem: For backward compatibility, I'm not sure if Click allows cli flags to have variable number of options (""arity""). So I guess it could also use a `:` delimiter like `--static`:

```
datasette my.db \
  --load-extension ./lines0:sqlite3_lines0_noread_init
```

Or maybe even a new flag name?

```
datasette my.db \
  --load-extension-entrypoint ./lines0 sqlite3_lines0_noread_init
```


Personally I prefer the `:` option... and maybe even `--load-extension` -> `--load`? Definitely out of scope for this issue tho

```
datasette my.db \
  --load./lines0:sqlite3_lines0_noread_init
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1784/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1698865182,I_kwDOBm6k_c5lQqAe,2069,[BUG] Cannot insert new data to deployed instance,31861128,yqlbu,open,0,,,,,1,2023-05-07T02:59:42Z,2023-05-07T03:17:35Z,,NONE,,"## Summary

Recently, I deployed an instance of datasette to Vercel with the following plugins:

- datasette-auth-tokens
- datasette-insert

With the above plugins, I was able to insert new data to local sqlite db. However, when it comes to the deployment on Vercel, things behave differently. I observed some errors from the logs console on Vercel:

```console
File ""/var/task/datasette/database.py"", line 179, in _execute_writes
conn = self.connect(write=True)
File ""/var/task/datasette/database.py"", line 93, in connect
assert not (write and not self.is_mutable)
AssertionError
``` 

<img width=""1380"" alt=""image"" src=""https://user-images.githubusercontent.com/31861128/236655274-3fd5e16f-4e06-4ba8-87ff-41130b8ebe90.png"">

I think it is a potential bug.

## Reproduce

<details><summary>metadata.json</summary>
</br>

```json
{
  ""plugins"": {
    ""datasette-insert"": {
      ""allow"": {
        ""id"": ""*""
      }
    },
    ""datasette-auth-tokens"": {
      ""tokens"": [
        {
          ""token"": {
            ""$env"": ""INSERT_TOKEN""
          },
          ""actor"": {
            ""id"": ""repeater""
          }
        }
      ],
      ""param"": ""_auth_token""
    }
  }
}
```

</details>

<details><summary>commands</summary>
</br>

```bash
# deploy
datasette publish vercel remote.db \
  --project=repeater-bot-sqlite \
  --metadata metadata.json \
  --install datasette-auth-tokens \
  --install datasette-insert \
  --vercel-json=vercel.json

# test insert
cat fixtures/dogs.json | curl --request POST -d @- -H ""Authorization: Bearer <token>"" \
  'https://repeater-bot-sqlite.vercel.app/-/insert/remote/dogs?pk=id'
```

</details>

<details><summary>logs</summary>
</br>

```console
Traceback (most recent call last):
File ""/var/task/datasette/app.py"", line 1354, in route_path
response = await view(request, send)
File ""/var/task/datasette/app.py"", line 1500, in async_view_fn
response = await async_call_with_supported_arguments(
File ""/var/task/datasette/utils/__init__.py"", line 1005, in async_call_with_supported_arguments
return await fn(*call_with)
File ""/var/task/datasette_insert/__init__.py"", line 14, in insert_or_upsert
response = await insert_or_upsert_implementation(request, datasette)
File ""/var/task/datasette_insert/__init__.py"", line 91, in insert_or_upsert_implementation
table_count = await db.execute_write_fn(write_in_thread, block=True)
File ""/var/task/datasette/database.py"", line 167, in execute_write_fn
raise result
File ""/var/task/datasette/database.py"", line 179, in _execute_writes
conn = self.connect(write=True)
File ""/var/task/datasette/database.py"", line 93, in connect
assert not (write and not self.is_mutable)
AssertionError
```

</details>",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2069/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1575131737,I_kwDOCGYnMM5d4ppZ,525,Repeated calls to `Table.convert()` fail,167893,mcarpenter,closed,0,,,,,4,2023-02-07T22:40:47Z,2023-05-08T21:59:41Z,2023-05-08T21:54:02Z,CONTRIBUTOR,,"## Summary
When using the API, repeated calls to `Table.convert()`  do not work correctly since all conversions quietly use the callable (function, lambda) from the first call to `convert()` only. Subsequent invocations with different callables use the callable from the first invocation only.

## Example
```python
from sqlite_utils import Database

db = Database(memory=True)
table = db['table']
col = 'x'
table.insert_all([{col: 1}])
print(table.get(1))

table.convert(col, lambda x: x*2)
print(table.get(1))

def zeroize(x):
    return 0
#zeroize = lambda x: 0
#zeroize.__name__ = 'zeroize'
table.convert(col, zeroize)
print(table.get(1))
```

Output:
```
{'x': 1}
{'x': 2}
{'x': 4}
```
Expected:
```
{'x': 1}
{'x': 2}
{'x': 0}
```

## Explanation
This is some relevant [documentation](https://github.com/simonw/sqlite-utils/blob/1491b66dd7439dd87cd5cd4c4684f46eb3c5751b/docs/python-api.rst#registering-custom-sql-functions:~:text=By%20default%20registering%20a%20function%20with%20the%20same%20name%20and%20number%20of%20arguments%20will%20have%20no%20effect).

 * `Table.convert()` takes a `Callable` to perform data conversion on a column
 * The `Callable` is passed to `Database.register_function()`
 * `Database.register_function()` uses the callable's `__name__` attribute for registration
 * (Aside: all lambdas have a `__name__` of `<lambda>`: I thought this was the problem, and it was close, but not quite)
 * However `convert()` first wraps the callable by local function [`convert_value()`](https://github.com/simonw/sqlite-utils/blob/fc221f9b62ed8624b1d2098e564f525c84497969/sqlite_utils/db.py#L2661)
 * Consequently `register_function()` sees name `convert_value` for all invocations from `convert()`
 * `register_function()` silently ignores registrations using the same name, retaining only the first such registration

There's a mismatch between the comments and the code: https://github.com/simonw/sqlite-utils/blob/fc221f9b62ed8624b1d2098e564f525c84497969/sqlite_utils/db.py#L404

but actually the existing function is returned/used instead (as the ""registering custom sql functions"" doc I linked above says too). Seems like this can be rectified to match the comment?

## Suggested fix
I think there are four things:
1. The call to `register_function()` from `convert()`should have an explicit `name=` parameter (to continue using `convert_value()` and the progress bar).
2. For functions, this name can be the real function name. (I understand the sqlite api needs a name, and it's nice if those are recognizable names where possible). For lambdas would `'lambda-{uuid}'` or similar be acceptable? 
3. `register_function()` really should throw an error on repeated attempts to register a duplicate (function, arity)-pair.
4. A test? I haven't looked at the test framework here but seems this should be testable.

## See also 
- #458 ",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/525/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1154399841,I_kwDOBm6k_c5Ezr5h,1645,"Sensible `cache-control` headers for static assets, including those served by plugins",697092,curiousleo,open,0,,,3268330,Datasette 1.0,4,2022-02-28T18:12:03Z,2022-03-08T02:59:29Z,,NONE,,"## What I'm seeing

With `default_cache_ttl = 86400`, I see the following:

A table view returns `Cache-control: max-age=86400`:

![Screenshot_20220228_190000](https://user-images.githubusercontent.com/697092/156034352-4d64683e-39c8-49af-81df-0217a5957bbd.png)

A static asset returns no `Cache-control` header:

![Screenshot_20220228_185933](https://user-images.githubusercontent.com/697092/156034363-d0b03cc2-5889-4ed2-b601-8c1846b8469a.png)

## What I expected to see

I expected the static asset to return a `Cache-control` header indicating that this response can be cached.

## Why this matters

I'm productionising a Datasette deployment right now and was looking into putting it behind a Varnish instance. I was surprised to see requests for static assets being served from Datasette rather than Varnish, this is what led me to look more closely at the response headers.

While Datasette serves those static assets pretty quickly, I don't see why Datasette should serve them. By their nature, static assets like images and JS files are very cacheable, so it should be easy to serve them from a cache like Varnish.

(Note that Varnish can easily be configured to override this header, enabling caching for static assets. But it would be better if this override was not necessary.)

## Discussion

It seems clear to me that serving static assets without a `Cache-control` header is not ideal.

I see two options here:

A. Static assets use the same logic as table / SQL views to set the `Cache-control` header based on `default_cache_ttl`.
B. An additional setting for static assets is introduced (`default_static_cache_ttl`, say).",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1645/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1978603203,I_kwDOCGYnMM517xbD,602,`sqlite-utils transform` removes the `AUTOINCREMENT` keyword,4472046,ArsTapatun,open,0,,,,,0,2023-11-06T08:48:43Z,2023-11-06T08:48:43Z,,NONE,,"### Context

We ran into this bug randomly, noticing that deleted `ROWID` would get reused after migrating the DB. Using `transform` to change any column in the table will also unexpectedly strip away the `AUTOINCREMENT` keyword from the primary key definition, even if it was not the transformation target.

### Reproducible example

**Original database**

```sql
$ sqlite3 test.db << EOF
CREATE TABLE mytable (
    col1 INTEGER PRIMARY KEY AUTOINCREMENT,
    col2 TEXT NOT NULL
)
EOF

$ sqlite3 test.db "".schema mytable""
CREATE TABLE mytable (
    col1 INTEGER PRIMARY KEY AUTOINCREMENT,
    col2 TEXT NOT NULL
);
```

**Modified database after sqlite-utils**

```sql
$ sqlite-utils transform test.db mytable --rename col2 renamedcol2

$ sqlite3 test.db ""SELECT sql FROM sqlite_master WHERE name = 'mytable';""
CREATE TABLE IF NOT EXISTS ""mytable"" (
   [col1] INTEGER PRIMARY KEY,
   [renamedcol2] TEXT NOT NULL
);
```",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/602/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1161937073,I_kwDOBm6k_c5FQcCx,1653,Mechanism to default a table to sorting by multiple columns,9599,simonw,open,0,,,,,2,2022-03-07T21:20:11Z,2022-03-07T21:23:39Z,,OWNER,,"### Discussed in https://github.com/simonw/datasette/discussions/1652

<div type='discussions-op-text'>

<sup>Originally posted by **zaneselvans** March  7, 2022</sup>
It's easy to tell datasette to sort tables using a single column, as [described in the docs](https://docs.datasette.io/en/stable/metadata.html#setting-a-default-sort-order):

```yaml
databases:
  ferc1:
    tables:
      f1_edcfu_epda:
        sort: created_time
```

But is there some way to tell it to sort using a composite key, like you would in an `ORDER BY` clause instead? For example, the way it's being done **[in this query](https://data.catalyst.coop/ferc1?sql=select%0D%0A++rowid%2C%0D%0A++respondent_id%2C%0D%0A++report_year%2C%0D%0A++spplmnt_num%2C%0D%0A++row_number%2C%0D%0A++row_seq%2C%0D%0A++row_prvlg%2C%0D%0A++acct_num%2C%0D%0A++depr_plnt_base%2C%0D%0A++est_avg_srvce_lf%2C%0D%0A++net_salvage%2C%0D%0A++apply_depr_rate%2C%0D%0A++mrtlty_crv_typ%2C%0D%0A++avg_remaining_lf%2C%0D%0A++report_prd%0D%0Afrom%0D%0A++f1_edcfu_epda%0D%0Awhere%0D%0A++respondent_id+%3D+210%0D%0A++AND+report_year+%3D+2020%0D%0Aorder+by%0D%0A++report_year%2C+report_prd%2C+respondent_id%2C+spplmnt_num%2C+row_number%0D%0Alimit%0D%0A++1000)** on our Datasette?

```sql
SELECT
  respondent_id,
  report_year,
  spplmnt_num,
  row_number,
  row_seq,
  row_prvlg,
  acct_num,
  depr_plnt_base,
  est_avg_srvce_lf,
  net_salvage,
  apply_depr_rate,
  mrtlty_crv_typ,
  avg_remaining_lf,
  report_prd
FROM
  f1_edcfu_epda
WHERE
  respondent_id = 210
  AND report_year = 2020
ORDER BY
  report_year, report_prd, respondent_id, spplmnt_num, row_number
LIMIT
  1000
```

The problem here is that by default it's using `rowid` (the SQLite assigned autoincrementing integer key) to order the records, but the table **should** have a natural composite primary key, but the original database that this data is being migrated from doesn't enforce unique primary keys, so there are dupes, and we don't want to drop those rows, and the records are somehow getting jumbled in the database (the `rowid` ordering isn't lined up with the expected ordering based on the composite primary key, though it's close) and this jumbling is confusing to users that expect to see the data ordered based on the natural primary key.

I've tried setting the `sort` metadata parameter to a list of column names, a tuple of column names, a quoted string of comma-separated column names, a quoted string of a tuple of column names...

```yaml
databases:
  ferc1:
    tables:
      f1_edcfu_epda:
        sort: ""(report_year, report_prd, respondent_id, spplmnt_num, row_number)""
```

and they all give me server errors like:

```
Cannot sort table by (report_year, report_prd, respondent_id, spplmnt_num, row_number)
```</div>",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1653/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
559964149,MDU6SXNzdWU1NTk5NjQxNDk=,665,Introduce a SQL statement parser in Python,9599,simonw,open,0,,,,,1,2020-02-04T20:36:05Z,2020-02-04T20:36:48Z,,OWNER,,#254 and #653 are both examples of problems that could be solved using a real SQL parser in Python.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/665/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
323830051,MDU6SXNzdWUzMjM4MzAwNTE=,270,--limit= CLI option for setting limits,9599,simonw,closed,0,,,,,1,2018-05-17T00:14:24Z,2018-05-18T06:19:31Z,2018-05-18T06:16:39Z,OWNER,,"#264 calls for four new datasette limit options, on top of the two existing ones:

* `--max_returned_rows`
* `--sql_time_limit_ms`

These are already clogging up `datasette serve --help` a bit.

How about this syntax instead?

    datasette --limit max_returned_rows:100 \
        --limit facet_timeout_ms:500 demo.db

Then we can add as many new user over-rideable limits as we like without clogging up `--help` too much - though it would be good to have a way of optionally listings their documentation as well.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/270/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
324652142,MDU6SXNzdWUzMjQ2NTIxNDI=,274,"Rename --limit to --config, add --help-config",9599,simonw,closed,0,,,,,2,2018-05-19T18:57:42Z,2018-05-20T17:04:55Z,2018-05-20T17:04:11Z,OWNER,,"#270 introduced `--limit` but on further thought it should be called `--config` instead.

`--page_size` should becomes `--config default_page_size:1000`

Add `--help-config` to show full help showing all config settings.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/274/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
976399638,MDU6SXNzdWU5NzYzOTk2Mzg=,319,[Enhancement] Please allow 'insert-files' to insert content as text.,66709385,pjamargh,closed,0,,,,,10,2021-08-22T15:10:46Z,2021-08-24T23:33:45Z,2021-08-24T23:33:44Z,NONE,,"'insert-files' creates BLOB columns for file contents. Transforming the column to TEXT still keep the content as binary. Even though I'm sure there is a transform that can be applied decoding the text it would be great to have a argument to make 'insert-files' to do it as text (with optional text encoding).

The use case is a bunch of htmls (single file) on a directory structure that inserted with this command could be served in Datasette allowing full text search.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/319/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1251710928,I_kwDOBm6k_c5Km5fQ,1751,Add scrollbars to table presentation in default layout,408765,knutwannheden,closed,0,,,,,1,2022-05-28T19:44:57Z,2022-05-28T19:52:17Z,2022-05-28T19:52:17Z,NONE,,"(As you will be able to tell from the terminology I use, I am not a frontend guy, but I hope you will understand.)

When a table is wide and needs horizontal scrolling to see the columns towards the end, the user needs to scroll horizontally. However, since the container for the HTML table (`div` with class `table-wrapper`) isn't limited by the window size, I first need to vertically scroll near to the bottom of the page in order to scroll horizontally. Then I can scroll back up again. This isn't very user friendly. Instead, I think it would make sense to constrain the table's size (when necessary), so that the vertical and horizontal scrollbars either always are visible or at least not far out of reach.

I understand that I could provide my own template and / or CSS, but I think it would probably make sense to adjust the default in this regard.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1751/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
853672224,MDU6SXNzdWU4NTM2NzIyMjQ=,1294,"""You can check out any time you like. But you can never leave!""",192568,mroswell,open,0,,,,,0,2021-04-08T17:02:15Z,2021-04-08T18:35:50Z,,CONTRIBUTOR,,"(Feel free to rename this one.)

- The column gear lets you ""Show not-blank rows."" Then it places a parameter in the URL, which a web developer would notice, but a lot of users won't notice, or know to delete it. Would be good to toggle ""Show not-blank rows"" with ""Show all rows."" (Also would be quite helpful to have a ""Show blank rows | Show all rows"" option)
- The column gear lets you ""Sort ascending"" and ""Sort descending"" but then you're stuck with some sort of sorted version thereafter, unless you know to sort the ID column, or to remove the full _sort parameter and its value in the URL. Would be good to offer a ""Remove sort"" option in the gear.
- These requests are in the same camp as: https://github.com/simonw/datasette-vega/issues/36
- I suspect there are other url parameter instances where similar analysis would be helpful, but the three above are the use cases I've run across. 

UPDATE:
- It would be helpful to have a ""Previous page"" available for all but the first table page.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1294/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1465194249,I_kwDOCGYnMM5XVRcJ,514,upsert of new row with check constraints fails,193185,cldellow,closed,0,,,,,5,2022-11-26T16:12:23Z,2023-05-08T21:50:52Z,2023-05-08T21:50:51Z,NONE,,"(I originally opened this in https://github.com/simonw/datasette-insert/issues/20, but I see that that library depends on sqlite-utils)

In the case of a new row, upsert first adds the row, specifying only its pkeys: https://github.com/simonw/sqlite-utils/blob/965ca0d5f5bffe06cc02cd7741344d1ddddf9d56/sqlite_utils/db.py#L2783-L2787

This means that a table with NON NULL (or other constraint) columns that aren't part of the pkey can't have new rows upserted.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/514/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
988556488,MDU6SXNzdWU5ODg1NTY0ODg=,1459,suggestion: allow `datasette --open` to take a relative URL,51016,ctb,open,0,,,,,1,2021-09-05T17:17:07Z,2021-09-05T19:59:15Z,,CONTRIBUTOR,,"(soft suggestion because I'm not sure I'm using datasette right yet)

Over at https://github.com/ctb/2021-sourmash-datasette, I'm playing around with datasette, and I'm creating some static pages to send people to the right facets. There may well be better ways of achieving this end goal, and I will find out if so, I'm sure!

But regardless I think it might be neat to support an option to allow `-o/--open` to take a relative URL, that then gets appended to the hostname and port. This would let me improve my documentation. I don't see any downsides, either, but 🤷 there may well be some :)

Happy to dig in and provide a PR if it's of interest. I'm not sure off the top of my head how to support an optional value to a parameter in argparse - the current `-o` behavior is kinda nice so it'd be suboptimal to require a url for `-o`. Maybe `--open-url=` or something would work?

",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1459/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1538197093,I_kwDOBm6k_c5brwZl,1995,foreign_keys error 500,137183,jonschoning,open,0,,,,,0,2023-01-18T15:27:36Z,2023-01-18T16:44:01Z,,NONE,,"**Error 500 expected string or bytes-like object**

[espial-new.sqlite3.zip](https://github.com/simonw/datasette/files/10447965/espial-new.sqlite3.zip)

run `datasette espial-new.sqlite3` & click on any table other than `User`

```
/home/jon/.local/lib/python3.10/site-packages/datasette/app.py:814 in           │
│ expand_foreign_keys                                                             │
│                                                                                 │
│    811 │   │   │   from {other_table}                                           │
│    812 │   │   │   where {other_column} in ({placeholders})                     │
│    813 │   │   """""".format(                                                      │
│ ❱  814 │   │   │   other_column=escape_sqlite(fk[""other_column""]),              │
│    815 │   │   │   label_column=escape_sqlite(label_column),                    │
│    816 │   │   │   other_table=escape_sqlite(fk[""other_table""]),                │
│    817 │   │   │   placeholders="", "".join([""?""] * len(set(values))),            │
│                                                                                 │
│ ╭───────────────────────────── locals ──────────────────────────────╮           │
│ │       column = 'user_id'                                          │           │
│ │     database = 'espial-new'                                       │           │
│ │           db = <Database: espial-new (mutable, size=53248)>       │           │
│ │           fk = {                                                  │           │
│ │                │   'column': 'user_id',                           │           │
│ │                │   'other_table': 'user',                         │           │
│ │                │   'other_column': None                           │           │
│ │                }                                                  │           │
│ │ foreign_keys = [                                                  │           │
│ │                │   {                                              │           │
│ │                │   │   'column': 'user_id',                       │           │
│ │                │   │   'other_table': 'user',                     │           │
│ │                │   │   'other_column': None                       │           │
│ │                │   }                                              │           │
│ │                ]                                                  │           │
│ │ label_column = 'name'                                             │           │
│ │  labeled_fks = {}                                                 │           │
│ │         self = <datasette.app.Datasette object at 0x7f0f2e77e980> │           │
│ │        table = 'bookmark'                                         │           │
│ │       values = []                                                 │           │
│ ╰───────────────────────────────────────────────────────────────────╯           │
│                                                                                 │
│ /home/jon/.local/lib/python3.10/site-packages/datasette/utils/__init__.py:346   │
│ in escape_sqlite                                                                │
│                                                                                 │
│    343                                                                          │
│    344                                                                          │
│    345 def escape_sqlite(s):                                                    │
│ ❱  346 │   if _boring_keyword_re.match(s) and (s.lower() not in reserved_words) │
│    347 │   │   return s                                                         │
│    348 │   else:                                                                │
│    349 │   │   return f""[{s}]""                                                  │
│                                                                                 │
│ ╭─ locals ─╮                                                                    │
│ │ s = None │                                                                    │
│ ╰──────────╯                                                                    │
╰─────────────────────────────────────────────────────────────────────────────────╯
TypeError: expected string or bytes-like object
Traceback (most recent call last):
  File ""/home/jon/.local/lib/python3.10/site-packages/datasette/app.py"", line 1354, in route_path
    response = await view(request, send)
  File ""/home/jon/.local/lib/python3.10/site-packages/datasette/views/base.py"", line 134, in view
    return await self.dispatch_request(request)
  File ""/home/jon/.local/lib/python3.10/site-packages/datasette/views/base.py"", line 91, in dispatch_request
    return await handler(request)
  File ""/home/jon/.local/lib/python3.10/site-packages/datasette/views/base.py"", line 361, in get
    response_or_template_contexts = await self.data(request, **data_kwargs)
  File ""/home/jon/.local/lib/python3.10/site-packages/datasette/views/table.py"", line 158, in data
    return await self._data_traced(request, default_labels, _next, _size)
  File ""/home/jon/.local/lib/python3.10/site-packages/datasette/views/table.py"", line 603, in _data_traced
    await self.ds.expand_foreign_keys(
  File ""/home/jon/.local/lib/python3.10/site-packages/datasette/app.py"", line 814, in expand_foreign_keys
    other_column=escape_sqlite(fk[""other_column""]),
  File ""/home/jon/.local/lib/python3.10/site-packages/datasette/utils/__init__.py"", line 346, in escape_sqlite
    if _boring_keyword_re.match(s) and (s.lower() not in reserved_words):
TypeError: expected string or bytes-like object
INFO:     127.0.0.1:38574 - ""GET /espial-new/bookmark HTTP/1.1"" 500 Internal Server Error
INFO:     127.0.0.1:38574 - ""GET /-/static/app.css?d59929 HTTP/1.1"" 200 OK
```

Schema:
```
CREATE TABLE IF NOT EXISTS ""user""
  (
     ""id""              INTEGER PRIMARY KEY,
     ""name""            VARCHAR NOT NULL,
     ""password_hash""   VARCHAR NOT NULL,
     ""api_token""       VARCHAR NULL,
     ""private_default"" BOOLEAN NOT NULL,
     ""archive_default"" BOOLEAN NOT NULL,
     ""privacy_lock""    BOOLEAN NOT NULL,
     CONSTRAINT ""unique_user_name"" UNIQUE (""name"")
  );

CREATE TABLE IF NOT EXISTS ""bookmark""
  (
     ""id""           INTEGER PRIMARY KEY,
     ""user_id""      INTEGER NOT NULL REFERENCES ""user"" ON DELETE RESTRICT ON UPDATE RESTRICT,
     ""slug""         VARCHAR NOT NULL DEFAULT (Lower(Hex(Randomblob(6)))),
     ""href""         VARCHAR NOT NULL,
     ""description""  VARCHAR NOT NULL,
     ""extended""     VARCHAR NOT NULL,
     ""time""         TIMESTAMP NOT NULL,
     ""shared""       BOOLEAN NOT NULL,
     ""to_read""      BOOLEAN NOT NULL,
     ""selected""     BOOLEAN NOT NULL,
     ""archive_href"" VARCHAR NULL,
     CONSTRAINT ""unique_user_href"" UNIQUE (""user_id"", ""href""),
     CONSTRAINT ""unique_user_slug"" UNIQUE (""user_id"", ""slug"")
  );

CREATE TABLE IF NOT EXISTS ""bookmark_tag""
  (
     ""id""          INTEGER PRIMARY KEY,
     ""user_id""     INTEGER NOT NULL REFERENCES ""user"" ON DELETE RESTRICT ON UPDATE RESTRICT,
     ""tag""         VARCHAR NOT NULL,
     ""bookmark_id"" INTEGER NOT NULL REFERENCES ""bookmark"" ON DELETE RESTRICT ON UPDATE RESTRICT,
     ""seq""         INTEGER NOT NULL,
     CONSTRAINT ""unique_user_tag_bookmark_id"" UNIQUE (""user_id"", ""tag"", ""bookmark_id""),
     CONSTRAINT ""unique_user_bookmark_id_tag_seq"" UNIQUE (""user_id"", ""bookmark_id"", ""tag"", ""seq"")
  );

CREATE TABLE IF NOT EXISTS ""note""
  (
     ""id""          INTEGER PRIMARY KEY,
     ""user_id""     INTEGER NOT NULL REFERENCES ""user"" ON DELETE RESTRICT ON UPDATE RESTRICT,
     ""slug""        VARCHAR NOT NULL DEFAULT (Lower(Hex(Randomblob(10)))),
     ""length""      INTEGER NOT NULL,
     ""title""       VARCHAR NOT NULL,
     ""text""        VARCHAR NOT NULL,
     ""is_markdown"" BOOLEAN NOT NULL,
     ""shared""      BOOLEAN NOT NULL DEFAULT false,
     ""created""     TIMESTAMP NOT NULL,
     ""updated""     TIMESTAMP NOT NULL
  );
CREATE INDEX idx_bookmark_time ON bookmark (user_id, time DESC);
CREATE INDEX idx_bookmark_tag_bookmark_id ON bookmark_tag (bookmark_id, id, tag, seq);
CREATE INDEX idx_note_user_created ON note (user_id, created DESC); 
```
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1995/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
807174161,MDU6SXNzdWU4MDcxNzQxNjE=,227,Error reading csv files with large column data,295329,camallen,closed,0,,,,,4,2021-02-12T11:51:47Z,2021-02-16T11:48:03Z,2021-02-14T21:17:19Z,NONE,,"*Feel free to close this issue - I mostly added it for reference for future folks that run into this :)*

I have a CSV file with one column that has very long strings.  When i try to import this file via the `insert` command I get the following error: 
```
sqlite-utils insert database.db table_name file_with_large_column.csv

Traceback (most recent call last):
  File ""/usr/local/bin/sqlite-utils"", line 10, in <module>
    sys.exit(cli())
  File ""/usr/local/lib/python3.7/site-packages/click/core.py"", line 829, in __call__
    return self.main(*args, **kwargs)
  File ""/usr/local/lib/python3.7/site-packages/click/core.py"", line 782, in main
    rv = self.invoke(ctx)
  File ""/usr/local/lib/python3.7/site-packages/click/core.py"", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File ""/usr/local/lib/python3.7/site-packages/click/core.py"", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File ""/usr/local/lib/python3.7/site-packages/click/core.py"", line 610, in invoke
    return callback(*args, **kwargs)
  File ""/usr/local/lib/python3.7/site-packages/sqlite_utils/cli.py"", line 774, in insert
    default=default,
  File ""/usr/local/lib/python3.7/site-packages/sqlite_utils/cli.py"", line 705, in insert_upsert_implementation
    docs, pk=pk, batch_size=batch_size, alter=alter, **extra_kwargs
  File ""/usr/local/lib/python3.7/site-packages/sqlite_utils/db.py"", line 1852, in insert_all
    first_record = next(records)
  File ""/usr/local/lib/python3.7/site-packages/sqlite_utils/cli.py"", line 703, in <genexpr>
    docs = (decode_base64_values(doc) for doc in docs)
  File ""/usr/local/lib/python3.7/site-packages/sqlite_utils/cli.py"", line 681, in <genexpr>
    docs = (dict(zip(headers, row)) for row in reader)
_csv.Error: field larger than field limit (131072)
```
Built with the docker image `datasetteproject/datasette:0.54` with the following versions:
```
# sqlite-utils --version
sqlite-utils, version 3.4.1

# datasette --version
datasette, version 0.54
```
It appears this is a [known issue](https://stackoverflow.com/a/54517228/2761423) reading in csv files in python and [doesn't look to be modifiable](https://github.com/python/cpython/blob/ea46579067fd2d4e164d6605719ffec690c4d621/Modules/_csv.c#L1685) through system / env vars (i may be very wrong on this).

Noting that using sqlite3 `import` command work without error (not using the python csv reader)
```
sqlite3 database.db
sqlite> .mode csv
sqlite> .import file_with_large_column.csv table_name
```
Sadly I couldn't see an easy way around this while using the cli as it appears this value needs to be changed in python code. FWIW I've switched to using https://datasette.io/tools/csvs-to-sqlite for importing csv data and it's working well. 

Finally, I'm loving https://datasette.io/ thank you very much for an amazing tool and data ecosytem 🙇‍♀️ ",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/227/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1114147905,I_kwDOBm6k_c5CaIxB,1612,Move canned queries closer to the SQL input area,639012,jsfenfen,closed,0,,,3268330,Datasette 1.0,5,2022-01-25T17:06:39Z,2022-03-19T04:04:49Z,2022-01-25T18:34:21Z,CONTRIBUTOR,,"*Original title:  Consider placing example queries above the sql input?*

Hi! Have been enjoying deploying ad hoc datasettes for collaborators to pick over! 

I keep finding myself manually ""fixing"" the database.html template so that the ""example queries"" (canned queries) appear directly *over* the sql box? So they are sorta more a suggestion for collaborators who aren't inclined to write their own queries? 

My sense is any time I go to the trouble of writing canned queries my users should see 'em? 

(( I have also considered a client-side reactive-ish option where selecting a query just places the raw SQL in the box and doesn't execute it, but this seems to end up being an inconvenience, rather than a teaching tool. )) 

",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1612/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
860625833,MDU6SXNzdWU4NjA2MjU4MzM=,1300,Make row available to `render_cell` plugin hook,3243482,abdusco,closed,0,,,,,5,2021-04-18T10:14:37Z,2022-07-07T16:34:05Z,2022-07-07T16:31:22Z,CONTRIBUTOR,,"*Original title: **Generating URL for a row inside `render_cell` hook***

Hey,
I am using Datasette to view a database that contains video metadata. It has BLOB columns that contain video thumbnails in JPG format (around 100-500KB per row). 

I've registered an output formatter that extends `datasette.blob_renderer.render_blob` function and serves the column with `image/jpeg` content type.

```python
from datasette.blob_renderer import render_blob

async def render_jpg(datasette, database, rows, columns, request, table, view_name):
    response = await render_blob(datasette, database, rows, columns, request, table, view_name)
    response.content_type = ""image/jpeg""
    response.headers[""Content-Disposition""] = f'inline; filename=""image.jpg""'
    return response


@hookimpl
def register_output_renderer():
    return {
        ""extension"": ""jpg"",
        ""render"": render_jpg,
        ""can_render"": lambda: True,
    }
```

This works well. I can visit `http://localhost:8001/mydb/videos/1.jpg?_blob_column=thumbnail` and view the image.

I want to display the image directly with an `<img>` tag (lazy-loaded of course). So, I need a URL, because embedding base64 would increase the page size too much (each image > 100KB). 

Datasette generates a link with `.blob` extension for blob columns. It does this by calling `datasette.urls.row_blob`

https://github.com/simonw/datasette/blob/7a2ed9f8a119e220b66d67c7b9e07cbab47b1196/datasette/views/table.py#L169-L179

But I have no way of getting the row inside the `render_cell` hook. 

```python
@hookimpl
def render_cell(value, column, table, database, datasette):
    if isinstance(value, bytes) and imghdr.what(None, value):
        # generate url
        return '$renderedLink'
```

Any pointers?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1300/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1126692066,I_kwDOCGYnMM5DJ_Ti,403,Document how to add a primary key to a rowid table using `sqlite-utils transform --pk`,536941,fgregg,closed,0,,,,,4,2022-02-08T01:39:40Z,2022-02-09T04:22:43Z,2022-02-08T19:33:59Z,CONTRIBUTOR,,"*Original title: Add option for adding a new, serial, primary key*

sometimes we have tables that don't have primary keys, but ought to have them. we *can* use rowid for that, but it would often be nicer to have an explicit primary key. using the current value of rowid would be fine.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/403/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1087913724,I_kwDOBm6k_c5A2D78,1577,Drop support for Python 3.6,9599,simonw,closed,0,,,3268330,Datasette 1.0,6,2021-12-23T18:17:03Z,2022-01-25T23:30:03Z,2022-01-20T04:31:41Z,OWNER,,"*Original title: Decide when to drop support for Python 3.6*

> `context_vars` can solve this but they were introduced in Python 3.7: https://www.python.org/dev/peps/pep-0567/
>
> Python 3.6 support ends in a few days time, and it looks like Glitch has updated to 3.7 now - so maybe I can get away with Datasette needing 3.7 these days?
>
> Tweeted about that here: https://twitter.com/simonw/status/1473761478155010048

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1576#issuecomment-999878907_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1577/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
816560819,MDU6SXNzdWU4MTY1NjA4MTk=,240,table.pks_and_rows_where() method returning primary keys along with the rows,9599,simonw,closed,0,,,,,7,2021-02-25T15:49:28Z,2021-02-25T16:39:23Z,2021-02-25T16:28:23Z,OWNER,,"*Original title: Easier way to update a row returned from .rows*

Here's a surprisingly hard problem I ran into while trying to implement #239 - given a row returned by `db[table].rows` how can you update that row?

The problem is that the `db[table].update(...)` method requires a primary key. But if you have a row from the `db[table].rows` iterator it might not even contain the primary key - provided the table is a `rowid` table.

Instead, currently, you need to introspect the table and, if `rowid` is a primary key, explicitly include that in the `select=`  argument to `table.rows_where(...)` - otherwise it will not be returned.

A utility mechanism to make this easier would be very welcome.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/240/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1077620955,I_kwDOBm6k_c5AOzDb,1549,Redesign CSV export to improve usability,536941,fgregg,open,0,,,3268330,Datasette 1.0,5,2021-12-11T19:02:12Z,2022-04-04T11:17:13Z,,CONTRIBUTOR,,"*Original title: Set content type for CSV so that browsers will attempt to download instead opening in the browser*

Right now, if the user clicks on the CSV related to a <s>table or a</s> query, the response header for the content type is 

""content-type: text/plain; charset=utf-8""

Most browsers will try to open a file with this content-type in the browser. 

This is not what most people want to do, and lots of folks don't know that if they want to download the CSV and open it in the a spreadsheet program they next need to save the page through their browser.

It would be great if the response header could be something like 

```
'Content-type: text/csv');
'Content-disposition: attachment;filename=MyVerySpecial.csv');
```

which would lead browsers to open a download dialog.


",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1549/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1250629388,I_kwDOCGYnMM5KixcM,440,CSV files with too many values in a row cause errors,4068,frafra,closed,0,,,,,20,2022-05-27T10:54:44Z,2022-06-14T22:23:01Z,2022-06-14T20:12:46Z,NONE,,"*Original title: csv.DictReader can have None as key*

In some cases, `csv.DictReader` can have `None` as key for unnamed columns, and a list of values as value.
`sqlite_utils.utils.rows_from_file` cannot handle that:

```python
url=""https://artsdatabanken.no/Fab2018/api/export/csv""
db = sqlite_utils.Database("":memory"")

with urlopen(url) as fab:
    reader, _ = sqlite_utils.utils.rows_from_file(fab, encoding=""utf-16le"")   
    db[""fab2018""].insert_all(reader, pk=""Id"")
```

Result:
```
Traceback (most recent call last):
  File ""<stdin>"", line 3, in <module>
  File ""/home/user/.local/pipx/venvs/sqlite-utils/lib/python3.8/site-packages/sqlite_utils/db.py"", line 2924, in insert_all
    chunk = list(chunk)
  File ""/home/user/.local/pipx/venvs/sqlite-utils/lib/python3.8/site-packages/sqlite_utils/db.py"", line 3454, in fix_square_braces
    if any(""["" in key or ""]"" in key for key in record.keys()):
  File ""/home/user/.local/pipx/venvs/sqlite-utils/lib/python3.8/site-packages/sqlite_utils/db.py"", line 3454, in <genexpr>
    if any(""["" in key or ""]"" in key for key in record.keys()):
TypeError: argument of type 'NoneType' is not iterable
```

Code:
https://github.com/simonw/sqlite-utils/blob/59be60c471fd7a2c4be7f75e8911163e618ff5ca/sqlite_utils/db.py#L3454

`sqlite-utils insert` from command line is not affected by this issue.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/440/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
656959584,MDU6SXNzdWU2NTY5NTk1ODQ=,893,pip3 install datasette not serving static on linuxbrew.,44167,zodman,closed,0,,,,,1,2020-07-14T23:33:38Z,2021-06-02T04:29:56Z,2021-06-02T04:29:56Z,NONE,,"*This error wasn't thrown*

```
Traceback (most recent call last):
  File ""/home/linuxbrew/.linuxbrew/opt/python@3.8/lib/python3.8/site-packages/datasette/utils/asgi.py"", line 289, in inner_static
    full_path.relative_to(root_path)
  File ""/home/linuxbrew/.linuxbrew/opt/python@3.8/lib/python3.8/pathlib.py"", line 904, in relative_to
    raise ValueError(""{!r} does not start with {!r}""
ValueError: '/home/linuxbrew/.linuxbrew/lib/python3.8/site-packages/datasette/static/app.css' does not start with '/home/linuxbrew/.linuxbrew/opt/python@3.8/lib/python3.8/site-packages/datasette/static'
```

Linuxbrew install python@3.8 with symbolic links when  You call the full_path.relative_to(root_path) throw ValueError. This happened when you install from pip3

when you install with python3 setup.py develop , works good.

Well at the end the static wasn't serving.
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/893/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1822939274,I_kwDOBm6k_c5sp9iK,2113,Implement and document extras for the new query view page,9599,simonw,open,0,,,8755003,Datasette 1.0a-next,3,2023-07-26T18:24:01Z,2023-08-09T17:35:22Z,,OWNER,,- #2109 ,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2113/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1822940263,I_kwDOBm6k_c5sp9xn,2114,Implement canned queries against new query JSON work,9599,simonw,closed,0,,,9700784,Datasette 1.0a3,3,2023-07-26T18:24:50Z,2023-08-09T15:26:58Z,2023-08-09T15:26:57Z,OWNER,,- #2109 ,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2114/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1822940964,I_kwDOBm6k_c5sp98k,2115,Ensure all tests pass against new query view JSON,9599,simonw,closed,0,,,9700784,Datasette 1.0a3,0,2023-07-26T18:25:20Z,2023-08-08T02:01:39Z,2023-08-08T02:01:38Z,OWNER,,- #2109 ,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2115/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
924992318,MDU6SXNzdWU5MjQ5OTIzMTg=,281,Mechanism for explicitly stating CSV or JSON or TSV for sqlite-utils memory,9599,simonw,closed,0,,,,,1,2021-06-18T15:04:53Z,2021-06-19T03:11:59Z,2021-06-19T03:11:59Z,OWNER,,"- Implement `filename.json:json` and `-:nl` and suchlike options for specifying the format rather than guessing it - see https://github.com/simonw/sqlite-utils/issues/272#issuecomment-861985944

Follows #272",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/281/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
924990677,MDU6SXNzdWU5MjQ5OTA2Nzc=,279,sqlite-utils memory should handle TSV and JSON in addition to CSV,9599,simonw,closed,0,,,,,7,2021-06-18T15:02:54Z,2021-06-19T03:11:59Z,2021-06-19T03:11:59Z,OWNER,,"- Use sniff to detect CSV or TSV (if `:tsv` or `:csv` was not specified) and delimiters

Follow-on from #272",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/279/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
276704327,MDU6SXNzdWUyNzY3MDQzMjc=,150,_group_count= feature improvements,9599,simonw,closed,0,,,,,3,2017-11-24T22:06:18Z,2018-05-28T16:41:28Z,2018-05-28T16:41:28Z,OWNER,,"- [ ] The ""apply filters"" form should keep you on the _group_count= page
- [ ] Foreign key references should be expand
- [ ] Page title should reflect the view you are on",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/150/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
268262480,MDU6SXNzdWUyNjgyNjI0ODA=,36,"date, year, month and day querystring lookups",9599,simonw,closed,0,,,,,3,2017-10-25T04:23:45Z,2018-05-28T17:30:53Z,2018-05-28T17:30:53Z,OWNER,,"- [ ] `?timestamp___date=2017-07-17` - return every item where the timestamp falls on that date
- [ ] `?timestamp___year=2017` - return every item where the timestamp falls within 2017
- [ ] `?timestamp___month=1` - return every item where the month component is January
- [ ] `?timestamp___day=10` - return every item where the day-of-the-month component is 10

Follow on from #23 ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/36/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
602533539,MDU6SXNzdWU2MDI1MzM1Mzk=,4,Upload all my photos to a secure S3 bucket,9599,simonw,closed,0,,,5324096,Apple Photos online and securely browsable,14,2020-04-18T19:24:50Z,2020-04-18T21:58:11Z,2020-04-18T21:57:13Z,MEMBER,,"- [x] Create a bucket with bucket credentials
- [x] Programmatically upload some recent photos to it (from a notebook)
- [x] Turn this into a script",256834907,dogsheep-photos,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
601265023,MDU6SXNzdWU2MDEyNjUwMjM=,25,Improvements to demo instance,9599,simonw,closed,0,,,,,1,2020-04-16T17:26:55Z,2020-04-16T18:07:12Z,2020-04-16T18:07:12Z,MEMBER,,- [x] Demo should pull issue-comments as well,207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/25/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
732685643,MDU6SXNzdWU3MzI2ODU2NDM=,1063,.csv should link to .blob downloads,9599,simonw,closed,0,,,6026070,0.51,3,2020-10-29T21:45:58Z,2021-06-17T18:12:30Z,2020-10-29T22:47:45Z,OWNER,,"- [x] Update `.csv` output to link to these things (and get that `xfail` test to pass)
- ~~Add a `.csv?_blob_base64=1` argument that causes them to be output in base64 in the CSV~~

> Moving the CSV work to a separate ticket.
_Originally posted by @simonw in https://github.com/simonw/datasette/pull/1061#issuecomment-719042601_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1063/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
679779797,MDU6SXNzdWU2Nzk3Nzk3OTc=,939,extra_ plugin hooks should take the same arguments,9599,simonw,closed,0,,,,,6,2020-08-16T16:04:54Z,2020-08-16T18:25:05Z,2020-08-16T16:50:29Z,OWNER,,"- [x] `extra_css_urls(template, database, table, datasette)`
- [x] `extra_js_urls(template, database, table, datasette)`
- [x] `extra_body_script(template, database, table, view_name, datasette)`
- [x] `extra_template_vars(template, database, table, view_name, request, datasette)`

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/938#issuecomment-674544691_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/939/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
565041624,MDU6SXNzdWU1NjUwNDE2MjQ=,671,"datasette.add_database(name, db) and datasette.remove_database(name) methods",9599,simonw,closed,0,,,,,1,2020-02-14T01:05:48Z,2020-02-14T01:30:35Z,2020-02-14T01:30:30Z,OWNER,,"- `datasette.add_database(name, db)` - adds a new named database to the list of connected databases. `db` will be a `Database()` object, which may prove useful in the future for things like #670 and could also allow some plugins to provide in-memory SQLite databases.
- `datasette.remove_database(name)`

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/417#issuecomment-586047995_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/671/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1432012302,I_kwDOBm6k_c5VWsYO,1877,Refactor and tidy up final write API code,9599,simonw,closed,0,,,,,1,2022-11-01T20:00:11Z,2022-11-29T19:44:16Z,2022-11-29T19:44:07Z,OWNER,,"- `views/table.py` has got a bit too big - I think the write classes should be pulled out into a separate module.
- [x] There's duplicate logic for deciding if the table and database exist and checking permissions",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1877/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
608613033,MDU6SXNzdWU2MDg2MTMwMzM=,745,Extract the hash-URL mechanism out into a plugin,9599,simonw,closed,0,,,,,2,2020-04-28T21:00:38Z,2020-10-23T19:47:18Z,2020-10-23T19:47:10Z,OWNER,,"0.28 in May 2019 made this feature not-the-default: https://datasette.readthedocs.io/en/stable/changelog.html#v0-28 - see #418

I've not felt the need to use it myself since. I think I should move it into a plugin.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/745/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
860734722,MDU6SXNzdWU4NjA3MzQ3MjI=,1302,Fix disappearing facets,192568,mroswell,open,0,,,,,0,2021-04-18T18:42:33Z,2021-04-20T07:40:15Z,,CONTRIBUTOR,,"1. Clone https://github.com/mroswell/list-N
2. Run `datasette disinfectants.db -o`
3. Select the `Safer_or_Toxic` facet.
4. Select `Toxic`.
5. Close out the `Safer_or_Toxic` facet.
6. Examine `Suggested facets` list. `Safer_or_Toxic` is GONE.
7. Try some other facets. When you select an element, and then close the list, in some cases, the facet properly returns to the `Suggested facet` list... Arrays and dates properly return to the list, but fields with strings don't return to the list. 

Since my site is devoted to whether disinfectants are Safer or Toxic, having the suggested facet disappear from the suggested facet list is very confusing* to end-users. This, along with a few other issues, unfortunately proved beyond my own programming ability to address. So I hired a Senior-level developer to address a number of issues, including this disappearing act.

8. Open a new terminal. Run `datasette disinfectants.db -m metadata.json --static static:static/  --template-dir templates/ --plugins-dir plugins/ -p 8001 -o`
9. Repeat steps 3-6, but this time, the Safer_or_Toxic facet returns to the list (and the related URL parameters are removed).

I'm not sure how to do a pull request for this, because the plugin contains other functionality that goes beyond this bug. I wanted the facets sorted in a certain order (both in the suggested facet list, and the detail lists) (... the detail lists were hopping around all over the place before...) I wanted the duplicate facets removed (leaving only the one where you can facet by individual item in an array.) I wanted the arrays to be presented in a prettier fashion (I did that in the template... That could be moved over to the plugin at some point)

I'm thinking it'll be very helpful if applicable parts of my project's plugin (sort_suggested_facets_plugin.py) will be able to be incorporated back into datasette, but I leave that to you to consider.

(* The disappearing facet bug was especially confusing because I'm removing the filters and sql from the table page, at the request of the organization. The filters and sql detail created a lot of confusion for end users who try to find disinfectants used by Hospitals, for instance, as an '=' won't find them, since they are part of the Use_site array.) My disappearing-facet confusion was documented in my own issue: https://github.com/mroswell/list-N/issues/57 (addressed by the plugin). Other facet-related issues here: https://github.com/mroswell/list-N/issues/54 (addressed by the plugin); https://github.com/mroswell/list-N/issues/15 (addressed by template); https://github.com/mroswell/list-N/issues/53 (not yet addressed). 
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1302/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
735852274,MDU6SXNzdWU3MzU4NTIyNzQ=,1082,DigitalOcean buildpack memory errors for large sqlite db?,39538958,justmars,open,0,,,,,3,2020-11-04T06:35:32Z,2020-11-04T19:35:44Z,,NONE,,"1. Have a sqlite db stored in Dropbox
2. Previously tried the Digital Ocean build pack minimal approach (e.g. Procfile, requirements.txt, bin/post_compile)
3. bin/post_compile with wget from Dropbox
4. download of large sqlite db is successful 
5. log reveals that when building Docker container, Digital Ocean runs out of memory for 5gb+ sqlite db but works fine for 2gb+ sqlite db",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1082/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1428560020,I_kwDOBm6k_c5VJhiU,1872,"SITE-BUSTING ERROR: ""render_template() called before await ds.invoke_startup()""",192568,mroswell,closed,0,,,,,3,2022-10-30T02:28:39Z,2022-10-30T06:26:01Z,2022-10-30T06:26:01Z,CONTRIBUTOR,,"1. My https://list.saferdisinfectants.org/disinfectants/listN page (linked from https://SaferDisinfectants.org ) has been running beautifully for a year and a half, including a GitHub Actions workflow that's been routinely updating the database.

2. I received a recent report that the list page is down. I don't know when it went down, but the content is replaced with: ""render_template() called before await ds.invoke_startup()""

3. The local datasette repo runs without incident.

4. The site is hosted on vercel, linked to my github repo. Perhaps  some vercel changes were made, but not by anyone on our side. Here is a screenshot of the current project settings:

<img width=""877"" alt=""Screen Shot 2022-10-29 at 9 23 30 PM"" src=""https://user-images.githubusercontent.com/192568/198859474-341a5504-88b8-452d-bd06-e988cbe75e6c.png"">

Here a screenshot of the latest deployment status:
<img width=""1351"" alt=""Screen Shot 2022-10-29 at 10 19 21 PM"" src=""https://user-images.githubusercontent.com/192568/198859641-ad3d99ba-a41d-4b98-88f6-e3a01fe9f238.png"">

This is my repository:
https://github.com/mroswell/list-N
(I notice: datasette==0.59 in my requirements.txt file)

Because it's been long while since I actively worked on this or any other datasette project, I forget a lot of what I knew at one point. Perhaps some configuration file could be missing? Or perhaps I just need to know the right incantation to add to that vercel settings page. 

Help is welcome as the nonprofit org is soon hosting its annual conference, and we'd love to have the page working again.





",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1872/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
843884745,MDU6SXNzdWU4NDM4ODQ3NDU=,1283,advanced #export causes unexpected scrolling,192568,mroswell,open,0,,,,,0,2021-03-29T22:46:57Z,2021-03-29T22:46:57Z,,CONTRIBUTOR,,"1. Visit a datasette table page
2. Click on the ""(advanced)"" link. This adds a fragment identifier ""#export"" to the URL, and scrolls down to the ""Advanced export"" div with the ""export"" id.
3. Manually scroll back up, and click on a suggested facet. The fragment identifier is still present, and the app scrolls back down to the ""Advanced export"" div. I think this is unwanted behavior.

The user remedy seems to be to manually remove the ""#export"" from the URL.

This behavior happens in my project, and in:
https://covid-19.datasettes.com/covid/economist_excess_deaths  (for instance) 
but not in this table: 
https://global-power-plants.datasettes.com/global-power-plants/global-power-plants",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1283/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1843600087,I_kwDOBm6k_c5t4xrX,2135,Release notes for 1.0a3,9599,simonw,closed,0,,,9700784,Datasette 1.0a3,3,2023-08-09T16:09:26Z,2023-08-09T19:17:07Z,2023-08-09T19:17:06Z,OWNER,,118 commits! https://github.com/simonw/datasette/compare/1.0a2...26be9f0445b753fb84c802c356b0791a72269f25,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2135/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
315548495,MDU6SXNzdWUzMTU1NDg0OTU=,225,/-/(inspect|metadata|plugins)(.json)? introspection,9599,simonw,closed,0,,,,,0,2018-04-18T16:14:58Z,2018-04-19T05:25:33Z,2018-04-19T05:25:33Z,OWNER,,"3 pages (and accompanying .json endpoints) for viewing:

* the metadata.json that datasette was loaded with
* the output of ds.inspect()
* a list of installed plugins, detected by pluggy",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/225/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1178521513,I_kwDOBm6k_c5GPs-p,1682,SQL queries against databases with different routes are broken,9599,simonw,closed,0,,,,,1,2022-03-23T18:42:57Z,2022-03-23T18:48:16Z,2022-03-23T18:48:16Z,OWNER,,"500 error on https://datasette-hashed-urls-preview.vercel.app/fixtures-09f8f95?sql=select+*+from+facetable

Here's the trace:
```
  File ""/Users/simon/.local/share/virtualenvs/datasette-hashed-urls-ssI2fO50/lib/python3.10/site-packages/datasette/views/database.py"", line 54, in data
    return await QueryView(self.ds).data(
  File ""/Users/simon/.local/share/virtualenvs/datasette-hashed-urls-ssI2fO50/lib/python3.10/site-packages/datasette/views/database.py"", line 232, in data
    self.ds.get_database(database), sql
  File ""/Users/simon/.local/share/virtualenvs/datasette-hashed-urls-ssI2fO50/lib/python3.10/site-packages/datasette/app.py"", line 401, in get_database
    return self.databases[name]
KeyError: 'fixtures-aa7318b'
```
It looks like this is a Datasette bug, which is frustrating because I just shipped Datasette 0.61 five minutes ago!

_Originally posted by @simonw in https://github.com/simonw/datasette-hashed-urls/issues/13#issuecomment-1076693667_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1682/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1453813400,I_kwDOBm6k_c5Wp26Y,1901,"Some plugins show ""home"" breadcrumbs twice in the top left",95570,bgrins,closed,0,,,,,8,2022-11-17T18:44:58Z,2022-11-18T07:22:37Z,2022-11-18T07:02:56Z,CONTRIBUTOR,,"<img width=""1023"" alt=""Screenshot 2022-11-17 at 10 44 12 AM"" src=""https://user-images.githubusercontent.com/95570/202531505-b8632f70-d95a-425f-b2f2-6380b153478b.png"">
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1901/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
506432572,MDU6SXNzdWU1MDY0MzI1NzI=,21,Fix &amp; escapes in tweet text,9599,simonw,closed,0,,,,,1,2019-10-14T03:37:28Z,2019-10-15T18:48:16Z,2019-10-15T18:48:16Z,MEMBER,,"<img width=""1136"" alt=""twitter__tweets__21_773_rows_where_sorted_by_id_descending"" src=""https://user-images.githubusercontent.com/9599/66728360-38f91b80-edf9-11e9-95b5-ce6d097fe18e.png"">

Shouldn't be storing `&amp;` here.",206156866,twitter-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/21/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1493404423,I_kwDOBm6k_c5ZA4sH,1948,500 error on permission debug page when testing actors with _r,9599,simonw,open,0,,,,,1,2022-12-13T05:22:03Z,2022-12-13T05:22:19Z,,OWNER,,"<img width=""1272"" alt=""image"" src=""https://user-images.githubusercontent.com/9599/207233390-ecb1b81b-c9f4-40a3-8131-5537ab23e9fc.png"">

The 500 error is silent unless you are looking at the DevTools network pane.
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1948/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
583970196,MDU6SXNzdWU1ODM5NzAxOTY=,701,Search box CSS doesn't look great on OS X Safari,9599,simonw,closed,0,,,5234079,Datasette 0.39,3,2020-03-18T20:00:52Z,2020-03-24T22:57:18Z,2020-03-24T22:57:18Z,OWNER,,"<img width=""1447"" alt=""twitter__tweets__68_773_rows_where_sorted_by_id_descending_and_twitter__tweets__955_rows"" src=""https://user-images.githubusercontent.com/9599/77002262-73bb2e80-6918-11ea-87a3-e3f537365c9f.png"">
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/701/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
493668862,MDU6SXNzdWU0OTM2Njg4NjI=,2,Extract licenses from repos into a separate table,9599,simonw,closed,0,,,,,0,2019-09-14T21:33:41Z,2019-09-14T21:46:58Z,2019-09-14T21:46:58Z,MEMBER,,"<img width=""588"" alt=""starred3__repos__800_rows"" src=""https://user-images.githubusercontent.com/9599/64913845-a2074980-d6fc-11e9-815a-f103ad0c6f56.png"">
",207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/2/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
715072935,MDU6SXNzdWU3MTUwNzI5MzU=,993,Column action menu should show column type,9599,simonw,closed,0,,,5971510,Datasette 0.50,2,2020-10-05T18:40:49Z,2020-10-08T23:55:19Z,2020-10-06T00:33:15Z,OWNER,,"<img width=""616"" alt=""data__rent_burden_households_and_median_gross_rent_percent__97_rows"" src=""https://user-images.githubusercontent.com/9599/95118702-97509580-06ff-11eb-8523-39eb1b389acd.png"">
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/993/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1068791148,I_kwDOBm6k_c4_tHVs,1540,Idea: hover to reveal details of linked row,9599,simonw,open,0,,,,,6,2021-12-01T19:28:07Z,2021-12-09T23:38:39Z,,OWNER,,"<img width=""698"" alt=""fara__item_version__7_rows_where_where__item___5236"" src=""https://user-images.githubusercontent.com/9599/144300364-ee621811-8f5a-4c7e-9950-199119092cf3.png"">

Hovering over that could work a little bit like GitHub issue links:

![hover](https://user-images.githubusercontent.com/9599/144300537-9cd9e9af-ac16-42db-842f-37661bc94063.gif)
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1540/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
464905894,MDU6SXNzdWU0NjQ5MDU4OTQ=,545,Fix header on 404 page,9599,simonw,closed,0,,,4471010,Datasette 0.29,1,2019-07-07T01:47:40Z,2019-07-07T20:26:55Z,2019-07-07T20:26:55Z,OWNER,,"<img width=""707"" alt=""Error_404"" src=""https://user-images.githubusercontent.com/9599/60762932-83127a00-a01e-11e9-853a-6ea6aa5e8f6d.png"">
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/545/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
607211058,MDU6SXNzdWU2MDcyMTEwNTg=,740,Don't throw 500 error on attempted directory browse,9599,simonw,closed,0,,,,,1,2020-04-27T03:50:11Z,2020-04-27T18:29:15Z,2020-04-27T18:29:15Z,OWNER,,"<img width=""725"" alt=""Error_500"" src=""https://user-images.githubusercontent.com/9599/80332338-76eee780-87ff-11ea-85b3-32704d715cd9.png"">

This should be a 403 error instead, because the `--static` mechanism doesn't allow directory browsing.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/740/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
489419782,MDU6SXNzdWU0ODk0MTk3ODI=,6,Extract extended_entities into a media table,9599,simonw,closed,0,,,,,0,2019-09-04T21:59:10Z,2019-09-04T22:08:01Z,2019-09-04T22:08:01Z,MEMBER,,"<img width=""747"" alt=""cleo__select___from_tweets_order_by_id_limit_101"" src=""https://user-images.githubusercontent.com/9599/64294799-880f7f00-cf24-11e9-83b8-75c1ee54182c.png"">
",206156866,twitter-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/6/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1362402998,I_kwDOBm6k_c5RNJ62,1802,Tests reliably failing on Python 3.7,9599,simonw,closed,0,,,,,15,2022-09-05T19:21:16Z,2022-09-06T00:40:20Z,2022-09-06T00:40:20Z,OWNER,,"<img width=""819"" alt=""image"" src=""https://user-images.githubusercontent.com/9599/188504608-5be0f941-dca2-4f42-83bc-3d04bbad221d.png"">

https://github.com/simonw/datasette/runs/8194907739?check_suite_focus=true

I thought this might be an intermittent failure but attempts to re-run the tests have not made it pass.

End of that trace is:

```
/home/runner/work/datasette/datasette/datasette/app.py:234: in __init__
    self._refresh_schemas_lock = asyncio.Lock()
/opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/asyncio/locks.py:161: in __init__
    self._loop = events.get_event_loop()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <asyncio.unix_events._UnixDefaultEventLoopPolicy object at 0x7fb1fc799fd0>

    def get_event_loop(self):
        """"""Get the event loop for the current context.
    
        Returns an instance of EventLoop or raises an exception.
        """"""
        if (self._local._loop is None and
                not self._local._set_called and
                isinstance(threading.current_thread(), threading._MainThread)):
            self.set_event_loop(self.new_event_loop())
    
        if self._local._loop is None:
            raise RuntimeError('There is no current event loop in thread %r.'
>                              % threading.current_thread().name)
E           RuntimeError: There is no current event loop in thread 'MainThread'.
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1802/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
741021342,MDU6SXNzdWU3NDEwMjEzNDI=,1086,Foreign keys with blank titles result in non-clickable links,9599,simonw,closed,0,,,6055094,Datasette 0.52,3,2020-11-11T19:41:09Z,2020-11-28T23:28:29Z,2020-11-11T23:46:20Z,OWNER,,"<img width=""851"" alt=""index__core_snapshot_tags__3_rows_and_swarm__select_venues_name__checkins_id__createdAt__type__timeZoneOffset__editableUntil___like___isMayor__source__venue__createdBy__event__sticker__created__comments_count__entities__shout__checkins_priv"" src=""https://user-images.githubusercontent.com/9599/98856656-bafbaf80-2412-11eb-8070-4f3a47511973.png"">

The HTML looks like this:

```html
<td class=""col-tag_id type-int""><a href=""/index/core_tag/1""></a>&nbsp;<em>1</em></td>
```
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1086/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1795051447,I_kwDOBm6k_c5q_k-3,2097,Drop Python 3.7,9599,simonw,closed,0,,,,,0,2023-07-08T18:39:44Z,2023-08-23T18:18:00Z,2023-08-23T18:18:00Z,OWNER,,">  I'm going to drop Python 3.7.

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1153#issuecomment-1627455892_

It's not supported any more: https://devguide.python.org/versions/",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2097/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1452364777,I_kwDOBm6k_c5WkVPp,1896,Extract logic for resolving a URL to a database / table / row,9599,simonw,closed,0,,,8658075,Datasette 1.0a0,4,2022-11-16T22:25:20Z,2022-11-18T22:57:47Z,2022-11-18T22:56:55Z,OWNER,,">  In trying to write this I realize that there's a lot of duplicated code with delete row, specifically around resolving the incoming URL into a row (or a database or a table).
>
> Since this is so common, I think it's worth extracting the logic out first.

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1863#issuecomment-1317755263_
      ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1896/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1435294468,I_kwDOBm6k_c5VjNsE,1882,`/db/-/create` API for creating tables,9599,simonw,closed,0,,,8658075,Datasette 1.0a0,12,2022-11-03T21:44:32Z,2022-11-15T19:59:43Z,2022-11-15T06:00:41Z,OWNER,,">  It really feels like this should be accompanied by a `/db/-/create` API for creating tables. I had to add that to `sqlite-utils` eventually (initially it only supported creating by passing in an example document):
>
> https://sqlite-utils.datasette.io/en/stable/cli.html#cli-create-table

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1862#issuecomment-1299073433_
      ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1882/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1386734383,I_kwDOBm6k_c5Sp-Mv,1821,Release Datasette 0.63a0,9599,simonw,closed,0,,,,,1,2022-09-26T21:15:27Z,2022-09-26T22:06:39Z,2022-09-26T22:06:39Z,OWNER,,"> - The [prepare_jinja2_environment(env, datasette)](https://docs.datasette.io/en/latest/plugin_hooks.html#plugin-hook-prepare-jinja2-environment) plugin hook now accepts an optional `datasette` argument. Hook implementations can also now return an `async` function which will be awaited automatically. ([#1809](https://github.com/simonw/datasette/issues/1809))
> - `--load-extension` option now supports entrypoints. Thanks, Alex Garcia. ([#1789](https://github.com/simonw/datasette/pull/1789))
> - New tutorial: [Cleaning data with sqlite-utils and Datasette](https://datasette.io/tutorials/clean-data).
> - Facet size can now be set per-table with the new `facet_size` table metadata option. ([#1804](https://github.com/simonw/datasette/issues/1804))
> - `truncate_cells_html` setting now also affects long URLs in columns. ([#1805](https://github.com/simonw/datasette/issues/1805))
> - `Database(is_mutable=)` now defaults to `True`. ([#1808](https://github.com/simonw/datasette/issues/1808))
> - Non-JavaScript textarea now increases height to fit the SQL query. ([#1786](https://github.com/simonw/datasette/issues/1786))
> - More detailed command descriptions on the [CLI reference](https://docs.datasette.io/en/latest/cli-reference.html#cli-reference) page. ([#1787](https://github.com/simonw/datasette/issues/1787))
> - Datasette no longer enforces upper bounds on its depenedencies. ([#1800](https://github.com/simonw/datasette/issues/1800))
> - Facets are now displayed with better line-breaks in long values. Thanks, Daniel Rech. ([#1794](https://github.com/simonw/datasette/pull/1794))
> - The `settings.json` file used in [Configuration directory mode](https://docs.datasette.io/en/latest/settings.html#config-dir) is now validated on startup. ([#1816](https://github.com/simonw/datasette/issues/1816))",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1821/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1786243905,I_kwDOCGYnMM5qd-tB,564,Document that running `db.transform()` tidies up the schema indentation,9599,simonw,closed,0,,,,,0,2023-07-03T13:59:28Z,2023-07-22T22:15:34Z,2023-07-22T22:15:34Z,OWNER,,"> ... and it turns out running `.transform()` with no arguments still fixes the format of the schema!

```pycon
>>> db[""log""].add_column(""foo"", str)
<Table log (id, name2, age, weight, foo)>
>>> db[""log""].add_column(""bar"", str)
<Table log (id, name2, age, weight, foo, bar)>
>>> db[""log""].add_column(""baz"", str)
<Table log (id, name2, age, weight, foo, bar, baz)>
>>> print(db[""log""].schema)
CREATE TABLE ""log"" (
   [id] INTEGER PRIMARY KEY,
   [name2] TEXT,
   [age] INTEGER,
   [weight] FLOAT
, [foo] TEXT, [bar] TEXT, [baz] TEXT)
>>> db[""log""].transform()
<Table log (id, name2, age, weight, foo, bar, baz)>
>>> print(db[""log""].schema)
CREATE TABLE ""log"" (
   [id] INTEGER PRIMARY KEY,
   [name2] TEXT,
   [age] INTEGER,
   [weight] FLOAT,
   [foo] TEXT,
   [bar] TEXT,
   [baz] TEXT
)
```

_Originally posted by @simonw in https://github.com/simonw/llm/issues/65#issuecomment-1618347727_
            ",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/564/reactions"", ""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 1}",,completed
1174302994,I_kwDOBm6k_c5F_nES,1667,Make route matched pattern groups more consistent,9599,simonw,closed,0,,,3268330,Datasette 1.0,3,2022-03-19T16:32:35Z,2022-03-19T20:37:42Z,2022-03-19T20:37:41Z,OWNER,,"> ... highlights how inconsistent the way the capturing works is. Especially `as_format` which can be `None` or `""""` or `.json` or `json` or not used at all in the case of `TableView`.

https://github.com/simonw/datasette/blob/764738dfcb16cd98b0987d443f59d5baa9d3c332/tests/test_routes.py#L12-L36

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1666#issuecomment-1073039670_

Part of:
- #1660",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1667/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
705840673,MDU6SXNzdWU3MDU4NDA2NzM=,972,Support faceting against arbitrary SQL queries,9599,simonw,open,0,,,,,1,2020-09-21T19:00:43Z,2021-12-15T18:02:20Z,,OWNER,,"> ... support for running facets against arbitrary custom SQL queries is half-done in that facets now execute against wrapped subqueries as-of ea66c45df96479ef66a89caa71fff1a97a862646
> 
> https://github.com/simonw/datasette/blob/ea66c45df96479ef66a89caa71fff1a97a862646/datasette/facets.py#L192-L200
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/971#issuecomment-696307922_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/972/reactions"", ""total_count"": 3, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 3, ""rocket"": 0, ""eyes"": 0}",,
1907765514,I_kwDOBm6k_c5xtjEK,2195,`datasette publish` needs support for the new config/metadata split,9599,simonw,open,0,,,,,9,2023-09-21T21:08:12Z,2023-09-21T22:57:48Z,,OWNER,,"> ... which raises the challenge that `datasette publish` doesn't yet know what to do with a config file!

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/2194#issuecomment-1730259871_
            ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2195/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1097087280,I_kwDOCGYnMM5BZDkw,368,Offer `python -m sqlite_utils` as an alternative to `sqlite-utils`,9599,simonw,closed,0,,,7558727,3.21,3,2022-01-09T02:29:30Z,2022-01-10T19:27:20Z,2022-01-09T02:40:50Z,OWNER,,"> Add this to `sqlite_utils/cli.py`:
>
> ```python
> if __name__ == ""__main__"":
>     cli()
> ```
> Now the tool can be run using `python -m sqlite_utils.cli --help`

_Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/364#issuecomment-1008214998_",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/368/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
965210966,MDU6SXNzdWU5NjUyMTA5NjY=,314,Type signatures for `.create_table()` and `.create_table_sql()` and `.create()` and `Table.__init__`,9599,simonw,closed,0,,,,,2,2021-08-10T18:03:59Z,2021-08-18T22:25:21Z,2021-08-18T22:25:21Z,OWNER,,"> Adding type signatures to `create_table()` and `.create_table_sql()` is a bit too involved, I'll do that in a separate issue.

_Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/312#issuecomment-896200682_",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/314/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1784794489,I_kwDOCGYnMM5qYc15,562,Explore the intersection between sqlite-utils and dataclasses,9599,simonw,open,0,,,,,1,2023-07-02T19:23:08Z,2023-07-02T19:26:39Z,,OWNER,,"> Aside: this makes me think it might be cool if `sqlite-utils` had a way of working with dataclasses rather than just dicts, and knew how to create a SQLite table to match a dataclass and maybe how to code-generate dataclasses for a specific table schema (dynamically or even using code-generation that can be written to disk, for better editor integrations).

_Originally posted by @simonw in https://github.com/simonw/llm/issues/65#issuecomment-1616742529_
            ",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/562/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
972918533,MDU6SXNzdWU5NzI5MTg1MzM=,1438,Query page .csv and .json links are not correctly URL-encoded on Vercel under unknown specific conditions,9599,simonw,open,0,,,,,7,2021-08-17T17:35:36Z,2021-08-18T00:22:23Z,,OWNER,,"> Confirmed: https://thesession.vercel.app/thesession?sql=select+*+from+tunes+where+name+like+%22%25wise+maid%25%22%0D%0A is a page where the URL correctly encoded `%` as `%25` - but then in the HTML on that page that links to the CSV and JSON versions we get this:
>
> ```html
> <p class=""export-links"">This data as
>   <a href=""/thesession.json?sql=select * from tunes where name like &#34;%wise maid%&#34;"">json</a>,
>   <a href=""/thesession.csv?sql=select * from tunes where name like &#34;%wise maid%&#34;&amp;_size=max"">CSV</a>
> </p>
> ```
Those CSV and JSON links are incorrect.

_Originally posted by @simonw in https://github.com/simonw/datasette-publish-vercel/issues/48#issuecomment-900497579_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1438/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1114640101,I_kwDOCGYnMM5CcA7l,392,`sqlite-utils bulk --batch-size` option,9599,simonw,closed,0,,,,,4,2022-01-26T05:17:11Z,2022-01-26T18:17:59Z,2022-01-26T18:17:59Z,OWNER,,"> Could add support for `--batch-size` as seen in `insert`/`upsert` too - causing it to break the list up into batches and commit for each one.

_Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/391#issuecomment-1021876055_",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/392/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1433576351,I_kwDOBm6k_c5VcqOf,1880,Datasette with many and large databases > Memory use,525934,amitkoth,open,0,,,,,4,2022-11-02T18:10:27Z,2022-11-16T17:50:29Z,,NONE,,"> Datasette maintains an in-memory SQLite database with details of the the databases, tables and columns for all of the attached databases.

The above is from the docs ^. There's two problems here - the number of datasette ""instances"" in a single server/VM and the size of the database itself. We want the **opposite** of in-memory, including what happens on SQLlite - documented in https://www.sqlite.org/inmemorydb.html

From the context in https://github.com/simonw/datasette/issues/1150 - does it mean datasette is memory-bound to the size of the dataset - which might be a deal-breaker for many large-scale use cases?

In an extreme case - let's say a single server had 100 SQLlite databases, which would enable 100 ""instances"" of datasette to run, one per client (e.g. in a SaaS multi-tenant environment). How could we achieve all these goals:

1. Allow any _one_ of these 100 databases to grow to say 2Tb in size 
2. Have one datasette instance, which connects to 1 of the 100 instances, based on incoming credentials/tenant ID
3. Minimize memory use entirely - both by datasette and SQLlite, such that almost all operations are executed in real-time on-disk with little to no memory consumption per-tenant, or per-database.

Any ideas appreciated - we're looking to use this in a SaaS type of setting - many instances, single server.

@simonw great work on datasette, in general! Possibly related to https://github.com/simonw/datasette/issues/1480 but we don't want use any kind of serverless infra - this is a long-running VM/server.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1880/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
628121234,MDU6SXNzdWU2MjgxMjEyMzQ=,788, /-/permissions debugging tool,9599,simonw,closed,0,,,5512395,Datasette 0.44,2,2020-06-01T03:13:47Z,2020-06-06T00:43:40Z,2020-06-01T05:01:01Z,OWNER,,"> Debugging tool idea: `/-/permissions` page which shows you the actor and lets you type in the strings for `action`, `resource_type` and `resource_identifier` - then shows you EVERY plugin hook that would have executed and what it would have said, plus when the chain would have terminated.
>
> Bonus: if you're logged in as the `root` user (or a user that matches some kind of permission check, maybe a check for `permissions_debug`) you get to see a rolling log of the last 30 permission checks and what the results were across the whole of Datasette. This should make figuring out permissions policies a whole lot easier.

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/699#issuecomment-636576603_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/788/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1082584499,I_kwDOBm6k_c5Ahu2z,1558,Redesign `facet_results` JSON structure prior to Datasette 1.0,9599,simonw,open,0,,,3268330,Datasette 1.0,3,2021-12-16T19:45:10Z,2023-01-09T15:31:17Z,,OWNER,,"> Decision: as an initial fix I'm going to de-duplicate those keys by using `tags__array` etc - with a `_2` on the end if that key is already used.
>
> I'll open a separate issue to redesign this better for Datasette 1.0.

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/625#issuecomment-996130862_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1558/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
687694947,MDU6SXNzdWU2ODc2OTQ5NDc=,954,Remove old register_output_renderer dict mechanism in Datasette 1.0,9599,simonw,open,0,,,3268330,Datasette 1.0,1,2020-08-28T04:04:23Z,2020-08-28T04:56:31Z,,OWNER,,"> Documentation says that the old dictionary mechanism will be deprecated by 1.0:
> 
> https://github.com/simonw/datasette/blob/799ecae94824640bdff21f86997f69844048d5c3/docs/plugin_hooks.rst#L460
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/953#issuecomment-682312494_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/954/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1615891776,I_kwDOBm6k_c5gUI1A,2037,Test failure: FAILED tests/test_cli.py::test_install_requirements - FileNotFoundError,9599,simonw,closed,0,,,,,3,2023-03-08T20:30:06Z,2023-03-09T22:33:39Z,2023-03-09T22:33:39Z,OWNER,,"> FAILED tests/test_cli.py::test_install_requirements - FileNotFoundError: [Errno 2] No such file or directory

From https://github.com/simonw/datasette/actions/runs/4348548218/jobs/7597208191

```
=================================== FAILURES ===================================
__________________________ test_install_requirements ___________________________

run_module = <MagicMock name='run_module' id='139768358191936'>

    @mock.patch(""datasette.cli.run_module"")
    def test_install_requirements(run_module):
        runner = CliRunner()
>       with runner.isolated_filesystem():

/home/runner/work/datasette/datasette/tests/test_cli.py:184: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/opt/hostedtoolcache/Python/3.9.16/x64/lib/python3.9/contextlib.py:119: in __enter__
    return next(self.gen)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <click.testing.CliRunner object at 0x7f1e5bfb9490>, temp_dir = None

    @contextlib.contextmanager
    def isolated_filesystem(
        self, temp_dir: t.Optional[t.Union[str, os.PathLike]] = None
    ) -> t.Iterator[str]:
        """"""A context manager that creates a temporary directory and
        changes the current working directory to it. This isolates tests
        that affect the contents of the CWD to prevent them from
        interfering with each other.
    
        :param temp_dir: Create the temporary directory under this
            directory. If given, the created directory is not removed
            when exiting.
    
        .. versionchanged:: 8.0
            Added the ``temp_dir`` parameter.
        """"""
>       cwd = os.getcwd()
E       FileNotFoundError: [Errno 2] No such file or directory

/opt/hostedtoolcache/Python/3.9.16/x64/lib/python3.9/site-packages/click/testing.py:466: FileNotFoundError
```
Not sure why it only affected the ""[Calculate test coverage](https://github.com/simonw/datasette/actions/workflows/test-coverage.yml)"" one.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2037/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1454532488,I_kwDOBm6k_c5WsmeI,1902,Document {% block crumbs %} for plugin authors,9599,simonw,open,0,,,3268330,Datasette 1.0,0,2022-11-18T06:16:30Z,2022-11-18T06:16:39Z,,OWNER,,"> For `datasette-copyable` I want to show breadcrumbs that take database/instance permissions into account, so I'm removing `{% block nav %}` entirely and replacing it with this:
>
> ```html+jinja
> {% block crumbs %}
> {{ crumbs.nav(request=request, database=database, table=table) }}
> {% endblock %}
> ```

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1901#issuecomment-1319588163_

I should document this.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1902/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1053122092,I_kwDOCGYnMM4-xV4s,339,`table.lookup()` option to populate additional columns when creating a record,9599,simonw,closed,0,,,,,4,2021-11-15T01:41:17Z,2021-11-15T02:02:34Z,2021-11-15T02:02:00Z,OWNER,,"> For the commits table I feel like I want a version of `table.lookup()` that can be passed additional columns to populate only if the record does not exist yet.

_Originally posted by @simonw in https://github.com/simonw/git-history/issues/12#issuecomment-967455017_",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/339/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
732859030,MDU6SXNzdWU3MzI4NTkwMzA=,1066,Table actions menu plus plugin hook,9599,simonw,closed,0,,,6026070,0.51,3,2020-10-30T03:46:54Z,2020-10-30T05:18:36Z,2020-10-30T05:16:50Z,OWNER,,"> For the table actions: attaching it to a cog icon next to the table name could make sense.
>
> <img width=""503"" alt=""data__faa-wildlife-strikes__26_rows_where_where_AIRPORT____OIL_RIG__sorted_by_rowid_and_Fix_table_name_in_spatialite_example_command_by_jsfenfen_·_Pull_Request__1022_·_simonw_datasette"" src=""https://user-images.githubusercontent.com/9599/96168896-45192c80-0ed6-11eb-9d90-6172a8346468.png"">
>
> This is the column action icon at twice the size, color `#666`.
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/690#issuecomment-709497595_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1066/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1493339206,I_kwDOBm6k_c5ZAoxG,1946,`datasette --get` mechanism for sending tokens,9599,simonw,closed,0,,,8711695, Datasette 1.0a2,2,2022-12-13T04:25:05Z,2022-12-13T04:36:57Z,2022-12-13T04:36:57Z,OWNER,,"> For the tests for `datasette create-token` it would be useful if `datasette --get` had a mechanism for sending an `Authorization: Bearer X` header.

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1855#issuecomment-1347731288_
      ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1946/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1066501534,I_kwDOCGYnMM4_kYWe,345,`table.strict` introspection boolean for identifying STRICT mode tables,9599,simonw,closed,0,,,,,2,2021-11-29T21:05:10Z,2021-11-29T22:45:26Z,2021-11-29T22:44:36Z,OWNER,,"> From the STRICT docs:

>> The SQLite parser accepts a comma-separated list of table options after the final close parenthesis in a CREATE TABLE statement. As of this writing (2021-08-23) only two options are recognized:
>> 
>> -   STRICT
>> -   [WITHOUT ROWID](https://www.sqlite.org/withoutrowid.html)
> 
> So I think I need to read the `CREATE TABLE` statement from the `sqlite_master` table, split on the last `)`, split those tokens on `,` and see if `create` is in there (case insensitive).

_Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/344#issuecomment-982020757_",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/345/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
713209404,MDU6SXNzdWU3MTMyMDk0MDQ=,988,Mechanism for plugins to construct URLs that respect base_url,9599,simonw,closed,0,,,6026070,0.51,2,2020-10-01T21:54:15Z,2020-10-23T19:44:05Z,2020-10-15T23:01:02Z,OWNER,,"> Had a thought: this is likely to break in plugins too, such as `datasette-edit-schema` which constructs URLs for redirects e.g. here: https://github.com/simonw/datasette-edit-schema/blob/dbd0abee6dd3385b114cfe9671f7ead1c4855b60/datasette_edit_schema/__init__.py#L46-L48

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/865#issuecomment-702418045_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/988/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1058815557,I_kwDOBm6k_c4_HD5F,1521,Docker configuration for exercising Datasette behind Apache mod_proxy,9599,simonw,closed,0,,,,,10,2021-11-19T18:46:18Z,2021-11-19T20:32:29Z,2021-11-19T20:32:29Z,OWNER,,"> Having a live demo running on Cloud Run that proxies through Apache and uses `base_url` would be incredibly useful for replicating and debugging this kind of thing. I wonder how hard it is to run Apache and `mod_proxy` in the same Docker container as Datasette?

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1519#issuecomment-974310208_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1521/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
700728217,MDU6SXNzdWU3MDA3MjgyMTc=,964,raise_404 mechanism for custom templates,9599,simonw,closed,0,,,5818042,Datasette 0.49,1,2020-09-14T03:22:15Z,2020-09-14T17:49:44Z,2020-09-14T17:39:34Z,OWNER,,"> Having tried this out I think it does need a `raise_404()` mechanism - which needs to be smart enough to trigger the default 404 handler without accidentally going into an infinite loop.

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/944#issuecomment-691788478_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/964/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1083726550,I_kwDOBm6k_c5AmFrW,1568,Trace should show queries on the write connection too,9599,simonw,closed,0,,,7571612,Datasette 0.60,2,2021-12-18T02:34:12Z,2022-01-13T22:27:23Z,2021-12-18T02:42:34Z,OWNER,,"> Here's why - `trace` only applies to read, not write SQL operations: https://github.com/simonw/datasette/blob/7c8f8aa209e4ba7bf83976f8495d67c28fbfca24/datasette/database.py#L209-L211

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1555#issuecomment-997128508_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1568/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
665819048,MDU6SXNzdWU2NjU4MTkwNDg=,126,Ability to insert binary data on the CLI using JSON,9599,simonw,closed,0,,,,,2,2020-07-26T16:54:14Z,2020-07-27T04:00:33Z,2020-07-27T03:59:45Z,OWNER,,"> I could solve round tripping (at least a bit) by allowing insert to be run with a flag that says ""these columns are base64 encoded, store the decoded data in a BLOB"".
>
> That would solve inserting binary data using JSON too.
_Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/125#issuecomment-664012247_",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/126/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
990367646,MDU6SXNzdWU5OTAzNjc2NDY=,1462,"Separate out ""debug"" options from ""root"" options",9599,simonw,open,0,,,,,1,2021-09-07T21:27:34Z,2021-09-07T21:34:33Z,,OWNER,,"> I ditched ""root"" for ""admin"" because root by default gives you a whole bunch of stuff which I think could be confusing:
> 
> <img width=""816"" alt=""Datasette___internal__temporary_and_datasette-app_—_Electron_Helper__Renderer__◂_npm_start_TMPDIR__var_folders_wr_hn3206rs1yzgq3r49bz8nvnh0000gn_T__XPC_FLAGS_0x0_—_180×58_and_datasette-app-support_—_pipenv_shell_▸_zsh_—_147×76"" src=""https://user-images.githubusercontent.com/9599/132412648-b768495b-ead2-4abe-875a-fd090fe3d71d.png"">
>
> Maybe the real problem here is that I'm conflating ""root"" permissions with ""debug"" options. Perhaps there should be an extra Datasette mode that unlocks debug tools for the root user?

_Originally posted by @simonw in https://github.com/simonw/datasette-app-support/issues/8#issuecomment-914638998_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1462/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1468495358,I_kwDOBm6k_c5Xh3X-,1910,Check incoming column types on various write APIs,9599,simonw,open,0,,,8755003,Datasette 1.0a-next,0,2022-11-29T18:09:10Z,2022-12-13T05:29:09Z,,OWNER,,"> I do think this needs type checking - I just tried and you really can send a string to an integer column and have it work, which feels bad.

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1863#issuecomment-1331089156_
      ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1910/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1786258502,I_kwDOCGYnMM5qeCRG,565,Table renaming: db.rename_table() and sqlite-utils rename-table,9599,simonw,closed,0,,,,,6,2023-07-03T14:07:42Z,2023-07-22T22:12:40Z,2023-07-22T22:12:40Z,OWNER,,"> I find myself wanting two new features in `sqlite-utils`:
> - The ability to have the new transformed table set to a specific name, while keeping the old table around
> - The ability to rename a table (`sqlite-utils` doesn't have a table rename function at all right now)

_Originally posted by @simonw in https://github.com/simonw/llm/issues/65#issuecomment-1618375042_
            ",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/565/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1271426387,I_kwDOCGYnMM5LyG1T,444,CSV `extras_key=` and `ignore_extras=` equivalents for CLI tool,9599,simonw,open,0,,,,,5,2022-06-14T22:22:47Z,2022-07-07T16:39:18Z,,OWNER,,"> I forgot to add equivalents of `extras_key=` and `ignore_extras=` to the CLI tool - will do that in a separate issue.

_Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/440#issuecomment-1155767915_",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/444/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1529707837,I_kwDOBm6k_c5bLX09,1988,Reconsider pattern where plugins could break existing template context,9599,simonw,open,0,,,3268330,Datasette 1.0,4,2023-01-11T21:13:43Z,2023-01-11T21:25:05Z,,OWNER,,"> I hadn't run into an issue with plugins like `datasette-template-sql` interfering with the existing context for other features before! Definitely not a good thing.

_Originally posted by @simonw in https://github.com/simonw/datasette-write/issues/6#issuecomment-1379490596_
      ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1988/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
718949182,MDU6SXNzdWU3MTg5NDkxODI=,6,Better handling of OCR data,9599,simonw,closed,0,,,,,2,2020-10-11T23:20:52Z,2020-10-12T00:04:10Z,2020-10-12T00:04:10Z,MEMBER,,"> I haven't done the FTS on OCR yet. I'm going to move that to another ticket because it requires more thought.

_Originally posted by @simonw in https://github.com/dogsheep/evernote-to-sqlite/issues/4#issuecomment-706784028_",303218369,evernote-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/6/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1378636455,I_kwDOBm6k_c5SLFKn,1815,"`datasette publish provider .` to publish whole directory, similar to configuration directory mode",9599,simonw,open,0,,,,,0,2022-09-19T23:28:59Z,2022-09-19T23:29:11Z,,OWNER,,"> I haven't done this with any of my other `datasette publish` tools, but I do think it's a good idea. Being able to publish the entire directory - with templates and plugins and metadata - does seem very useful to me.

_Originally posted by @simonw in https://github.com/simonw/datasette-publish-fly/issues/23#issuecomment-1251673489_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1815/reactions"", ""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
808008305,MDU6SXNzdWU4MDgwMDgzMDU=,230,--sniff option for sniffing delimiters,9599,simonw,closed,0,,,,,8,2021-02-14T17:43:54Z,2021-02-14T21:15:33Z,2021-02-14T19:24:32Z,OWNER,,"> I just spotted that `csv.Sniffer` in the Python standard library has a `.has_header(sample)` method which detects if the first row appears to be a header or not, which is interesting. https://docs.python.org/3/library/csv.html#csv.Sniffer

_Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/228#issuecomment-778812050_",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/230/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
733390884,MDU6SXNzdWU3MzMzOTA4ODQ=,1070,load_template() example in documentation showing loading from a database,9599,simonw,closed,0,,,6026070,0.51,1,2020-10-30T17:45:03Z,2020-10-31T16:22:51Z,2020-10-31T16:22:45Z,OWNER,,"> I should include an example in the documentation that shows loading templates from a database table.
_Originally posted by @simonw in https://github.com/simonw/datasette/pull/1069#issuecomment-719664530_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1070/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1005891028,I_kwDOCGYnMM479K3U,329,Rethink approach to [ and ] in column names (currently throws error),9599,simonw,closed,0,,,,,12,2021-09-23T22:14:24Z,2021-11-15T02:57:51Z,2021-11-15T02:57:51Z,OWNER,,"> I think it's best to still keep `[` and `]` out of column names though. Transforming them into `(` and `)` seems reasonable - but should that happen here or in `sqlite-utils`? I think in `sqlite-utils`.

_Originally posted by @simonw in https://github.com/simonw/datasette-app/issues/121#issuecomment-926200398_

This is a rethinking of the solution to:

- https://github.com/simonw/sqlite-utils/issues/86",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/329/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
906993731,MDU6SXNzdWU5MDY5OTM3MzE=,1351,Get `?_trace=1` working with CSV and streaming CSVs,9599,simonw,closed,0,,,,,1,2021-05-31T03:02:15Z,2021-06-17T18:12:32Z,2021-06-01T15:50:09Z,OWNER,,"> I think it's worth getting `?_trace=1` to work with streaming CSV - this would have helped me spot this issue a long time ago.

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1349#issuecomment-851133125_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1351/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1077322009,I_kwDOCGYnMM5ANqEZ,355,Allow users to pass a full convert() function definition,9599,simonw,closed,0,,,,,4,2021-12-10T23:59:58Z,2021-12-11T00:51:15Z,2021-12-11T00:49:31Z,OWNER,,"> I think the fix for this is to change the rules about what code is accepted in both the `-` mode and the literal code string mode: you can pass in a Python expression, OR a fragment that gets turned into a function, OR code that implements its own `def convert(value)` function. So this would work too:
> ```sh
> sqlite-utils convert my.db mytable col1 '
> def convert(value):
>     return value.upper()
> '
> ```
_Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/353#issuecomment-991381679_",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/355/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1174697144,I_kwDOBm6k_c5GBHS4,1672,Refactor CSV handling code out of DataView,9599,simonw,open,0,,,3268330,Datasette 1.0,1,2022-03-20T21:47:00Z,2022-03-20T21:52:39Z,,OWNER,,"> I think the way to get rid of most of the remaining complexity in `DataView` is to refactor how CSV stuff works - pulling it in line with other export factors and extracting the streaming mechanism. Opening a fresh issue for that.

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1660#issuecomment-1073355032_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1672/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
723460107,MDU6SXNzdWU3MjM0NjAxMDc=,187,Maybe: Utility method / CLI tool for initializing SpatiaLite,9599,simonw,closed,0,,,,,2,2020-10-16T19:04:03Z,2022-02-05T00:04:26Z,2020-10-16T19:15:13Z,OWNER,,"> I think this should initialize SpatiaLite against the current database if it has not been initialized already.
>
> Relevant code: https://github.com/simonw/shapefile-to-sqlite/blob/e754d0747ca2facf9a7433e2d5d15a6a37a9cf6e/shapefile_to_sqlite/utils.py#L112-L126",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/187/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
736520310,MDU6SXNzdWU3MzY1MjAzMTA=,196,Introspect if table is FTS4 or FTS5,9599,simonw,closed,0,,,,,19,2020-11-05T00:45:50Z,2020-11-05T03:54:07Z,2020-11-05T03:54:07Z,OWNER,,"> I want `.search()` to work against both FTS5 and FTS4 tables - but sort by rank should only work for FTS5.
>
> This means I need to be able to introspect and tell if a table is FTS4 or FTS5.

_Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/192#issuecomment-722054264_",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/196/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1223241647,I_kwDOBm6k_c5I6S-v,1734,Remove python-baseconv dependency,9599,simonw,closed,0,,,,,3,2022-05-02T19:08:37Z,2022-05-02T23:25:49Z,2022-05-02T19:39:20Z,OWNER,,"> I was going to vendor `baseconv.py`, but then I reconsidered - what if there are plugins out there that expect `import baseconv` to work because they have depended on Datasette?
>
> I used https://cs.github.com/ and as far as I can tell there aren't any!
>
> So I'm going to remove that dependency and work out a smarter way to do this - probably by providing a utility function within Datasette itself.

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1733#issuecomment-1115258737_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1734/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
749283032,MDU6SXNzdWU3NDkyODMwMzI=,1101,register_output_renderer() should support streaming data,9599,simonw,open,0,,,3268330,Datasette 1.0,13,2020-11-24T02:17:09Z,2023-01-21T22:07:19Z,,OWNER,,"> I'd like to implement this by first extending the `register_output_renderer()` hook to support streaming huge responses, then switching CSV to use the plugin hook in addition to TSV using it.

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1096#issuecomment-732542285_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1101/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
749983857,MDU6SXNzdWU3NDk5ODM4NTc=,1106,Rebrand and redirect config.rst as settings.rst,9599,simonw,closed,0,,,6055094,Datasette 0.52,4,2020-11-24T19:38:17Z,2020-11-24T21:39:58Z,2020-11-24T21:39:58Z,OWNER,,"> I'd like to redirect https://docs.datasette.io/en/stable/config.html to a new https://docs.datasette.io/en/stable/settings.html page too. I can use https://docs.readthedocs.io/en/stable/user-defined-redirects.html for that.

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1105#issuecomment-733190827_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1106/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
704685890,MDU6SXNzdWU3MDQ2ODU4OTA=,25,template_debug mechanism,9599,simonw,closed,0,,,,,2,2020-09-18T22:11:09Z,2020-09-18T22:12:21Z,2020-09-18T22:12:03Z,MEMBER,,"> I'd prefer it if errors in these template fragments were displayed as errors inline where the fragment should have been inserted, rather than 500ing the whole page - especially since the template fragments are user-provided and could have all kinds of odd errors in them which should be as easy to debug as possible.
_Originally posted by @simonw in https://github.com/dogsheep/dogsheep-beta/issues/24#issuecomment-694554584_",197431109,dogsheep-beta,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-beta/issues/25/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
783714076,MDU6SXNzdWU3ODM3MTQwNzY=,1184,request.full_path property,9599,simonw,closed,0,,,6346396,Datasette 0.54,0,2021-01-11T21:21:58Z,2021-01-24T21:21:16Z,2021-01-11T21:34:47Z,OWNER,,"> I'll also add `request.full_path` for consistency with these: https://github.com/simonw/datasette/blob/97fb10c17dd007a275ab743742e93e932335ad67/datasette/utils/asgi.py#L77-L90

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1179#issuecomment-755495387_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1184/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1058896236,I_kwDOBm6k_c4_HXls,1522,Deploy a live instance of demos/apache-proxy,9599,simonw,closed,0,,,,,34,2021-11-19T20:32:55Z,2021-11-23T03:00:34Z,2021-11-20T18:51:56Z,OWNER,,"> I'll get this working on my laptop first, but then I want to get it up and running on Cloud Run - maybe with a GitHub Actions  workflow in this repo that re-deploys it on manual execution.

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1521#issuecomment-974322178_

I started by following https://ahmet.im/blog/cloud-run-multiple-processes-easy-way/ - see example in https://github.com/ahmetb/multi-process-container-lazy-solution",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1522/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
635076066,MDU6SXNzdWU2MzUwNzYwNjY=,821,Add Response class to internals documentation,9599,simonw,closed,0,,,5512395,Datasette 0.44,0,2020-06-09T03:11:06Z,2020-06-09T03:32:16Z,2020-06-09T03:32:16Z,OWNER,,"> I'll need to add documentation of the `Response` object (and `Response.html()` and `Response.text()` class methods - I should add `Response.json()` too) to the internals page https://datasette.readthedocs.io/en/stable/internals.html

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/215#issuecomment-640971470_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/821/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
642652808,MDU6SXNzdWU2NDI2NTI4MDg=,861,Script to generate larger SQLite test files,9599,simonw,closed,0,,,,,3,2020-06-21T22:30:58Z,2020-06-23T03:44:18Z,2020-06-23T03:44:18Z,OWNER,,"> I'll write a little script which generates a 300MB SQLite file with a bunch of tables with lots of randomly generated rows in to help test this.
> 
> Having a tool like that which can generate larger databases with different gnarly performance characteristics will be useful for other performance work too.
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/859#issuecomment-647189948_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/861/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1223263540,I_kwDOBm6k_c5I6YU0,1735,Datasette setting to disable threading (for Pyodide),9599,simonw,closed,0,,,,,1,2022-05-02T19:31:08Z,2022-05-02T23:25:49Z,2022-05-02T20:13:52Z,OWNER,,"> I'm going to add a Datasette setting to disable threading entirely, designed for usage in this particular case.
>
> I thought about adding a new setting, then I noticed this:
>
>     datasette mydatabase.db --setting num_sql_threads 10
>
> I'm going to let users set that to `0` to disable threaded execution of SQL queries.

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1733#issuecomment-1115278325_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1735/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
808028757,MDU6SXNzdWU4MDgwMjg3NTc=,231,"limit=X, offset=Y parameters for more Python methods",9599,simonw,closed,0,,,,,2,2021-02-14T19:31:23Z,2021-02-14T20:03:08Z,2021-02-14T20:03:08Z,OWNER,,"> I'm going to add a `offset=` parameter to support this case. Thanks for the suggestion!

_Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/224#issuecomment-778828495_",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/231/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
792931244,MDU6SXNzdWU3OTI5MzEyNDQ=,1202,Documentation convention for marking unstable APIs.,9599,simonw,closed,0,,,6346396,Datasette 0.54,2,2021-01-24T23:47:18Z,2021-01-25T00:01:02Z,2021-01-25T00:01:02Z,OWNER,,"> I'm going to document this but mark it as unstable, using a new documentation convention for marking unstable APIs.

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1154#issuecomment-766462197_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1202/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
718264811,MDU6SXNzdWU3MTgyNjQ4MTE=,1006,Documentation for datasette.client,9599,simonw,closed,0,,,5971510,Datasette 0.50,2,2020-10-09T16:09:02Z,2020-10-09T17:22:31Z,2020-10-09T17:20:37Z,OWNER,,"> I'm going to document this in a separate issue.

_Originally posted by @simonw in https://github.com/simonw/datasette/pull/1000#issuecomment-706269271_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1006/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
628003707,MDU6SXNzdWU2MjgwMDM3MDc=,784,Ability to sign in to Datasette as a root account,9599,simonw,closed,0,,,5512395,Datasette 0.44,5,2020-05-31T17:10:15Z,2020-07-06T19:31:53Z,2020-06-01T01:18:20Z,OWNER,,"> I'm going to draw the line here: default Datasette supports authentication but only for a single user account (""admin""). Plugins can then add support for multiple user accounts, social auth, SSO etc.

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/699#issuecomment-636498770_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/784/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
776635426,MDU6SXNzdWU3NzY2MzU0MjY=,1165,Mechanism for executing JavaScript unit tests,9599,simonw,open,0,,,,,9,2020-12-30T21:02:34Z,2022-01-13T22:21:29Z,,OWNER,,"> I'm going to need to add JavaScript unit tests for this new plugin system.

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/983#issuecomment-752757289_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1165/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1175648453,I_kwDOBm6k_c5GEvjF,1675,Extract out `check_permissions()` from `BaseView,9599,simonw,closed,0,,,,,7,2022-03-21T16:39:46Z,2022-03-21T17:14:31Z,2022-03-21T17:13:21Z,OWNER,,"> I'm going to refactor this stuff out and document it so it can be easily used by plugins:

https://github.com/simonw/datasette/blob/4a4164b81191dec35e423486a208b05a9edc65e4/datasette/views/base.py#L69-L103

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1660#issuecomment-1074136176_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1675/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1174404647,I_kwDOBm6k_c5F__4n,1669,Release 0.61 alpha,9599,simonw,closed,0,,,,,2,2022-03-20T00:35:35Z,2022-03-20T01:24:36Z,2022-03-20T01:24:36Z,OWNER,,"> I'm going to release this as a 0.61 alpha so I can more easily depend on it from `datasette-hashed-urls`.

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1668#issuecomment-1073136896_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1669/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1816852402,I_kwDOCGYnMM5sSvey,569,register_command plugin hook,9599,simonw,closed,0,,,,,3,2023-07-22T18:17:27Z,2023-07-22T19:19:35Z,2023-07-22T19:19:35Z,OWNER,,"> I'm going to start by adding the `register_command` hook using the exact same pattern as Datasette and LLM.

_Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/567#issuecomment-1646643450_
            ",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/569/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
808036774,MDU6SXNzdWU4MDgwMzY3NzQ=,232,Run tests against Windows in GitHub Actions,9599,simonw,closed,0,,,,,0,2021-02-14T20:09:45Z,2021-02-14T20:39:55Z,2021-02-14T20:39:55Z,OWNER,,"> I'm going to try and get the test suite to run in Windows on GitHub Actions.

_Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/225#issuecomment-778834504_",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/232/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1865649347,I_kwDOBm6k_c5vM4zD,2156,datasette -s/--setting option for setting nested configuration options,9599,simonw,open,0,,,,,4,2023-08-24T18:09:27Z,2023-08-28T19:33:05Z,,OWNER,,"> I've been thinking about what it might look like to allow command-line arguments to be used to define _any_ of the configuration options in `datasette.yml`, as alternative and more convenient syntax.
>
> Here's what I've come up with:
> ```
> datasette \
>   -s settings.sql_time_limit_ms 1000 \
>   -s plugins.datasette-auth-tokens.manage_tokens true \
>   -s plugins.datasette-auth-tokens.manage_tokens_database tokens \
>  mydatabase.db tokens.db
> ```
> Which would be equivalent to `datasette.yml` containing this:
> ```yaml
> plugins:
>   datasette-auth-tokens:
>     manage_tokens: true
>     manage_tokens_database: tokens
> settings:
>   sql_time_limit_ms: 1000
> ```
More details in https://github.com/simonw/datasette/issues/2143#issuecomment-1690792514
            ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2156/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1084257842,I_kwDOBm6k_c5AoHYy,1575,__call__() got an unexpected keyword argument 'specname',9599,simonw,closed,0,,,,,1,2021-12-20T01:24:04Z,2021-12-20T01:48:03Z,2021-12-20T01:47:57Z,OWNER,,"> I've installed the alpha version but get an error when starting up Datasette:

```
Traceback (most recent call last):
  File ""/Users/tim/.pyenv/versions/stock-exchange/bin/datasette"", line 5, in <module>
    from datasette.cli import cli
  File ""/Users/tim/.pyenv/versions/3.8.5/envs/stock-exchange/lib/python3.8/site-packages/datasette/cli.py"", line 15, in <module>
    from .app import Datasette, DEFAULT_SETTINGS, SETTINGS, SQLITE_LIMIT_ATTACHED, pm
  File ""/Users/tim/.pyenv/versions/3.8.5/envs/stock-exchange/lib/python3.8/site-packages/datasette/app.py"", line 31, in <module>
    from .views.database import DatabaseDownload, DatabaseView
  File ""/Users/tim/.pyenv/versions/3.8.5/envs/stock-exchange/lib/python3.8/site-packages/datasette/views/database.py"", line 25, in <module>
    from datasette.plugins import pm
  File ""/Users/tim/.pyenv/versions/3.8.5/envs/stock-exchange/lib/python3.8/site-packages/datasette/plugins.py"", line 29, in <module>
    mod = importlib.import_module(plugin)
  File ""/Users/tim/.pyenv/versions/3.8.5/lib/python3.8/importlib/__init__.py"", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File ""/Users/tim/.pyenv/versions/3.8.5/envs/stock-exchange/lib/python3.8/site-packages/datasette/filters.py"", line 9, in <module>
    @hookimpl(specname=""filters_from_request"")
TypeError: __call__() got an unexpected keyword argument 'specname'
```

_Originally posted by @wragge in https://github.com/simonw/datasette/issues/1547#issuecomment-997511968_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1575/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
448391492,MDU6SXNzdWU0NDgzOTE0OTI=,21,Option to ignore inserts if primary key exists already,9599,simonw,closed,0,,,,,3,2019-05-25T00:17:12Z,2019-05-29T05:09:01Z,2019-05-29T04:18:26Z,OWNER,,"> I've just noticed that SQLite lets you IGNORE inserts that collide with a pre-existing key. This can be quite handy if you have a dataset that keeps changing in part, and you don't want to upsert and replace pre-existing PK rows but you do want to ignore collisions to existing PK rows.
> 
> Do `sqlite_utils` support such (cavalier!) behaviour?

_Originally posted by @psychemedia in https://github.com/simonw/sqlite-utils/issues/18#issuecomment-480621924_",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/21/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1169840669,I_kwDOBm6k_c5Fulod,1658,Revert main to version that passes tests,9599,simonw,closed,0,,,3268330,Datasette 1.0,1,2022-03-15T15:37:02Z,2022-03-19T04:04:50Z,2022-03-15T15:42:58Z,OWNER,,"> I've made a real mess of this. I'm going to revert Datasette`main` back to the last commit that passed the tests and try this again in a branch.

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1657#issuecomment-1068125636_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1658/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1490576818,I_kwDOBm6k_c5Y2GWy,1943,`/-/permissions` should list available permissions,9599,simonw,open,0,,,8755003,Datasette 1.0a-next,1,2022-12-11T23:38:03Z,2022-12-15T00:41:37Z,,OWNER,,"> Idea: a `/-/permissions` introspection endpoint for listing registered permissions

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1939#issuecomment-1345691103_
      ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1943/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
631932926,MDU6SXNzdWU2MzE5MzI5MjY=,801,allow_by_query setting for configuring permissions with a SQL statement,9599,simonw,closed,0,,,3268330,Datasette 1.0,6,2020-06-05T20:30:19Z,2020-06-11T18:58:56Z,2020-06-11T18:58:49Z,OWNER,,"> Idea: an `""allow_sql""` key with a SQL query that gets passed the actor JSON as `:actor` and can extract the relevant keys from it and return 1 or 0.

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/698#issuecomment-639787304_

See also #800",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/801/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
631931408,MDU6SXNzdWU2MzE5MzE0MDg=,800,Canned query permissions mechanism,9599,simonw,closed,0,,,5512395,Datasette 0.44,14,2020-06-05T20:28:21Z,2020-06-07T16:22:53Z,2020-06-07T16:22:53Z,OWNER,,"> Idea: default is anyone can execute a query.
> 
> Or you can specify the following:
> 
> ```json
> 
> {
>     ""databases"": {
>        ""my-database"": {
>            ""queries"": {
>                ""add_twitter_handle"": {
>                    ""sql"": ""insert into twitter_handles (username) values (:username)"",
>                    ""write"": true,
>                    ""allow"": {
>                        ""id"": [""simon""],
>                        ""role"": [""staff""]
>                    }
>                }
>            }
>        }
>     }
> }
> ```
> These get matched against the actor JSON. If any of the fields in any of the keys of `""allow""` match a key on the actor, the query is allowed.
> 
> `""id"": ""*""` matches any actor with an `id` key.

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/698#issuecomment-639784651_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/800/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
708289783,MDU6SXNzdWU3MDgyODk3ODM=,976,Idea: -o could open to a more convenient location,9599,simonw,closed,0,,,,,2,2020-09-24T15:56:35Z,2020-10-26T05:07:10Z,2020-10-26T05:06:26Z,OWNER,,"> Idea: if a database only has a single table, this could open straight to `/db/table`. If it has multiple tables but a single database it could open straight to `/db`.
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/970#issuecomment-698434236_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/976/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1083895395,I_kwDOBm6k_c5Amu5j,1569,"db.execute_write(..., executescript=True) parameter",9599,simonw,closed,0,,,7571612,Datasette 0.60,2,2021-12-18T18:20:47Z,2022-01-13T22:27:27Z,2021-12-18T18:34:18Z,OWNER,,"> Idea: teach `execute_write` to accept an optional `executescript=True` parameter, like this:
```diff
diff --git a/datasette/database.py b/datasette/database.py
index 468e936..1a424f5 100644
--- a/datasette/database.py
+++ b/datasette/database.py
@@ -94,10 +94,14 @@ class Database:
             f""file:{self.path}{qs}"", uri=True, check_same_thread=False
         )
 
-    async def execute_write(self, sql, params=None, block=False):
+    async def execute_write(self, sql, params=None, executescript=False, block=False):
+        assert not executescript and params, ""Cannot use params with executescript=True""
         def _inner(conn):
             with conn:
-                return conn.execute(sql, params or [])
+                if executescript:
+                    return conn.executescript(sql)
+                else:
+                    return conn.execute(sql, params or [])
 
         with trace(""sql"", database=self.name, sql=sql.strip(), params=params):
             results = await self.execute_write_fn(_inner, block=block)
```

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1555#issuecomment-997248364_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1569/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
957315684,MDU6SXNzdWU5NTczMTU2ODQ=,1410,Rename settings to `default_allow_facet` and `default_allow_download` and `default_allow_csv_stream`,9599,simonw,open,0,,,3268330,Datasette 1.0,0,2021-07-31T20:27:12Z,2021-07-31T20:27:49Z,,OWNER,,"> If I was prone to over-thinking (which I am) I'd note that `allow_facet` and `allow_download` and `allow_csv_stream` are all settings that do NOT have an equivalent in the newer permissions system, which is itself a little weird and inconsistent.
>
> So maybe there's a future task where I introduce those as both permissions and metadata `""allow_x""` blocks, then rename the settings themselves to be called `default_allow_facet` and `default_allow_download` and `default_allow_csv_stream`.
>
> If I was going to do that I should get it in before Datasette 1.0.

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1409#issuecomment-890400425_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1410/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
776634318,MDU6SXNzdWU3NzY2MzQzMTg=,1164,Mechanism for minifying JavaScript that ships with Datasette,9599,simonw,open,0,,,,,9,2020-12-30T20:59:06Z,2022-01-13T22:21:29Z,,OWNER,,"> If I'm going to minify it I'll need to figure out a build step in Datasette itself so that I can easily work on that minified version.

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/983#issuecomment-752748496_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1164/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1740150327,I_kwDOCGYnMM5nuJY3,557,Aliased ROWID option for tables created from alter=True commands,7908073,chapmanjacobd,closed,0,,,,,2,2023-06-04T05:29:28Z,2023-06-14T06:09:21Z,2023-06-05T19:26:26Z,CONTRIBUTOR,,"> If you use INTEGER PRIMARY KEY column, the VACUUM does not change the values of that column. However, if you use unaliased rowid, the VACUUM command will reset the rowid values.

ROWID should never be used with foreign keys but the simple act of aliasing rowid to id (which is what happens when one does `id integer primary key` DDL) makes it OK.

It would be convenient if there were more options to use a string column (eg. filepath) as the PK, and be able to use it during upserts, but when creating a foreign key, to create an integer column which aliases rowid

I made an attempt to switch to integer primary keys here but it is not going well... In my usecase the path column is a business key. Yes, it should be as simple as including the `id` column in any select statement where I plan on using `upsert` but it would be nice if this could be abstracted away somehow  https://github.com/chapmanjacobd/library/commit/788cd125be01d76f0fe2153335d9f6b21db1343c

https://github.com/chapmanjacobd/library/actions/runs/5173602136/jobs/9319024777",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/557/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
903902495,MDU6SXNzdWU5MDM5MDI0OTU=,1342,Improve `path_with_replaced_args()` and friends and document them,9599,simonw,open,0,,,,,3,2021-05-27T15:18:28Z,2021-05-27T15:23:02Z,,OWNER,,"> In order to cleanly implement this I need to expose the `path_with_replaced_args` utility function to Datasette's template engine. This is the first time this will become an exposed (and hence should-by-documented) API and I don't like its shape much.

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1337#issuecomment-849721280_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1342/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1447388809,I_kwDOBm6k_c5WRWaJ,1887,Add a confirm step to the drop table API,9599,simonw,closed,0,,,8658075,Datasette 1.0a0,2,2022-11-14T04:59:53Z,2022-11-15T19:59:59Z,2022-11-14T05:18:51Z,OWNER,,"> In playing with the API explorer just now I realized it's way too easy to accidentally drop a table using it.

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1871#issuecomment-1313097057_

Added drop table API in:
- #1874",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1887/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
666040390,MDU6SXNzdWU2NjYwNDAzOTA=,127,Ability to insert files piped to insert-files stdin,9599,simonw,closed,0,,,,,3,2020-07-27T07:09:33Z,2020-07-30T03:08:52Z,2020-07-30T03:08:18Z,OWNER,,"> Inserting files by piping them in should work - but since a filename cannot be derived this will need a `--name blah.gif` option.
>
>    cat blah.gif | sqlite-utils insert-files files.db files - --name=blah.gif
>
_Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/122#issuecomment-664128071_",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/127/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1353189941,I_kwDOCGYnMM5QqAo1,475,table.default_values introspection property,9599,simonw,closed,0,,,8355157,3.29,1,2022-08-27T22:33:31Z,2022-08-27T22:44:46Z,2022-08-27T22:43:02Z,OWNER,,"> Interesting challenge with `default_value`: I need to be able to tell if the default values passed to `.create()` differ from those in the database already.
>
> Introspecting that is a bit tricky:
>
> ```pycon
> >>> import sqlite_utils
> >>> db = sqlite_utils.Database(memory=True)
> >>> db[""blah""].create({""id"": int, ""name"": str}, not_null=(""name"",), defaults={""name"": ""bob""})
> <Table blah (id, name)>
> >>> db[""blah""].columns
> [Column(cid=0, name='id', type='INTEGER', notnull=0, default_value=None, is_pk=0), Column(cid=1, name='name', type='TEXT', notnull=1, default_value=""'bob'"", is_pk=0)]
> ```
> Note how a default value of the Python string `bob` is represented in the results of `PRAGMA table_info()` as `default_value=""'bob'""` - it's got single quotes added to it!
> 
> So comparing default values from introspecting the database needs me to first parse that syntax. This may require a new table introspection method.

_Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/468#issuecomment-1229279539_",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/475/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1097091527,I_kwDOCGYnMM5BZEnH,369,Research how much of a difference analyze / sqlite_stat1 makes,9599,simonw,closed,0,,,,,11,2022-01-09T03:03:36Z,2022-02-03T21:07:41Z,2022-02-03T21:07:35Z,OWNER,,"> Is there a downside to having a `sqlite_stat1` table if it has wildly incorrect statistics in it?

_Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/365#issuecomment-1008163050_

More generally: how much of a difference does the `sqlite_stat1` table created by `ANALYZE` make to queries?

I'm particularly interested in `group by` / `count *` queries since Datasette uses those for faceting.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/369/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
733768037,MDU6SXNzdWU3MzM3NjgwMzc=,1074,latest.datasette.io should include plugins from fixtures,9599,simonw,closed,0,,,6026070,0.51,3,2020-10-31T17:23:23Z,2020-10-31T19:47:47Z,2020-10-31T19:47:47Z,OWNER,,"> It bothers me that these aren't visible in any public demos. Maybe `latest.datasette.io` should include the `my_plugins.py` and `my_plugins2.py` plugins?
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1067#issuecomment-719961701_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1074/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
706098005,MDU6SXNzdWU3MDYwOTgwMDU=,167,Review the foreign key pragma stuff,9599,simonw,closed,0,,,5897911,2.20,1,2020-09-22T05:55:20Z,2020-09-23T00:13:02Z,2020-09-23T00:13:02Z,OWNER,,"> It is not possible to enable or disable foreign key constraints in the middle of a multi-statement transaction (when SQLite is not in autocommit mode). Attempting to do so does not return an error; it simply has no effect.

https://sqlite.org/foreignkeys.html",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/167/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
903978133,MDU6SXNzdWU5MDM5NzgxMzM=,1343,Figure out how to publish alpha/beta releases to Docker Hub,9599,simonw,closed,0,,,,,4,2021-05-27T16:42:17Z,2021-05-27T16:46:37Z,2021-05-27T16:45:41Z,OWNER,,"> It looks like all I need to do to ship an alpha version to Docker Hub is NOT point the `latest` tag at it after it goes live: https://github.com/simonw/datasette/blob/1a8972f9c012cd22b088c6b70661a9c3d3847853/.github/workflows/publish.yml#L75-L77

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1319#issuecomment-849780481_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1343/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
665407663,MDU6SXNzdWU2NjU0MDc2NjM=,908,"Interactive debugging tool for ""allow"" blocks",9599,simonw,closed,0,,,5607421,Datasette 0.46,3,2020-07-24T20:43:44Z,2020-07-25T00:06:15Z,2020-07-24T22:56:52Z,OWNER,,"> It might be good to have a little interactive tool which helps debug these things, since there are quite a few edge-cases and the damage caused if people use them incorrectly is substantial.
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/907#issuecomment-663726146_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/908/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1378640768,I_kwDOBm6k_c5SLGOA,1816,Validate settings.json on startup in configuration directory mode,9599,simonw,closed,0,,,,,2,2022-09-19T23:35:18Z,2022-09-20T01:15:48Z,2022-09-20T01:15:48Z,OWNER,,"> It might have been useful for Datasette to show an error when started against a `settings.json` file that contains an invalid setting though.

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1814#issuecomment-1251677554_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1816/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1251739062,I_kwDOBm6k_c5KnAW2,1752,Research if I can drop Janus,9599,simonw,open,0,,,,,0,2022-05-28T22:46:52Z,2022-05-28T22:46:52Z,,OWNER,,"> It seems to me Janus dependency is not necessary, `async with app.database_write_mutex(): out = await app.transaction(func)` may be enough.

Comment here: https://lobste.rs/s/fki4tj/architecture_notes_datasette#c_a2ihon",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1752/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1423347412,I_kwDOBm6k_c5U1o7U,1857,Prevent API tokens from using /-/create-token to create more tokens,9599,simonw,closed,0,,,8658075,Datasette 1.0a0,1,2022-10-26T02:38:09Z,2022-11-15T19:57:11Z,2022-10-26T02:57:26Z,OWNER,,"> It strikes me that users should NOT be able to use a token to create additional tokens.
>
> The current design actually does allow that, since the `dstok_` Bearer token can be used to authenticate calls to the `/-/create-token` page.
>
> So I think I need a mechanism whereby that page can only allow access to users authenticated by cookie.
> 
> Not obvious how to do that though, since Datasette's authentication actor system is designed to abstract that detail away!

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1850#issuecomment-1291417100_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1857/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
684961449,MDU6SXNzdWU2ODQ5NjE0NDk=,949,Try out CodeMirror SQL hints,9599,simonw,closed,0,,,,,5,2020-08-24T20:58:21Z,2023-11-03T05:28:58Z,2020-11-01T03:29:48Z,OWNER,,"> It would also be interesting to try out the SQL hint mode, which can autocomplete against tables and columns. This demo shows how to configure that: https://codemirror.net/mode/sql/
> 
> Some missing documentation: https://stackoverflow.com/questions/20023381/codemirror-how-add-tables-to-sql-hint
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/948#issuecomment-679355426_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/949/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1495716243,I_kwDOBm6k_c5ZJtGT,1952,Improvements to /-/create-token restrictions interface,9599,simonw,open,0,,,8755003,Datasette 1.0a-next,1,2022-12-14T05:22:39Z,2022-12-14T05:23:13Z,,OWNER,,"> It would be neat not to show write permissions against immutable databases too - and not hard from a performance perspective since it doesn't involve hundreds more permission checks.
>
> That will need permissions to grow a flag for if they need a mutable database though, which is a bigger job.

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1947#issuecomment-1350414402_

Also, DO show the `_memory` database there if Datasette was started in `--crossdb` mode.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1952/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
712260429,MDU6SXNzdWU3MTIyNjA0Mjk=,983,JavaScript plugin hooks mechanism similar to pluggy,9599,simonw,open,0,,,,,47,2020-09-30T20:32:43Z,2021-01-25T04:43:58Z,,OWNER,,"> It would be neat to provide a JavaScript plugin hook that plugins can use to add their own options to this menu. No idea what that would look like though.

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/981#issuecomment-701616922_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/983/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
670209331,MDU6SXNzdWU2NzAyMDkzMzE=,913,Mechanism for passing additional options to `datasette my.db` that affect plugins,9599,simonw,open,0,,,,,5,2020-07-31T20:38:26Z,2021-01-04T20:04:11Z,,OWNER,,"> It's a shame there's no obvious mechanism for passing additional options to `datasette my.db` that affect how plugins work.
>
>The only way I can think of at the moment is via environment variables:
>
>     DATASETTE_INSERT_UNSAFE=1 datasette my.db
>
>This will have to do for the moment - it's ugly enough that people will at least know they are doing something unsafe, which is the goal here.
_Originally posted by @simonw in https://github.com/simonw/datasette-insert/issues/15#issuecomment-667346438_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/913/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
742011049,MDU6SXNzdWU3NDIwMTEwNDk=,1091,.json and .csv exports fail to apply base_url,9599,simonw,closed,0,,,6346396,Datasette 0.54,22,2020-11-12T23:45:16Z,2021-01-24T21:20:24Z,2021-01-09T22:19:29Z,OWNER,,"> Just tested with the latest Docker image, and it works pretty much everywhere!  THANK YOU!
> 
> I did notice that if I try to export json or csv, the base is not applied.  Not sure if I should reopen this issue or open a new one.
> 
> To see this, go here: https://corpora.tika.apache.org/datasette/corpora-metadata/REF_PARSE_EXCEPTION_TYPES
> 
> Click/hover over json or CSV and you'll see that the 'datasette' base is not included.

_Originally posted by @tballison in https://github.com/simonw/datasette/issues/865#issuecomment-726385422_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1091/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
837350092,MDU6SXNzdWU4MzczNTAwOTI=,1270,Try implementing SQLite timeouts using .interrupt() instead of using .set_progress_handler(),9599,simonw,open,0,,,,,3,2021-03-22T06:00:17Z,2021-03-23T16:45:39Z,,OWNER,,"> Maybe I could implement SQLite query timeouts using the `interrupt()` method instead of the progress handler hack I'm currently using?
>
> https://stackoverflow.com/questions/43240496/python-sqlite3-how-to-quickly-and-cleanly-interrupt-long-running-query-with-e has some tips.

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1268#issuecomment-803764919_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1270/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
718259202,MDU6SXNzdWU3MTgyNTkyMDI=,1005,Remove xfail tests when new httpx is released,9599,simonw,closed,0,,,3268330,Datasette 1.0,3,2020-10-09T16:00:19Z,2021-02-28T22:41:08Z,2021-02-28T22:41:08Z,OWNER,,"> My `httpx` pull request adding `raw_path` support was just merged: https://github.com/encode/httpx/pull/1357 - but it's not in a release yet.
>
> I'm going to mark these tests as `xfail` so I can land this change - I'll remove that once an `httpx` release comes out that I can use to get the tests passing.
>
_Originally posted by @simonw in https://github.com/simonw/datasette/pull/1000#issuecomment-706263157_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1005/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1066563554,I_kwDOCGYnMM4_knfi,346,Way to test SQLite 3.37 (and potentially other versions) in CI,9599,simonw,open,0,,,,,5,2021-11-29T22:21:06Z,2021-11-29T23:12:49Z,,OWNER,,"> Need to figure out a good pattern for testing this in CI too - it will currently skip the new tests if it doesn't have SQLite 3.37 or higher.

_Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/344#issuecomment-982076924_",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/346/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
708261775,MDU6SXNzdWU3MDgyNjE3NzU=,175,Add docs for .transform(column_order=),9599,simonw,closed,0,,,,,3,2020-09-24T15:19:04Z,2020-09-24T20:35:48Z,2020-09-24T16:00:56Z,OWNER,,"> Need to update docs for `.transform()` now that `column_order=` is available.
_Originally posted by @simonw in https://github.com/simonw/sqlite-utils/pull/174#discussion_r494403327_

Maybe also add this as an option to `sqlite-utils transform` - since reordering columns is actually a pretty nice capability.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/175/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
771208009,MDU6SXNzdWU3NzEyMDgwMDk=,1154,Documentation for new _internal database and tables,9599,simonw,closed,0,,,6346396,Datasette 0.54,2,2020-12-18T22:34:52Z,2021-01-25T00:09:22Z,2021-01-25T00:08:41Z,OWNER,,"> Needs documentation, but I can wait to write that until I've tested out the feature a bit more.

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1150#issuecomment-748352106_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1154/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
770448622,MDU6SXNzdWU3NzA0NDg2MjI=,1151,Database class mechanism for cross-connection in-memory databases,9599,simonw,closed,0,,,6346396,Datasette 0.54,11,2020-12-17T23:25:43Z,2021-01-26T19:07:44Z,2020-12-18T01:01:26Z,OWNER,,"> Next challenge: figure out how to use the `Database` class from https://github.com/simonw/datasette/blob/0.53/datasette/database.py for an in-memory database which persists data for the duration of the lifetime of the server, and allows access to that in-memory database from multiple threads in a way that lets them see each other's changes.

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1150#issuecomment-747768112_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1151/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1177101697,I_kwDOBm6k_c5GKSWB,1681,Potential bug in numeric handling where_clause for filters,9599,simonw,open,0,,,,,2,2022-03-22T17:43:50Z,2022-03-22T17:49:09Z,,OWNER,,"> Note that Datasette does already have special logic to convert parameters to integers for numeric comparisons like `>`:
>
> https://github.com/simonw/datasette/blob/c4c9dbd0386e46d2bf199f0ed34e4895c98cb78c/datasette/filters.py#L203-L212
> 
> Though... it looks like there's a bug in that? It doesn't account for `float` values - `""3.5"".isdigit()` return `False` - probably for the best, because `int(3.5)` would break that value anyway.

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1671#issuecomment-1075432283_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1681/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
959276629,MDU6SXNzdWU5NTkyNzY2Mjk=,1416,"Use rich to render tracebacks on errors, if available",9599,simonw,closed,0,,,,,0,2021-08-03T16:12:08Z,2021-08-03T16:12:51Z,2021-08-03T16:12:51Z,OWNER,,"> Now thinking I should try adding Rich as an optional dependency to Datasette - if it's there, show tracebacks using it. Could be really handy for development
> https://twitter.com/simonw/status/1422576091055616003",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1416/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1122557010,I_kwDOBm6k_c5C6NxS,1627,Get the tests passing against Windows,9599,simonw,open,0,,,,,0,2022-02-03T01:23:06Z,2022-02-03T01:23:32Z,,OWNER,,"> OK, the tests do NOT pass against Windows! https://github.com/simonw/datasette/runs/5044105941
>
> <img width=""767"" alt=""image"" src=""https://user-images.githubusercontent.com/9599/152265501-5b5af51b-0fae-4aa7-b86d-7b61aba24b3c.png"">

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1626#issuecomment-1028515161_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1627/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1160750713,I_kwDOBm6k_c5FL6Z5,1650,Implement redirects from old % encoding to new dash encoding,9599,simonw,closed,0,,,3268330,Datasette 1.0,5,2022-03-06T23:40:02Z,2022-03-07T19:26:15Z,2022-03-07T19:26:14Z,OWNER,,"> One big advantage to this scheme is that redirecting old links to `%2F` pages (e.g. https://fivethirtyeight.datasettes.com/fivethirtyeight/twitter-ratio%2Fsenators) is easy - if you see a `%` in the `raw_path`, redirect to that page with the `%` replaced by `-`.

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1439#issuecomment-1060044007_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1650/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1844213115,I_kwDOBm6k_c5t7HV7,2138,on_success_message_sql option for writable canned queries,9599,simonw,closed,0,,,8755003,Datasette 1.0a-next,2,2023-08-10T00:20:14Z,2023-08-10T00:39:40Z,2023-08-10T00:34:26Z,OWNER,,"> Or... how about if the `on_success_message` option could define a SQL query to be executed to generate that message? Maybe `on_success_message_sql`.

- https://github.com/simonw/datasette/issues/2134",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2138/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
628499086,MDU6SXNzdWU2Mjg0OTkwODY=,790,"""flash messages"" mechanism",9599,simonw,closed,0,,,5512395,Datasette 0.44,20,2020-06-01T14:55:44Z,2020-06-08T19:33:59Z,2020-06-02T21:14:03Z,OWNER,,"> Passing `?_success` like this isn't necessarily the best approach. Potential improvements include:
> 
> - Signing this message so it can't be tampered with (I could generate a signing secret on startup)
> - Using a cookie with a temporary flash message in it instead
> - Using HTML5 history API to remove the `?_success=` from the URL bar when the user lands on the page
> 
> If I add an option to redirect the user to another page after success I may need a mechanism to show a flash message on that page as well, in which case I'll need a general flash message solution that works for any page.

_Originally posted by @simonw in https://github.com/simonw/datasette/pull/703_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/790/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1083921371,I_kwDOBm6k_c5Am1Pb,1570,Separate db.execute_write() into three methods,9599,simonw,closed,0,,,7571612,Datasette 0.60,2,2021-12-18T18:45:54Z,2022-01-13T22:27:38Z,2021-12-18T18:57:25Z,OWNER,,"> Rather than adding a `executemany=True` parameter, I'm now thinking a better design might be to have three methods:
>
> - `db.execute_write(sql, params=None, block=False)`
> - `db.execute_write_script(sql, block=False)`
> - `db.execute_write_many(sql, params_seq, block=False)`

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1555#issuecomment-997267416_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1570/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1096563265,I_kwDOCGYnMM5BXDpB,366,Python library methods for calling ANALYZE,9599,simonw,closed,0,,,7558727,3.21,10,2022-01-07T18:28:01Z,2022-01-11T01:09:33Z,2022-01-11T01:09:33Z,OWNER,,"> Relevant documentation: https://www.sqlite.org/lang_analyze.html

_Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/365#issuecomment-1007633376_",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/366/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1097040427,I_kwDOBm6k_c5BY4Ir,1587,Add `sqlite_stat1`(-4) tables to hidden table list,9599,simonw,closed,0,,,,,2,2022-01-08T21:28:20Z,2022-01-20T04:12:59Z,2022-01-20T04:12:59Z,OWNER,,"> Running `ANALYZE` creates a new visible table called `sqlite_stat1`: https://www.sqlite.org/fileformat.html#the_sqlite_stat1_table
>
> This should be added to the default list of hidden tables in Datasette.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1587/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
706757891,MDU6SXNzdWU3MDY3NTc4OTE=,169,"Progress bar for ""sqlite-utils extract""",9599,simonw,closed,0,,,5897911,2.20,0,2020-09-22T23:40:21Z,2020-09-24T20:34:50Z,2020-09-23T00:02:40Z,OWNER,,"> Since these operations could take a long time against large tables, it would be neat if there was a progress bar option for the CLI command.
>
> The operations are full table scans so calculating progress shouldn't be too difficult.
_Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/42#issuecomment-513246831_",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/169/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1816918185,I_kwDOCGYnMM5sS_ip,574,`prepare_connection()` plugin hook,9599,simonw,closed,0,,,,,3,2023-07-22T22:52:47Z,2023-07-22T23:13:14Z,2023-07-22T22:59:10Z,OWNER,,"> Splitting off an issue for `prepare_connection()` since Alex got the PR in seconds before I shipped 3.34!

_Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/567#issuecomment-1646686424_
            ",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/574/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1805076818,I_kwDOBm6k_c5rl0lS,2102,API tokens with view-table but not view-database/view-instance cannot access the table,9599,simonw,closed,0,9599,simonw,,,20,2023-07-14T15:34:27Z,2023-08-29T16:32:36Z,2023-08-29T16:32:35Z,OWNER,,"> Spotted a problem while working on this: if you grant a token access to view table for a specific table but don't also grant view database and view instance permissions, that token is useless.
>
> This was a deliberate design decision in Datasette - it's documented on https://docs.datasette.io/en/1.0a2/authentication.html#access-permissions-in-metadata
>
>> If a user cannot access a specific database, they will not be able to access tables, views or queries within that database. If a user cannot access the instance they will not be able to access any of the databases, tables, views or queries.
>
> I'm now second-guessing if this was a good decision.

_Originally posted by @simonw in https://github.com/simonw/datasette-auth-tokens/issues/7#issuecomment-1636031702_
            ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2102/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
588108428,MDU6SXNzdWU1ODgxMDg0Mjg=,712,base_url doesn't entirely work for running Datasette inside Binder,9599,simonw,closed,0,,,,,12,2020-03-26T02:25:55Z,2020-03-26T15:11:49Z,2020-03-26T14:35:43Z,OWNER,,"> Thanks! I'm trying to launch Datasette from *within* a notebook using the jupyter-server-proxy and the new `base_url` parameter. While the assets load ok, and the breadcrumb navigation works, the facet links don't seem to use the `base_url`. Or have I missed something?

_Originally posted by @wragge in https://github.com/simonw/datasette/issues/394#issuecomment-604166918_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/712/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
725743755,MDU6SXNzdWU3MjU3NDM3NTU=,1035,"datasette.urls.table(..., format=""json"") argument",9599,simonw,closed,0,,,6026070,0.51,3,2020-10-20T16:09:34Z,2020-10-31T18:16:43Z,2020-10-31T18:16:43Z,OWNER,,"> That `datasette.urls.table(""db"", ""table"") + "".json""` example is bad because if the table name contains a `.` it should be `?_format=json` instead.
> 
> Maybe `.table()` should have a `format=""json""` option that knows how to do this.

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1026#issuecomment-712962517_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1035/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1907655261,I_kwDOBm6k_c5xtIJd,2193,"""Test DATASETTE_LOAD_PLUGINS"" test shows errors but did not fail the CI run",9599,simonw,closed,0,,,,,6,2023-09-21T19:49:34Z,2023-09-21T21:56:43Z,2023-09-21T21:56:43Z,OWNER,,"> That passed on 3.8 but should have failed: https://github.com/simonw/datasette/actions/runs/6266341481/job/17017099801 - the ""Test DATASETTE_LOAD_PLUGINS"" test shows errors but did not fail the CI run.

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/2057#issuecomment-1730201226_
            ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2193/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
913809802,MDU6SXNzdWU5MTM4MDk4MDI=,1366,Get rid of this `restore_working_directory` hack entirely,9599,simonw,open,0,,,,,2,2021-06-07T18:01:21Z,2021-06-07T18:03:03Z,,OWNER,,"> That seems to have fixed it. I'd love to get rid of this `restore_working_directory` hack entirely.

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1361#issuecomment-855308811_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1366/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1098544628,I_kwDOCGYnMM5BenX0,379,CLI options for running ANALYZE,9599,simonw,closed,0,,,7558727,3.21,0,2022-01-11T01:09:16Z,2022-01-11T01:38:01Z,2022-01-11T01:36:48Z,OWNER,,"> The Python methods are all done now, next step is the CLI options. I'll do those in a separate issue.

_Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/366#issuecomment-1009508865_

- [x] `sqlite-utils analyze` command
- [x] `sqlite-utils create-index --analyze` option (see #365)
- [x] `sqlite-utils insert --analyze` option
- [x] `sqlite-utils upsert --analyze` option

In #378 I also added `.delete_where(..., analyze=True)` but there isn't currently a `sqlite-utils delete-where` CLI command - deletions via CLI are expected to be handled using SQL queries.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/379/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
629524205,MDU6SXNzdWU2Mjk1MjQyMDU=,793,CSRF protection for /-/messages tool and writable canned queries,9599,simonw,closed,0,,,5512395,Datasette 0.44,3,2020-06-02T21:22:21Z,2020-06-06T00:43:41Z,2020-06-05T19:05:59Z,OWNER,,"> The `/-/messages` debug tool will need CSRF protection or people will be able to add messages using a hidden form on another website.
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/790#issuecomment-637790860_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/793/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1214859703,I_kwDOBm6k_c5IaUm3,1719,Refactor `RowView` and remove `RowTableShared`,9599,simonw,closed,0,,,,,3,2022-04-25T18:06:24Z,2022-12-01T21:15:19Z,2022-04-25T18:33:44Z,OWNER,,"> The `RowTableShared` class is making this a whole lot more complicated.
> 
> I'm going to split the `RowView` view out into an entirely separate `views/row.py` module.

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1715#issuecomment-1108875068_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1719/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
770598024,MDU6SXNzdWU3NzA1OTgwMjQ=,1152,Efficiently calculate list of databases/tables a user can view,9599,simonw,open,0,,,,,12,2020-12-18T06:13:01Z,2021-12-27T23:04:31Z,,OWNER,,"> The homepage currently performs a massive flurry of permission checks - one for each, database, table and view: https://github.com/simonw/datasette/blob/0.53/datasette/views/index.py#L21-L75
> 
> A paginated version of this is a little daunting as the permission checks would have to be carried out in every single table just to calculate the count that will be paginated.

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1150#issuecomment-747864831_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1152/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
811505638,MDU6SXNzdWU4MTE1MDU2Mzg=,1234,Runtime support for ATTACHing multiple databases,9599,simonw,open,0,,,,,1,2021-02-18T22:06:47Z,2021-02-22T21:06:28Z,,OWNER,,"> The implementation in #1232 is ready to land. It's the simplest-thing-that-could-possibly-work: you can run `datasette one.db two.db three.db --crossdb` and then use the `/_memory` page to run joins across tables from multiple databases.
>
> It only works on the first 10 databases that were passed to the command-line. This means that if you have a Datasette instance with hundreds of attached databases (see [Datasette Library](https://github.com/simonw/datasette/issues/417)) this won't be particularly useful for you.
>
> So... a better, future version of this feature would be one that lets you join across databases on command - maybe by hitting `/_memory?attach=db1&attach=db2` to get a special connection.

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/283#issuecomment-781665560_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1234/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1059219106,I_kwDOBm6k_c4_Imai,1524,"Improve Apache proxy documentation, link to demo",9599,simonw,closed,0,,,,,4,2021-11-20T20:03:14Z,2021-11-20T23:34:03Z,2021-11-20T23:34:03Z,OWNER,,"> The latest demo is now live at https://datasette-apache-proxy-demo.fly.dev/prefix/fixtures/sortable?_facet=pk2

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1519#issuecomment-974697824_

I'm going to put out 0.59.3 bugfix release with this, but I'd like to first improve the documentation on https://docs.datasette.io/en/stable/deploying.html#apache-proxy-configuration to highlight the new demo.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1524/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
695377804,MDU6SXNzdWU2OTUzNzc4MDQ=,153,table.optimize() should delete junk rows from *_fts_docsize,9599,simonw,closed,0,,,,,3,2020-09-07T20:31:09Z,2020-09-24T20:35:46Z,2020-09-07T21:16:33Z,OWNER,,"> The second challenge here is cleaning up all of those junk rows in existing `*_fts_docsize` tables. Doing that just to the demo database from https://github-to-sqlite.dogsheep.net/github.db dropped its size from 22MB to 16MB! Here's the SQL:
> ```sql
> DELETE FROM [licenses_fts_docsize] WHERE id NOT IN (
>   SELECT rowid FROM [licenses_fts]);
> ```
> I can do that as part of the existing `table.optimize()` method, which optimizes FTS tables.
_Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/149#issuecomment-688501064_",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/153/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1426080014,I_kwDOBm6k_c5VAEEO,1867,/db/table/-/rename API (also allows atomic replace),9599,simonw,open,0,,,8755003,Datasette 1.0a-next,1,2022-10-27T18:13:23Z,2023-01-09T15:34:12Z,,OWNER,,"> There's one catch with batched inserts: if your CLI tool fails half way through you could end up with a partially populated table - since a bunch of batches will have succeeded first.
>
> ...
>
> If people care about that kind of thing they could always push all of their inserts to a table called `_tablename` and then atomically rename that once they've uploaded all of the data (assuming I provide an atomic-rename-this-table mechanism).

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1866#issuecomment-1293893789_
      ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1867/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1456013930,I_kwDOBm6k_c5WyQJq,1906,Extract publish Heroku support to a plugin,9599,simonw,open,0,,,3268330,Datasette 1.0,0,2022-11-19T00:02:51Z,2022-11-19T00:03:10Z,,OWNER,,"> This is a strong argument for extracting the Heroku support out to a plugin - it would allow this to be fixed with a plugin release without needing to push a full release of Datasette itself.

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1905#issuecomment-1320678715_
      ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1906/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
582713554,MDU6SXNzdWU1ODI3MTM1NTQ=,700,Request object utility for handling POST form data,9599,simonw,closed,0,,,,,1,2020-03-17T02:44:59Z,2020-03-17T02:47:50Z,2020-03-17T02:47:50Z,OWNER,,"> This is also going to need me to handle POST form submissions which means I need to be able to parse the form body. I guess that will go in [datasette/utils/asgi.py](https://github.com/simonw/datasette/blob/master/datasette/utils/asgi.py).

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/698#issuecomment-599704264_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/700/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1895266807,I_kwDOBm6k_c5w93n3,2184,Design decision - should configuration be exposed at /-/config ?,9599,simonw,open,0,,,,,0,2023-09-13T21:07:08Z,2023-09-13T21:07:38Z,,OWNER,,"> This made me think. That `{""$env"": ""ENV_VAR""}` hack was introduced back here:
>
> - https://github.com/simonw/datasette/issues/538
>
> The problem it was solving was that metadata was visible to everyone with access to the instance at `/-/metadata` but plugins clearly needed a way to set secret settings.
>
> Now that this stuff is moving to config, we have some decisions to make:
>
> 1. Add `/-/config` to let people see the configuration of their instance, and keep the `$env` trick for secret settings.
> 2. Say all configuration aside from metadata is secret and make `$env` optional or ditch it entirely.
> 3. Allow plugins to announce which of their configuration options are secret so we can automatically redact them from `/-/config`
>
> I've found `/-/metadata` extraordinarily useful as a user of Datasette - it really helps me understand exactly what's going on if I run into any problems with a plugin, if I can quickly check what the settings look like.
>
> So I'm leaning towards option 1 or 3.

_Originally posted by @simonw in https://github.com/simonw/datasette/pull/2183#discussion_r1325076924_

Also refs:
- #2093",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2184/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1053087862,I_kwDOCGYnMM4-xNh2,338,"dict, list, tuple should all map to TEXT",9599,simonw,closed,0,,,,,0,2021-11-15T00:28:01Z,2021-11-15T00:36:03Z,2021-11-15T00:36:03Z,OWNER,,"> This relates to the fact that dictionaries, lists and tuples get special treatment and are converted to JSON strings, using this code: https://github.com/simonw/sqlite-utils/blob/e8d958109ee290cfa1b44ef7a39629bb50ab673e/sqlite_utils/db.py#L2937-L2947
>
> So the `COLUMN_TYPE_MAPPING` should include those  too - right now it looks like this: https://github.com/simonw/sqlite-utils/blob/e8d958109ee290cfa1b44ef7a39629bb50ab673e/sqlite_utils/db.py#L165-L188

_Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/322#issuecomment-968401459_",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/338/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
777707544,MDU6SXNzdWU3Nzc3MDc1NDQ=,219,reset_counts() method and command,9599,simonw,closed,0,,,,,4,2021-01-03T20:08:28Z,2021-01-03T20:59:37Z,2021-01-03T20:59:37Z,OWNER,,"> Thought: maybe there should be a `.reset_counts()` method too, for if the table gets out of date with the triggers.
>
> One way that could happen is if a table is dropped and recreated - the counts in the `_counts` table would likely no longer match the number of rows in that table.

_Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/215#issuecomment-753545757_",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/219/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
957302085,MDU6SXNzdWU5NTczMDIwODU=,1408,"Review places in codebase that use os.chdir(), in particularly relating to tests",9599,simonw,open,0,,,,,2,2021-07-31T18:57:06Z,2021-07-31T19:00:32Z,,OWNER,,"> To clarify: the core problem here is that an error is thrown any time you call `os.getcwd()` but the directory you are currently in has been deleted.
>
> `runner.isolated_filesystem()` assumes that the current directory in has not been deleted. But the various temporary directory utilities in `pytest` work by creating directories and then deleting them.
>
> Maybe there's a larger problem here that I play a bit fast and loose with `os.chdir()` in both the test suite and in various lines of code in Datasette itself (in particular in the publish commands)?

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1406#issuecomment-890390198_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1408/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1175690070,I_kwDOBm6k_c5GE5tW,1676,"Reconsider ensure_permissions() logic, can it be less confusing?",9599,simonw,open,0,,,3268330,Datasette 1.0,3,2022-03-21T17:14:57Z,2022-12-02T01:23:40Z,,OWNER,,"> Updated documentation: https://github.com/simonw/datasette/blob/e627510b760198ccedba9e5af47a771e847785c9/docs/internals.rst#await-ensure_permissionsactor-permissions
>
>> This method allows multiple permissions to be checked at onced. It raises a `datasette.Forbidden` exception if any of the checks are denied before one of them is explicitly granted.
>> 
>> This is useful when you need to check multiple permissions at once. For example, an actor should be able to view a table if either one of the following checks returns `True` or not a single one of them returns `False`:
>
> That's pretty hard to understand! I'm going to open a separate issue to reconsider if this is a useful enough abstraction given how confusing it is.

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1675#issuecomment-1074177827_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1676/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
400340905,MDU6SXNzdWU0MDAzNDA5MDU=,402,Use SQLITE_DBCONFIG_DEFENSIVE plus other recommendations from SQLite security docs,9599,simonw,open,0,,,,,3,2019-01-17T15:52:28Z,2019-01-17T16:15:21Z,,OWNER,,"> Was just having a skim through the datasette source. Given that the vuln impacts shadow tables, wasn't sure whether these are also covered by the immutable flag. Latest release introduced a SQLITE_DBCONFIG_DEFENSIVE flag that they recommend setting: https://sqlite.org/security.html

https://twitter.com/ignoredambience/status/1085926961413869568",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/402/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1818838294,I_kwDOCGYnMM5saUUW,578,Plugin hook for adding new output formats,9599,simonw,open,0,,,,,5,2023-07-24T17:29:18Z,2023-08-07T15:41:49Z,,OWNER,,"> What would it take to add a format hook? I'm still thinking about my GIS workflow, and being able to do `sqlite-utils query ... --geojson` would be nice. It's the one place my Datasette workflow is messy, having to do `datasette . --get /path/to/query.geojson --setting max_rows_returned 10000 --load-extension spatialite`.
> I know the current pattern is `--csv`, but maybe `--format geojson` is more future-proof.

https://discord.com/channels/823971286308356157/997738192360964156/1133076679011602432",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/578/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
969855774,MDU6SXNzdWU5Njk4NTU3NzQ=,1432,Rename Datasette.__init__(config=) parameter to settings=,9599,simonw,open,0,,,,,8,2021-08-13T01:00:27Z,2021-10-19T01:16:41Z,,OWNER,,"> While I'm doing this I should rename this internal variable to avoid confusion in the future:
>
> https://github.com/simonw/datasette/blob/e837095ef35ae155b4c78cc9a8b7133a48c94f03/datasette/app.py#L203

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1431#issuecomment-898072940_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1432/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
780153562,MDU6SXNzdWU3ODAxNTM1NjI=,1177,Ability to stream all rows as newline-delimited JSON,9599,simonw,open,0,,,3268330,Datasette 1.0,1,2021-01-06T07:10:48Z,2022-03-21T15:08:52Z,,OWNER,,"> Yet another use-case for this: I want to be able to stream newline-delimited JSON in order to better import into Pandas:
> 
>     pandas.read_json(""https://latest.datasette.io/fixtures/compound_three_primary_keys.json?_shape=array&_nl=on"", lines=True)
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1101#issuecomment-755128038_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1177/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1081318247,I_kwDOBm6k_c5Ac5tn,1556,"Show count of facet values always, not just for `?_facet_size=max`",9599,simonw,closed,0,,,7571612,Datasette 0.60,1,2021-12-15T17:49:01Z,2022-01-13T22:26:07Z,2021-12-15T17:58:06Z,OWNER,,"> You've caused me to rethink this feature - I no longer think there's value in only showing these numbers if `?_facet_size=max` as opposed to all of the time.

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1423#issuecomment-995023410_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1556/reactions"", ""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 1, ""rocket"": 0, ""eyes"": 0}",,completed
1487764628,I_kwDOCGYnMM5YrXyU,518,flake8 ValueError: Error code '#' supplied to 'extend-ignore' option...,9599,simonw,closed,0,,,,,0,2022-12-10T01:30:24Z,2022-12-10T01:36:46Z,2022-12-10T01:36:46Z,OWNER,,"> `Error code '#' supplied to 'extend-ignore' option does not match '^[A-Z]{1,3}[0-9]{0,3}$'`

https://github.com/simonw/sqlite-utils/actions/runs/3662011265/jobs/6190770361

I think from this:

https://github.com/simonw/sqlite-utils/blob/e660635cea6c32f4022818380b1e1ee88e7c93a6/setup.cfg#L1-L3
",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/518/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1940346034,I_kwDOBm6k_c5zp1Sy,2199,Detailed upgrade instructions for metadata.yaml -> datasette.yaml,9599,simonw,open,0,,,3268330,Datasette 1.0,7,2023-10-12T16:21:25Z,2023-10-12T22:08:42Z,,OWNER,,"> `Exception: Datasette no longer accepts plugin configuration in --metadata. Move your ""plugins"" configuration blocks to a separate file - we suggest calling that datasette..json - and start Datasette with datasette -c datasette..json. See https://docs.datasette.io/en/latest/configuration.html for more details.`
>
> I think we should link directly to documentation that tells people how to perform this upgrade.

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/2190#issuecomment-1759947021_
            ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2199/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
906345899,MDU6SXNzdWU5MDYzNDU4OTk=,261,`table.xindexes` using `PRAGMA index_xinfo(table)`,9599,simonw,closed,0,,,,,5,2021-05-29T04:23:48Z,2021-06-03T03:54:14Z,2021-06-03T03:51:32Z,OWNER,,"> `PRAGMA index_xinfo(table)` DOES return that data:
> ```
> (Pdb) [c[0] for c in fresh_db.execute(""PRAGMA > index_xinfo('idx_dogs_age_name')"").description]
> ['seqno', 'cid', 'name', 'desc', 'coll', 'key']
> (Pdb) fresh_db.execute(""PRAGMA index_xinfo('idx_dogs_age_name')"").fetchall()
> [(0, 2, 'age', 1, 'BINARY', 1), (1, 0, 'name', 0, 'BINARY', 1), (2, -1, None, 0, 'BINARY', 0)]
> ```
> See https://sqlite.org/pragma.html#pragma_index_xinfo
> 
> Example output: https://covid-19.datasettes.com/covid?sql=select+*+from+pragma_index_xinfo%28%27idx_ny_times_us_counties_date%27%29
_Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/260#issuecomment-850766552_",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/261/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
677037043,MDU6SXNzdWU2NzcwMzcwNDM=,923,Add homebrew installation to documentation,9599,simonw,closed,0,,,,,5,2020-08-11T16:54:31Z,2020-08-11T22:53:07Z,2020-08-11T22:52:46Z,OWNER,,"> ```
> $ brew tap simonw/datasette
> $ brew install simonw/datasette/datasette
> $ datasette --version
> datasette, version 0.46
> ```
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/335#issuecomment-672088880_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/923/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1126604194,I_kwDOBm6k_c5DJp2i,1632,"datasette one.db one.db opens database twice, as one and one_2",9599,simonw,closed,0,,,3268330,Datasette 1.0,6,2022-02-07T23:14:47Z,2022-03-19T04:04:49Z,2022-02-07T23:50:01Z,OWNER,,"> ```
> % mkdir /tmp/data
> % cp ~/Dropbox/Development/datasette/fixtures.db /tmp/data 
> % datasette /tmp/data/*.db /tmp/data/created.db --create -p 8852
> ...
> INFO:     Uvicorn running on http://127.0.0.1:8852 (Press CTRL+C to quit)
> ^CINFO:     Shutting down
> % datasette /tmp/data/*.db /tmp/data/created.db --create -p 8852
> ...
> INFO:     127.0.0.1:49533 - ""GET / HTTP/1.1"" 200 OK
> ```
> The first time I ran Datasette I got two databases - `fixtures` and `created`
> 
> BUT... when I ran Datasette the second time it looked like this:
> 
> <img width=""697"" alt=""image"" src=""https://user-images.githubusercontent.com/9599/152887735-5e2b4f21-001d-4e7a-a23a-16a32fdcf554.png"">
> 
> This is the same result you get if you run:
> 
>     datasette /tmp/data/fixtures.db /tmp/data/created.db /tmp/data/created.db
> 
> This is caused by this Datasette issue:
> - https://github.com/simonw/datasette/issues/509
> 
> So... either I teach Datasette to de-duplicate multiple identical file paths passed to the command, or I can't use `/data/*.db` in the `Dockerfile` here and I need to go back to other solutions for the challenge described in this comment: https://github.com/simonw/datasette-publish-fly/pull/12#issuecomment-1031971831

_Originally posted by @simonw in https://github.com/simonw/datasette-publish-fly/pull/12#issuecomment-1032029874_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1632/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1269998342,I_kwDOCGYnMM5LsqMG,443,Make `utils.rows_from_file()` a documented API,9599,simonw,closed,0,,,,,2,2022-06-13T21:53:24Z,2022-06-20T19:49:37Z,2022-06-14T20:12:46Z,OWNER,,"> `rows_from_file()` isn't part of the documented API but maybe it should be!

_Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/440#issuecomment-1154385916_",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/443/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
940077168,MDU6SXNzdWU5NDAwNzcxNjg=,1389,"""searchmode"": ""raw"" in table metadata",9599,simonw,closed,0,,,,,6,2021-07-08T17:32:10Z,2021-07-10T18:33:13Z,2021-07-10T18:33:13Z,OWNER,,"> http://localhost:8001/index/summary?_search=language%3Aeng&_sort=title&_searchmode=raw
>
> But I'm not able to manage it in the metadata file. Here is mine (note that the sort column is taken into account)
> Here it is:
>
> ```
> {
>     ""databases"": {
>       ""index"": {
>         ""tables"": {
>             ""summary"": {
>                 ""sort"": ""title"",
>                 ""searchmode"": ""raw""
>             }
>         }
>       }
>     }
>   }

_Originally posted by @Krazybug in https://github.com/simonw/datasette/issues/759#issuecomment-624860451_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1389/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1840324765,I_kwDOBm6k_c5tsSCd,2129,CSV ?sql= should indicate errors,9599,simonw,open,0,,,3268330,Datasette 1.0,1,2023-08-07T23:13:04Z,2023-08-08T02:02:21Z,,OWNER,,"> https://latest.datasette.io/_memory.csv?sql=select+blah is a blank page right now:

```bash
curl -I 'https://latest.datasette.io/_memory.csv?sql=select+blah'
```
```
HTTP/2 200 
access-control-allow-origin: *
access-control-allow-headers: Authorization, Content-Type
access-control-expose-headers: Link
access-control-allow-methods: GET, POST, HEAD, OPTIONS
access-control-max-age: 3600
content-type: text/plain; charset=utf-8
x-databases: _memory, _internal, fixtures, fixtures2, extra_database, ephemeral
date: Mon, 07 Aug 2023 23:12:15 GMT
server: Google Frontend
```

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/2118#issuecomment-1668688947_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2129/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1816876211,I_kwDOCGYnMM5sS1Sz,571,`.transform(keep_table=...)` option,9599,simonw,closed,0,,,,,1,2023-07-22T19:49:29Z,2023-07-22T22:32:18Z,2023-07-22T22:32:18Z,OWNER,,">> Also need a design for an option for the `.transform()` method to indicate that the new table should be created with a new name without dropping the old one.
>
> I think `keep_table=""name_of_table""` is good for this.

_Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/565#issuecomment-1646657324_
            ",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/571/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
930807135,MDU6SXNzdWU5MzA4MDcxMzU=,1384,Plugin hook for dynamic metadata,9599,simonw,open,0,,,,,22,2021-06-26T22:36:03Z,2022-03-14T00:36:42Z,,OWNER,,"@brandonrobertz contributed an implementation of this in PR #1368, which I just merged. Opening this ticket to track further work on this before it goes out in a Datasette release (likely preceded by an alpha).",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1384/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
573578548,MDU6SXNzdWU1NzM1Nzg1NDg=,89,Ability to customize columns used by extracts= feature,9599,simonw,open,0,,,,,3,2020-03-01T16:54:48Z,2020-10-16T19:17:50Z,,OWNER,,"@simonw any thoughts on allow extracts to specify the lookup column name? If I'm understanding the documentation right, `.lookup()` allows you to define the ""value"" column (the documentation uses name), but when you use `extracts` keyword as part of `.insert()`, `.upsert()` etc. the lookup must be done against a column named ""value"". I have an existing lookup table that I've populated with columns ""id"" and ""name"" as opposed to ""id"" and ""value"", and seems I can't use `extracts=`, unless I'm missing something...

Initial thought on how to do this would be to allow the dictionary value to be a tuple of table name column pair... so:
```
table = db.table(""trees"", extracts={""species_id"": (""Species"", ""name""})
```

I haven't dug too much into the existing code yet, but does this make sense? Worth doing?

_Originally posted by @chrishas35 in https://github.com/simonw/sqlite-utils/issues/46#issuecomment-592999503_",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/89/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
274160723,MDU6SXNzdWUyNzQxNjA3MjM=,100,TemplateAssertionError: no filter named 'tojson',13304454,coisnepe,closed,0,,,,,2,2017-11-15T13:43:41Z,2017-11-16T09:25:10Z,2017-11-16T00:14:13Z,NONE,,"A 500 error is raised upon clicking on the name of a table on the homepage, say _http://0.0.0.0:8001/_ to _http://0.0.0.0:8001/test_check-c1f4771/users_ The API part seems to function as intended, though...

```
2017-11-15 14:33:57 - (sanic)[ERROR]: Traceback (most recent call last):
  File ""/usr/local/lib/python3.5/dist-packages/sanic/app.py"", line 503, in handle_request
    response = await response
  File ""/usr/local/lib/python3.5/dist-packages/datasette/app.py"", line 155, in get
    return await self.view_get(request, name, hash, **kwargs)
  File ""/usr/local/lib/python3.5/dist-packages/datasette/app.py"", line 219, in view_get
    **context,
  File ""/usr/local/lib/python3.5/dist-packages/sanic_jinja2/__init__.py"", line 84, in render
    return html(self.render_string(template, request, **context))
  File ""/usr/local/lib/python3.5/dist-packages/sanic_jinja2/__init__.py"", line 81, in render_string
    return self.env.get_template(template).render(**context)
  File ""/usr/lib/python3/dist-packages/jinja2/environment.py"", line 812, in get_template
    return self._load_template(name, self.make_globals(globals))
  File ""/usr/lib/python3/dist-packages/jinja2/environment.py"", line 786, in _load_template
    template = self.loader.load(self, name, globals)
  File ""/usr/lib/python3/dist-packages/jinja2/loaders.py"", line 125, in load
    code = environment.compile(source, name, filename)
  File ""/usr/lib/python3/dist-packages/jinja2/environment.py"", line 565, in compile
    self.handle_exception(exc_info, source_hint=source_hint)
  File ""/usr/lib/python3/dist-packages/jinja2/environment.py"", line 754, in handle_exception
    reraise(exc_type, exc_value, tb)
  File ""/usr/lib/python3/dist-packages/jinja2/_compat.py"", line 37, in reraise
    raise value.with_traceback(tb)
  File ""/usr/local/lib/python3.5/dist-packages/datasette/templates/table.html"", line 29, in template
    <pre>params = {{ query.params|tojson(4) }}</pre>
  File ""/usr/lib/python3/dist-packages/jinja2/environment.py"", line 515, in _generate
    return generate(source, self, name, filename, defer_init=defer_init)
  File ""/usr/lib/python3/dist-packages/jinja2/compiler.py"", line 62, in generate
    generator.visit(node)
  File ""/usr/lib/python3/dist-packages/jinja2/visitor.py"", line 38, in visit
    return f(node, *args, **kwargs)
  File ""/usr/lib/python3/dist-packages/jinja2/compiler.py"", line 849, in visit_Template
    self.blockvisit(block.body, block_frame)
  File ""/usr/lib/python3/dist-packages/jinja2/compiler.py"", line 492, in blockvisit
    self.visit(node, frame)
  File ""/usr/lib/python3/dist-packages/jinja2/visitor.py"", line 38, in visit
    return f(node, *args, **kwargs)
  File ""/usr/lib/python3/dist-packages/jinja2/compiler.py"", line 1172, in visit_If
    self.blockvisit(node.body, if_frame)
  File ""/usr/lib/python3/dist-packages/jinja2/compiler.py"", line 492, in blockvisit
    self.visit(node, frame)
  File ""/usr/lib/python3/dist-packages/jinja2/visitor.py"", line 38, in visit
    return f(node, *args, **kwargs)
  File ""/usr/lib/python3/dist-packages/jinja2/compiler.py"", line 1353, in visit_Output
    self.visit(argument, frame)
  File ""/usr/lib/python3/dist-packages/jinja2/visitor.py"", line 38, in visit
    return f(node, *args, **kwargs)
  File ""/usr/lib/python3/dist-packages/jinja2/compiler.py"", line 1565, in visit_Filter
    self.fail('no filter named %r' % node.name, node.lineno)
  File ""/usr/lib/python3/dist-packages/jinja2/compiler.py"", line 427, in fail
    raise TemplateAssertionError(msg, lineno, self.name, self.filename)
jinja2.exceptions.TemplateAssertionError: no filter named 'tojson'

2017-11-15 14:33:57 - (network)[INFO][127.0.0.1:41316]: GET http://0.0.0.0:8001/test_check-c1f4771/users  500 144
2017-11-15 14:33:57 - (network)[INFO][127.0.0.1:41316]: GET http://0.0.0.0:8001/favicon.ico  200 0
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/100/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
325294102,MDU6SXNzdWUzMjUyOTQxMDI=,278,Build smallest possible Docker image with Datasette plus recent SQLite (with json1) plus Spatialite 4.4.0,9599,simonw,closed,0,,,,,3,2018-05-22T13:28:40Z,2018-05-23T17:43:36Z,2018-05-23T17:43:36Z,OWNER,,"A Dockerfile that does the following:

* Bundles Datasette master
* Python 3.6 most recent version (or 3.7 if it has been released)
* SQLite 3.23.1 (or most recent release) such that ""import sqite3"" in Python gets that version. Ideally with the json1 module baked in by default, but having it loadable as an optional module is fine too
* SpatiaLite 4.4.0-RC0 (or most recent version) such that it can be loaded as an optional module
* Uses multi-stage builds to stay as small as possible

Note that the current ""release"" of SpatiaLite is 4.3.0 which is missing key features like https://www.gaia-gis.it/fossil/libspatialite/wiki?name=KNN - 4.4.0 probably needs to be compiled from source.

I don't know the best way to get a current SQLite version bundled for Python 3. Maybe https://github.com/coleifer/pysqlite3 ?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/278/reactions"", ""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
273703829,MDU6SXNzdWUyNzM3MDM4Mjk=,86,Filter UI on table page,9599,simonw,closed,0,,,2919870,Foreign key edition,10,2017-11-14T08:22:43Z,2017-11-23T20:34:32Z,2017-11-23T20:34:32Z,OWNER,,A UI for building up simple table queries by adding additional filter rules that get executed as query parameters in the URL.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/86/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1247315144,I_kwDOBm6k_c5KWITI,1749,LDAP auth plugin,380241,benswift,open,0,,,,,0,2022-05-25T01:35:12Z,2022-05-25T01:35:12Z,,NONE,,"A [search of the plugins directory](https://datasette.io/plugins?q=ldap) doesn't turn up anything, but is is possible to set up a Datasette app which uses my organisation's LDAP for auth?

If not, how much work would it be to write one (I _may_ have some spare cycles on my team to do this, but we haven't written a datasette plugin before).",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1749/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
668308777,MDU6SXNzdWU2NjgzMDg3Nzc=,129,"""insert-files --sqlar"" for creating SQLite archives",9599,simonw,closed,0,,,,,2,2020-07-30T02:28:29Z,2020-07-30T22:41:01Z,2020-07-30T22:40:55Z,OWNER,,"A `--sqlar` option could cause `insert-files` to behave in the same way as SQLite's own sqlar mechanism.

https://www.sqlite.org/sqlar.html and https://sqlite.org/sqlar/doc/trunk/README.md",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/129/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
691557547,MDU6SXNzdWU2OTE1NTc1NDc=,10,Category 3: received,9599,simonw,closed,0,,,,,1,2020-09-03T01:40:36Z,2020-09-03T17:38:51Z,2020-09-03T17:38:51Z,MEMBER,,"A category for things that were sent to me: DMs, emails etc. Follows #7.",197431109,dogsheep-beta,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-beta/issues/10/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
753026003,MDU6SXNzdWU3NTMwMjYwMDM=,54,github-to-sqlite workflows command,9599,simonw,closed,0,,,,,3,2020-11-29T21:56:42Z,2020-11-29T22:08:46Z,2020-11-29T21:57:17Z,MEMBER,,"A command that fetches the YAML workflows for different repos, parses them and stores them in relational tables would be really useful for maintaining larger numbers of workflows.",207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/54/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
735532751,MDU6SXNzdWU3MzU1MzI3NTE=,192,sqlite-utils search command,9599,simonw,closed,0,,,6079500,3.0,9,2020-11-03T18:07:59Z,2020-11-08T17:07:01Z,2020-11-08T17:07:01Z,OWNER,,A command that knows how to run a search against a FTS enabled table and return results ranked by relevance.,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/192/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
777560474,MDU6SXNzdWU3Nzc1NjA0NzQ=,218,"""sqlite-utils triggers"" command",9599,simonw,closed,0,,,,,1,2021-01-03T02:34:50Z,2021-01-03T03:49:51Z,2021-01-03T03:03:35Z,OWNER,,"A command to list the triggers in the database.

    sqlite-utils triggers my.db

Can optionally take one or more tables:

    sqlite-utils triggers my.db table1 table2",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/218/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
763283616,MDU6SXNzdWU3NjMyODM2MTY=,207,sqlite-utils analyze-tables command,9599,simonw,closed,0,,,,,4,2020-12-12T04:33:12Z,2020-12-13T07:25:23Z,2020-12-13T07:20:13Z,OWNER,,"A command which analyzes a table (potentially taking quite a while if the table is large) and outputs information for each column - things like:

- How many unique values does this column have?
- How many null rows?
- How many blank rows? (defined as empty string)
- What are the 10 most common values?
- What are the 10 least common values?

The command can output this information to the terminal, but it should also provide an option for writing the information to a database table so it can be explored later.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/207/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
613006393,MDU6SXNzdWU2MTMwMDYzOTM=,20,Ability to serve thumbnailed Apple Photo from its place on disk,9599,simonw,closed,0,,,,,10,2020-05-06T02:17:50Z,2020-05-25T20:14:22Z,2020-05-25T20:09:41Z,MEMBER,,"A custom Datasette plugin that can be run locally on a Mac laptop which knows how to serve photos such that they can be seen in the browser.

_Originally posted by @simonw in https://github.com/dogsheep/photos-to-sqlite/issues/19#issuecomment-624406285_",256834907,dogsheep-photos,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-photos/issues/20/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1515185383,I_kwDOBm6k_c5aT-Tn,1971,Upgrade for Sphinx 6.0 (once Furo has support for it),9599,simonw,closed,0,,,,,3,2022-12-31T19:04:35Z,2023-01-10T02:02:34Z,2023-01-10T02:02:34Z,OWNER,,"A deployment of #1967 to ReadTheDocs just failed like this: https://readthedocs.org/projects/datasette/builds/19045460/

```
Running Sphinx v6.0.0
making output directory... done
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 28 source files that are out of date
updating environment: [new config] 28 added, 0 changed, 0 removed
reading sources... [  3%] authentication
reading sources... [  7%] binary_data
reading sources... [ 10%] changelog

Traceback (most recent call last):
  File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/statemachine.py"", line 299, in next_line
    self.line = self.input_lines[self.line_offset]
  File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/statemachine.py"", line 1136, in __getitem__
    return self.data[i]
IndexError: list index out of range

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/statemachine.py"", line 226, in run
    self.next_line()
  File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/statemachine.py"", line 302, in next_line
    raise EOFError
EOFError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/sphinx/cmd/build.py"", line 281, in build_main
    app.build(args.force_all, args.filenames)
  File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/sphinx/application.py"", line 344, in build
    self.builder.build_update()
  File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/sphinx/builders/__init__.py"", line 310, in build_update
    self.build(to_build,
  File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/sphinx/builders/__init__.py"", line 326, in build
    updated_docnames = set(self.read())
  File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/sphinx/builders/__init__.py"", line 433, in read
    self._read_serial(docnames)
  File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/sphinx/builders/__init__.py"", line 454, in _read_serial
    self.read_doc(docname)
  File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/sphinx/builders/__init__.py"", line 510, in read_doc
    publisher.publish()
  File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/core.py"", line 224, in publish
    self.document = self.reader.read(self.source, self.parser,
  File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/sphinx/io.py"", line 103, in read
    self.parse()
  File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/readers/__init__.py"", line 76, in parse
    self.parser.parse(self.input, document)
  File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/sphinx/parsers.py"", line 78, in parse
    self.statemachine.run(inputlines, document, inliner=self.inliner)
  File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py"", line 169, in run
    results = StateMachineWS.run(self, input_lines, input_offset,
  File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/statemachine.py"", line 233, in run
    context, next_state, result = self.check_line(
  File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/statemachine.py"", line 445, in check_line
    return method(match, context, next_state)
  File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py"", line 3024, in text
    self.section(title.lstrip(), source, style, lineno + 1, messages)
  File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py"", line 325, in section
    self.new_subsection(title, lineno, messages)
  File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py"", line 391, in new_subsection
    newabsoffset = self.nested_parse(
  File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py"", line 279, in nested_parse
    state_machine.run(block, input_offset, memo=self.memo,
  File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py"", line 195, in run
    results = StateMachineWS.run(self, input_lines, input_offset)
  File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/statemachine.py"", line 233, in run
    context, next_state, result = self.check_line(
  File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/statemachine.py"", line 445, in check_line
    return method(match, context, next_state)
  File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py"", line 2785, in underline
    self.section(title, source, style, lineno - 1, messages)
  File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py"", line 325, in section
    self.new_subsection(title, lineno, messages)
  File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py"", line 391, in new_subsection
    newabsoffset = self.nested_parse(
  File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py"", line 279, in nested_parse
    state_machine.run(block, input_offset, memo=self.memo,
  File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py"", line 195, in run
    results = StateMachineWS.run(self, input_lines, input_offset)
  File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/statemachine.py"", line 233, in run
    context, next_state, result = self.check_line(
  File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/statemachine.py"", line 445, in check_line
    return method(match, context, next_state)
  File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py"", line 1273, in bullet
    i, blank_finish = self.list_item(match.end())
  File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py"", line 1295, in list_item
    self.nested_parse(indented, input_offset=line_offset,
  File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py"", line 279, in nested_parse
    state_machine.run(block, input_offset, memo=self.memo,
  File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py"", line 195, in run
    results = StateMachineWS.run(self, input_lines, input_offset)
  File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/statemachine.py"", line 239, in run
    result = state.eof(context)
  File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py"", line 2725, in eof
    self.blank(None, context, None)
  File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py"", line 2716, in blank
    paragraph, literalnext = self.paragraph(
  File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py"", line 416, in paragraph
    textnodes, messages = self.inline_text(text, lineno)
  File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py"", line 425, in inline_text
    nodes, messages = self.inliner.parse(text, lineno,
  File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py"", line 649, in parse
    before, inlines, remaining, sysmessages = method(self, match,
  File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py"", line 792, in interpreted_or_phrase_ref
    nodelist, messages = self.interpreted(rawsource, escaped, role,
  File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py"", line 889, in interpreted
    nodes, messages2 = role_fn(role, rawsource, text, lineno, self)
  File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/sphinx/ext/extlinks.py"", line 101, in role
    title = caption % part
TypeError: not all arguments converted during string formatting

Exception occurred:
  File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/sphinx/ext/extlinks.py"", line 101, in role
    title = caption % part
TypeError: not all arguments converted during string formatting
The full traceback has been saved in /tmp/sphinx-err-kq7ylgqo.log, if you want to report the issue to the developers.
Please also report this if it was a user error, so that a better error message can be provided next time.
A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks! 
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1971/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
317714268,MDU6SXNzdWUzMTc3MTQyNjg=,238,External metadata.json,9599,simonw,closed,0,,,,,3,2018-04-25T17:02:30Z,2019-06-24T06:52:55Z,2019-06-24T06:52:45Z,OWNER,,"A frustration I'm having with https://register-of-members-interests.datasettes.com/ is that I keep coming up with new canned queries but I don't want to redeploy the whole thing just to add them to `metadata.json`

Maybe Datasette could optionally take a `--metadata-url` option which causes it to load from a URL instead and occasionally check for updates.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/238/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
913135723,MDU6SXNzdWU5MTMxMzU3MjM=,266,"Add some types, enforce with mypy",9599,simonw,closed,0,,,,,3,2021-06-07T06:05:56Z,2021-08-18T22:25:38Z,2021-08-18T22:25:38Z,OWNER,,"A good starting point would be adding type information to the members of these named tuples and the introspection methods that return them:

https://github.com/simonw/sqlite-utils/blob/9dff7a38831d471b1dff16d40d89eb5c3b4e84d6/sqlite_utils/db.py#L51-L75",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/266/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
709789634,MDU6SXNzdWU3MDk3ODk2MzQ=,27,Sort order is not persisted by facet filter links,9599,simonw,open,0,,,,,0,2020-09-27T18:22:07Z,2020-09-27T18:22:07Z,,MEMBER,,A link to `/-/beta?category=1&timestamp__date=2018-08-01&q=swedish` should be to `/-/beta?category=1&timestamp__date=2018-08-01&q=swedish&sort=newest`,197431109,dogsheep-beta,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-beta/issues/27/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
396215043,MDU6SXNzdWUzOTYyMTUwNDM=,395,Find a cleaner pattern for fixtures with arguments,9599,simonw,closed,0,,,,,1,2019-01-06T00:31:22Z,2020-06-07T21:23:22Z,2020-06-07T21:23:22Z,OWNER,,"A lot of Datasette tests look like this:

https://github.com/simonw/datasette/blob/b65d97792a53f78cb14b226231063209d22c4602/tests/test_api.py#L438-L444

The loop here isn't actually expected to loop - it's there because the `make_app_client` function yields a value and then cleans it up afterwards.

This pattern works, but it is a little confusing. It would be nice to replace it with something less strange looking.

The answer may be to switch to the ""factories as fixtures"" pattern described here: https://docs.pytest.org/en/latest/fixture.html#factories-as-fixtures

In particular some variant of this example:

```
@pytest.fixture
def make_customer_record():

    created_records = []

    def _make_customer_record(name):
        record = models.Customer(name=name, orders=[])
        created_records.append(record)
        return record

    yield _make_customer_record

    for record in created_records:
        record.destroy()


def test_customer_records(make_customer_record):
    customer_1 = make_customer_record(""Lisa"")
    customer_2 = make_customer_record(""Mike"")
    customer_3 = make_customer_record(""Meredith"")
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/395/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
320592643,MDU6SXNzdWUzMjA1OTI2NDM=,251,"Explore ""distinct values for column"" in inspect()",9599,simonw,closed,0,,,,,4,2018-05-06T13:27:24Z,2018-05-14T22:47:55Z,2018-05-14T22:47:55Z,OWNER,,"A lot of datasets have columns which have a small number of possible values in them - this one for example: https://fivethirtyeight.datasettes.com/fivethirtyeight-2628db9?sql=select+distinct+category+from+%5Binconvenient-sequel%2Fratings%5D%3B

Detecting these could be interesting as part of `.inspect()`, since it would allow for various UI enhancements like autocomplete / select box filters for those columns.

The problem is detecting them efficiently. `.inspect()` shouldn't spend 5 minutes churning through columns on giant tables trying to determine if they have a small collection of unique values.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/251/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
738514367,MDU6SXNzdWU3Mzg1MTQzNjc=,202,sqlite-utils insert -f colname - for configuring full-text search,9599,simonw,closed,0,,,,,2,2020-11-08T17:30:09Z,2021-01-03T05:00:36Z,2021-01-03T05:00:27Z,OWNER,,"A mechanism for specifying columns that should be configured for full-text search as part of the initial data import:

    sqlite-utils insert mydb.db articles articles.csv --csv -f title -f body",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/202/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1822949756,I_kwDOBm6k_c5sqAF8,2116,Turn DatabaseDownload into an async view function,9599,simonw,closed,0,,,9700784,Datasette 1.0a3,3,2023-07-26T18:31:59Z,2023-07-26T18:44:00Z,2023-07-26T18:44:00Z,OWNER,,"A minor refactor, but it is a good starting point for this new branch. Refs:
- #2109",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2116/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1217014076,I_kwDOBm6k_c5Iiik8,1726,Security page in the documentation,9599,simonw,open,0,,,,,0,2022-04-27T08:43:30Z,2022-04-27T08:43:30Z,,OWNER,,"A page talking about how to run Datasette securely, and security concerns to take into account.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1726/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1485757511,I_kwDOBm6k_c5YjtxH,1939,register_permissions(datasette) plugin hook,9599,simonw,closed,0,,,8711695, Datasette 1.0a2,20,2022-12-09T01:33:25Z,2022-12-13T02:07:50Z,2022-12-13T02:05:56Z,OWNER,,"A plugin hook that adds more named permissions to the list which is initially populated here:

https://github.com/simonw/datasette/blob/e539c1c024bc62d88df91d9107cbe37e7f0fe55f/datasette/permissions.py#L1-L19

Originally imagined this hook in this comment:

- https://github.com/simonw/datasette/issues/1881#issuecomment-1301639370

I need this for a few reasons:

- https://github.com/simonw/datasette/issues/1636
  - Needs it in order to validate that permissions defined in `metadata.json` are set in the right place (don't set an instance permissions at table level for example)
- https://github.com/simonw/datasette/issues/1855
  - Needs it to be able to register additional abbreviations for use in signed cookies
  - And for validation when you use `datasette create-token` and pass in extra permissions
- The https://latest.datasette.io/-/permissions debug interface needs it to add extra debug options to the `<select>`",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1939/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
953218043,MDU6SXNzdWU5NTMyMTgwNDM=,1403,Labels explaining what hidden tables are for,9599,simonw,open,0,,,,,0,2021-07-26T19:29:22Z,2022-03-21T22:20:37Z,,OWNER,,"A reasonable question: ""What are those hidden tables for?""

This could be answered by adding a small piece of explanatory text to each table - based on if it's related to FTS or to SpatiaLite or configured to be hidden for some other reason.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1403/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1618249044,I_kwDOBm6k_c5gdIVU,2038,Consider a `strict_templates` setting,9599,simonw,open,0,,,,,2,2023-03-10T02:09:13Z,2023-03-10T02:11:06Z,,OWNER,,"A setting which turns on Jinja strict mode, so any templates that access undefined variables raise a hard error.

Prototype here:
```diff
diff --git a/datasette/app.py b/datasette/app.py
index 40416713..1428a3f0 100644
--- a/datasette/app.py
+++ b/datasette/app.py
@@ -200,6 +200,7 @@ SETTINGS = (
         ""Allow display of SQL trace debug information with ?_trace=1"",
     ),
     Setting(""base_url"", ""/"", ""Datasette URLs should use this base path""),
+    Setting(""strict_templates"", False, ""Raise errors for undefined template variables""),
 )
 _HASH_URLS_REMOVED = ""The hash_urls setting has been removed, try the datasette-hashed-urls plugin instead""
 OBSOLETE_SETTINGS = {
@@ -399,11 +400,14 @@ class Datasette:
                 ),
             ]
         )
+        env_extras = {}
+        if self.setting(""strict_templates""):
+            env_extras[""undefined""] = StrictUndefined
         self.jinja_env = Environment(
             loader=template_loader,
             autoescape=True,
             enable_async=True,
-            undefined=StrictUndefined,
+            **env_extras,
         )
         self.jinja_env.filters[""escape_css_string""] = escape_css_string
         self.jinja_env.filters[""quote_plus""] = urllib.parse.quote_plus
```
Explored this idea a bit in:
- #1999",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2038/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
716988478,MDU6SXNzdWU3MTY5ODg0Nzg=,997,Documentation covering buildpack deployment,9599,simonw,closed,0,,,5971510,Datasette 0.50,3,2020-10-08T03:21:52Z,2020-10-08T23:56:03Z,2020-10-08T23:32:10Z,OWNER,,A tidied up version of https://til.simonwillison.net/til/til/digitalocean_datasette-on-digitalocean-app-platform.md - but mention that you can deploy to Heroku using the same mechanism.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/997/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
974987856,MDU6SXNzdWU5NzQ5ODc4NTY=,1442,Mechanism to cause specific branches to deploy their own demos,9599,simonw,closed,0,,,,,6,2021-08-19T19:41:39Z,2021-08-19T21:11:45Z,2021-08-19T21:09:40Z,OWNER,,"A useful capability would be if it was super-easy to say ""any pushes to branch X should be deployed to `latest-X.datasette.io`"".

I'd like to use this for the column query information work in #1434",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1442/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
893537744,MDU6SXNzdWU4OTM1Mzc3NDQ=,1331,Add support for Jinja2 version 3.0,475613,MarkusH,closed,0,,,,,10,2021-05-17T17:14:36Z,2021-05-23T00:57:39Z,2021-05-23T00:57:39Z,NONE,,"A week ago, [The Pallets Project](https://github.com/pallets) released [new major versions of several of its projects](https://palletsprojects.com/blog/flask-2-0-released/). Among those updates is one for Jinja2, which bumps it to version 3.0.0.

I'd like for datasette to support Jinaj2 version 3.0.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1331/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1421552095,I_kwDOBm6k_c5Uuynf,1852,Default API token authentication mechanism,9599,simonw,closed,0,,,8658075,Datasette 1.0a0,30,2022-10-24T22:31:07Z,2022-11-15T19:57:00Z,2022-10-26T02:19:54Z,OWNER,,"API authentication will be via `Authorization: Bearer XXX` request headers.

I'm inclined to add a default token mechanism to Datasette based on tokens that are signed with the `DATASETTE_SECRET`. Maybe the root user can access `/-/create-token` which provides a UI for generating a time-limited signed token? Could also have a `datasette token` command for creating such tokens at the command-line.

Plugins can then define alternative ways of creating tokens, such as the existing https://datasette.io/plugins/datasette-auth-tokens plugin.

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1850#issuecomment-1289706439_
      ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1852/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1425029242,I_kwDOBm6k_c5U8Dh6,1863,Update a single record in an existing table,9599,simonw,closed,0,,,8658075,Datasette 1.0a0,16,2022-10-27T04:53:17Z,2022-11-29T18:08:53Z,2022-11-29T18:06:37Z,OWNER,,"API design:

```
POST /db/table/row-pks/-/update
{
   ""field"": ""updated_value""
}
```
Only the fields that you pass will be updated.

Maybe this is the wrong design though? The design for insert currently looks like this:

- https://github.com/simonw/datasette/issues/1851#issuecomment-1294224185

```
POST /db/table/-/insert
Authorization: Bearer xxx
Content-Type: application/json
{
  ""row"": {
    ""id"": 1,
    ""name"": ""New name""
  }
}
```
I could use the same format for `/-/update`, but in this case the API doesn't require you to pass every field so `""row""` doesn't seem like the right key.

I think I'll go with this:

```
POST /db/table/1/-/update
Authorization: Bearer xxx
Content-Type: application/json
{
  ""update"": {
    ""name"": ""New name""
  }
}
```
The benefit of having an `""update""` key is that it allows me to use other keys in the future. Maybe a `""alter"": true` key to indicate that new columns should be added if they are missing.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1863/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1425029275,I_kwDOBm6k_c5U8Dib,1864,Delete a single record from an existing table,9599,simonw,closed,0,,,8658075,Datasette 1.0a0,4,2022-10-27T04:53:22Z,2022-11-29T18:54:04Z,2022-11-29T18:54:04Z,OWNER,,"API design:
```
POST /db/table/row-pks/-/delete
Or...
DELETE /db/table/row-pks/-/delete
```
I'm just going to do `POST` for the moment, like I did here:
- #1874

Permission: `delete-row`

Still needed:

- [ ] Tests for rowid tables
- [ ] Tests for compound primary keys",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1864/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
641460179,MDU6SXNzdWU2NDE0NjAxNzk=,854,"Respect default scope[""actor""] if one exists",9599,simonw,closed,0,,,5533512,Datasette 0.45,0,2020-06-18T18:25:08Z,2020-06-18T18:39:22Z,2020-06-18T18:39:22Z,OWNER,,"ASGI wrapper plugins that themselves set the `actor` scope variable should be respected (though `actor_from_request` plugins should still execute and get the chance to replace that initial `actor` value).

Relevant code: https://github.com/simonw/datasette/blob/09a3479a5402df96489ed6cab6cc9fd674bf3433/datasette/app.py#L910-L921",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/854/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
275476839,MDU6SXNzdWUyNzU0NzY4Mzk=,138,"Per-database and per-table metadata, probably using data-package",9599,simonw,closed,0,,,,,1,2017-11-20T19:50:10Z,2017-12-10T03:08:36Z,2017-12-10T03:08:26Z,OWNER,,"Ability to annotate databases and tables with extra metadata describing their purpose, providing source and licensing information and describing individual columns.

http://frictionlessdata.io/specs/data-package/ looks like a great format for this, see #105 ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/138/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
694493566,MDU6SXNzdWU2OTQ0OTM1NjY=,16,Timeline view,9599,simonw,open,0,,,,,3,2020-09-06T19:13:58Z,2020-09-21T02:42:29Z,,MEMBER,,Ability to browse (and facet) by date.,197431109,dogsheep-beta,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-beta/issues/16/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
322477187,MDU6SXNzdWUzMjI0NzcxODc=,255,Facets,9599,simonw,closed,0,,,,,16,2018-05-12T03:00:07Z,2019-05-29T21:39:12Z,2018-05-16T15:32:12Z,OWNER,,"Ability to display facets and facet counts on the table view.

Facets can be specified in the URL with `?_facet=column&_facet=othercolumn` or the default facets for a table can be set using a new `""facets"": [...]` property in `metadata.json`

- [x] Implement `?_facet=`
- [x] Implement `metadata.json` `facets` key
- [x] Design for how facets should be presented
- [x] Facets should be able to toggle off as well as on
- [x] Expand labels for facets that are foreign keys
- [x] Suggest potential facets (if we can do so within a tight time limit)
- [x] Documentation",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/255/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
446429421,MDU6SXNzdWU0NDY0Mjk0MjE=,481,Facet by date,9599,simonw,closed,0,,,,,1,2019-05-21T05:55:54Z,2019-05-29T21:39:12Z,2019-05-21T06:09:49Z,OWNER,,Ability to facet on datetime fields by their date.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/481/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
267515836,MDU6SXNzdWUyNjc1MTU4MzY=,4,Make URLs immutable,9599,simonw,closed,0,,,2857392,Ship first public release,8,2017-10-23T01:13:30Z,2017-10-24T02:38:24Z,2017-10-24T02:38:24Z,OWNER,,"Absolutely everything should have a far-future expires header

Part of the URL will be the truncated sha1 hash of the database file itself, calculated at build time",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/4/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
688395275,MDU6SXNzdWU2ODgzOTUyNzU=,144,Run some tests against numpy,9599,simonw,closed,0,,,,,2,2020-08-28T22:53:00Z,2020-08-28T22:57:05Z,2020-08-28T22:57:04Z,OWNER,,"Accidentally removed in #143:

https://github.com/simonw/sqlite-utils/blob/d7d3f962861ef32c5ead8f514c8756f5b6f7c4a0/.travis.yml#L18-L19",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/144/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
577302229,MDU6SXNzdWU1NzczMDIyMjk=,91,Enable ordering FTS results by rank,416374,gfrmin,closed,0,,,6079500,3.0,1,2020-03-07T08:43:51Z,2020-11-06T23:53:26Z,2020-11-06T23:53:25Z,NONE,,According to https://www.sqlite.org/fts5.html (not sure about FTS4) results can be sorted by relevance. At the moment results are returned by default by `rowid`. Perhaps a flag can be added to the `search` method?,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/91/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1822918995,I_kwDOCGYnMM5sp4lT,580,Add way to export to a csv file using the Python library,44324811,kevinlinxc,open,0,,,,,0,2023-07-26T18:09:26Z,2023-07-26T18:09:26Z,,NONE,,"According to the documentation, we can make a csv output using the CLI tool, but not the Python library. Could we have the latter?",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/580/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
612089949,MDU6SXNzdWU2MTIwODk5NDk=,756,Add pipx to installation documentation,9599,simonw,closed,0,,,,,2,2020-05-04T18:49:01Z,2020-05-04T19:19:06Z,2020-05-04T19:10:33Z,OWNER,,"Add to this page: https://datasette.readthedocs.io/en/stable/installation.html

Here's how to install plugins: https://twitter.com/simonw/status/1257348687979778050
```
$ datasette plugins
[]

$ pipx inject datasette datasette-json-html            
  injected package datasette-json-html into venv datasette
done! ✨ 🌟 ✨

$ datasette plugins
[
    {
        ""name"": ""datasette-json-html"",
        ""static"": false,
        ""templates"": false,
        ""version"": ""0.6""
    }
]
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/756/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
630120235,MDU6SXNzdWU2MzAxMjAyMzU=,797,"Documentation for new ""params"" setting for canned queries",9599,simonw,closed,0,,,5512395,Datasette 0.44,3,2020-06-03T15:55:11Z,2020-06-09T04:00:40Z,2020-06-03T21:04:51Z,OWNER,,Added here: https://github.com/simonw/datasette/commit/aa82d0370463580f2cb10d9617f1bcbe45cc994a#diff-5e0ffd62fced7d46339b9b2cd167c2f9R236,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/797/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
733796942,MDU6SXNzdWU3MzM3OTY5NDI=,1075,PrefixedUrlString mechanism broke everything,9599,simonw,closed,0,,,6026070,0.51,6,2020-10-31T19:58:05Z,2020-10-31T20:48:51Z,2020-10-31T20:48:51Z,OWNER,,Added in 7a67bc7a569509d65b3a8661e0ad2c65f0b09166 refs #1026. Lots of tests are failing now.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1075/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
445855910,MDU6SXNzdWU0NDU4NTU5MTA=,475,Documentation for about and about_url metadata,9599,simonw,closed,0,,,4305096,0.28,0,2019-05-19T19:36:59Z,2019-05-19T20:13:36Z,2019-05-19T20:13:36Z,OWNER,,Added in https://github.com/simonw/datasette/commit/bf6b0f918de4aeee7c1036ac975ce2fb23237da7 without docs.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/475/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1386562662,I_kwDOCGYnMM5SpURm,493,Tiny typographical error in install/uninstall docs,9599,simonw,open,0,,,,,3,2022-09-26T19:00:42Z,2022-10-25T21:31:15Z,,OWNER,,"Added in:
- #483

I don't know how to fix this in Sphinx: I'm getting this: https://sqlite-utils.datasette.io/en/latest/cli.html#cli-install

> The [insert –convert](https://sqlite-utils.datasette.io/en/latest/cli.html#cli-insert-convert) and [query –functions](https://sqlite-utils.datasette.io/en/latest/cli.html#cli-query-functions) options

<img width=""849"" alt=""image"" src=""https://user-images.githubusercontent.com/9599/192358225-4fae509e-9fa8-4e8d-91d4-48aa1b79225e.png"">

But I want it to display `insert --convert` and not `insert –convert` there.

Here's the code: https://github.com/simonw/sqlite-utils/blob/85247038f70d7eb2f3e272cfeaa4c44459cafba8/docs/cli.rst#L2125",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/493/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
610853393,MDU6SXNzdWU2MTA4NTMzOTM=,104,"--schema option to ""sqlite-utils tables""",9599,simonw,closed,0,,,,,0,2020-05-01T16:55:49Z,2020-05-01T17:12:37Z,2020-05-01T17:12:37Z,OWNER,,Adds output showing the table schema.,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/104/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
459598080,MDU6SXNzdWU0NTk1OTgwODA=,520,asgi_wrapper plugin hook,9599,simonw,closed,0,9599,simonw,,,3,2019-06-23T17:16:45Z,2019-07-03T04:40:34Z,2019-07-03T04:06:28Z,OWNER,,"After #272 we can finally add this hook. It will allow plugins to wrap their own ASGI middleware around Datasette. Potential use-cases include:

* adding authentication
* custom CORS headers (see #454)
* maybe gzip support?
* possibly defining entirely new routes, though that may be better handled by a separate hook",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/520/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
465002978,MDU6SXNzdWU0NjUwMDI5Nzg=,550,Pull m2m faceting out of master so we can ship a release without it,9599,simonw,closed,0,,,4471010,Datasette 0.29,1,2019-07-07T23:10:48Z,2019-07-07T23:21:22Z,2019-07-07T23:21:22Z,OWNER,,After spending some time with #495 I believe I need to make some pretty major changes to how m2m faceting works. I don't want it to block the release of ASGI Datasette so I'm going to revert it back out of master for the moment and merge it back in after the release has gone out.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/550/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1452495049,I_kwDOBm6k_c5Wk1DJ,1899,Clicking within the CodeMirror area below the SQL (i.e. when there's only a single line) doesn't cause the editor to get focused ,95570,bgrins,closed,0,,,,,4,2022-11-17T00:29:52Z,2022-11-18T07:28:28Z,2022-11-18T07:20:53Z,CONTRIBUTOR,,"After the upgrade to 6 (#1893) I noticed this. I think it's because we're doing overflow:hidden to accomplish the CSS resizer.

When there's a single line of SQL there's a gap below that line where clicking doesn't do anything. It should focus at the end of the line.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1899/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
568091133,MDU6SXNzdWU1NjgwOTExMzM=,676,?_searchmode=raw option for running FTS searches without escaping characters,58088336,tunguyenatwork,closed,0,,,,,9,2020-02-20T06:56:57Z,2020-02-25T05:57:24Z,2020-02-25T05:56:04Z,NONE,,"After the version 0.34.  I am not able to use the wildchar in the _search option( or the full text search). It will not return any result unless I specify the whole word for text search.  

If I use 'match :search || ""*"" ' in the sql statement then it will work as expected.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/676/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
316123256,MDU6SXNzdWUzMTYxMjMyNTY=,229,Table view should support ?_size=400 parameter,9599,simonw,closed,0,,,,,1,2018-04-20T04:23:18Z,2018-04-26T04:49:46Z,2018-04-26T04:48:32Z,OWNER,,Allows callers to request more rows at once. The limit will still be `max_returned_rows` (defaults to 1000).,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/229/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
273537940,MDU6SXNzdWUyNzM1Mzc5NDA=,77,Add Travis CI badge to README,9599,simonw,closed,0,,,2857392,Ship first public release,0,2017-11-13T18:52:25Z,2017-11-13T21:24:15Z,2017-11-13T21:24:15Z,OWNER,,"Also fix this newline issue:

<img width=""647"" alt=""simonw_datasette__instant_json_api_for_your_sqlite_database"" src=""https://user-images.githubusercontent.com/9599/32743234-ae81b224-c860-11e7-98a9-980b7b448ffc.png"">
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/77/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1197926598,I_kwDOBm6k_c5HZujG,1705,How to upgrade your plugin for 1.0 documentation,9599,simonw,open,0,,,8755003,Datasette 1.0a-next,1,2022-04-08T23:16:47Z,2022-12-13T05:29:05Z,,OWNER,,"Among other things, needed by:
- #1704",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1705/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
503218205,MDU6SXNzdWU1MDMyMTgyMDU=,586,Enable browser caching for plugin statics with datasette-auth,9599,simonw,closed,0,,,,,2,2019-10-07T03:47:14Z,2019-10-07T15:46:04Z,2019-10-07T15:46:03Z,OWNER,,"An authenticated Datasette I run is seeing delays on every page load. On looking at the network inspector it turns out it's because datasette-vega is nearly 1MB and a `cache-control: private` is preventing it from being cached!

<img width=""1071"" alt=""github__repos__32_rows_where_where_private___1"" src=""https://user-images.githubusercontent.com/9599/66284064-f96f8400-e87a-11e9-875f-5a6c4a8c4f41.png"">


This may well turn out to be a bug in `datasette-auth-github` but it's still worth tracking here because caching of static assets from plugins is very important.
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/586/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
706091046,MDU6SXNzdWU3MDYwOTEwNDY=,165,Make .transform() a keyword arguments only function,9599,simonw,closed,0,,,5897911,2.20,0,2020-09-22T05:37:29Z,2020-09-24T20:35:47Z,2020-09-22T06:39:12Z,OWNER,,And rename the first argument from `columns=` to `types=`,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/165/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
749981663,MDU6SXNzdWU3NDk5ODE2NjM=,1104,config.json in directory config mode should be settings.json,9599,simonw,closed,0,,,6055094,Datasette 0.52,2,2020-11-24T19:34:38Z,2020-11-24T20:37:42Z,2020-11-24T20:37:41Z,OWNER,,"Another knock-on effect of #992.

https://github.com/simonw/datasette/blob/4bac9f18f9d04e5ed10f072502bcc508e365438e/docs/config.rst#L51-L55",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1104/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1175744654,I_kwDOCGYnMM5GFHCO,417,insert fails on JSONL with whitespace,9954,blaine,closed,0,,,,,3,2022-03-21T17:58:14Z,2022-03-25T21:19:06Z,2022-03-25T21:17:13Z,NONE,,"Any JSON that is newline-delimited and has whitespace (newlines) between the start of a JSON object and an attribute fails due to a parse error.

e.g. given the valid JSONL:

```{
  ""attribute"": ""value""
}
{
  ""attribute"": ""value2""
}
```

I would expect that `sqlite-utils insert --nl my.db mytable file.jsonl` would properly import the data into `mytable`. However, the following error is thrown instead:

`json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 2 column 1 (char 2)`

It makes sense that since the file is intended to be newline separated, the thing being parsed is ""{"" (which obviously fails), however the default newline-separated output of `jq` isn't compact. Using `jq -c` avoids this problem, but the fix is unintuitive and undocumented.

Proposed solutions:
1. Default to a ""loose"" newline-separated parse; this could be implemented internally as [the equivalent of] a `jq -c` filter ahead of the insert step.
2. Catch the JSONDecodeError (or pre-empt it in the case of a record === ""{\n"") and give the user a ""it looks like your json isn't _actually_ newline-delimited; try running it through `jq -c` instead"" error message.

It might just have been too early in the morning when I was playing with this, but running pipes of data through sqlite-utils without the 'knack' of it led to some false starts.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/417/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
797097140,MDU6SXNzdWU3OTcwOTcxNDA=,60,Use Data from SQLite in other commands,22578954,daniel-butler,open,0,,,,,3,2021-01-29T18:35:52Z,2021-02-12T18:29:43Z,,CONTRIBUTOR,,"As a total beginner here how could you access data from the sqlite table to run other commands.

What I am thinking is I want to get all the repos in an organization then using the repo list pull all the commit messages for each repo. 

I love this project by the way!",207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/60/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
503243784,MDU6SXNzdWU1MDMyNDM3ODQ=,3,Extract images into separate tables,9599,simonw,open,0,,,,,1,2019-10-07T05:43:01Z,2020-09-01T06:17:45Z,,MEMBER,,"As already done with authors. Slightly harder because images do not have a universally unique ID. Also need to figure out what to do about there being columns for both `image` and `images`.

<img width=""1522"" alt=""memory__items"" src=""https://user-images.githubusercontent.com/9599/66287418-9ab20680-e88a-11e9-96bf-6c80d881eff0.png"">
",213286752,pocket-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/3/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
557842245,MDU6SXNzdWU1NTc4NDIyNDU=,79,Helper methods for working with SpatiaLite,9599,simonw,closed,0,,,,,8,2020-01-31T00:39:19Z,2022-02-05T00:04:25Z,2022-02-04T05:55:11Z,OWNER,,"As demonstrated by this piece of documentation, using SpatiaLite with sqlite-utils requires a fair bit of boilerplate:
https://github.com/simonw/sqlite-utils/blob/f7289174e66ae4d91d57de94bbd9d09fabf7aff4/docs/python-api.rst#L880-L909",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/79/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
924203783,MDU6SXNzdWU5MjQyMDM3ODM=,1379,Idea: ?_end=1 option for streaming CSV responses,9599,simonw,open,0,,,,,0,2021-06-17T18:11:21Z,2021-06-17T18:11:30Z,,OWNER,,"As discussed in this thread: https://twitter.com/simonw/status/1405554676993433605 - one of the disadvantages of Datasette's streaming CSV feature is that it's hard to tell if you got the whole file or if the connection ended early - or if an error occurred.

Idea: offer an optional `?_end=1` parameter which, if enabled, adds a single row to the end of the CSV file that looks like this:

`END,,,,,,,,,`

For however many columns the CSV file usually has.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1379/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1338137350,I_kwDOBm6k_c5PwlsG,1781,Ensure Datasette Lite is promoted in docs and README,9599,simonw,closed,0,,,8303187,Datasette 0.62,1,2022-08-14T05:12:35Z,2022-08-14T15:24:40Z,2022-08-14T15:24:40Z,OWNER,,As of 0.62 https://lite.datasette.io is a supported piece of the overall Datasette ecosystem.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1781/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
749979454,MDU6SXNzdWU3NDk5Nzk0NTQ=,1103,Rename /-/config to /-/settings,9599,simonw,closed,0,,,6055094,Datasette 0.52,2,2020-11-24T19:31:00Z,2020-11-24T20:19:20Z,2020-11-24T20:19:19Z,OWNER,,"As part of rebranding config to settings, see also #992.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1103/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1197925865,I_kwDOBm6k_c5HZuXp,1704,File PRs against incompatible plugins pinning to datasette<1.0,9599,simonw,open,0,,,3268330,Datasette 1.0,0,2022-04-08T23:15:30Z,2022-04-08T23:15:30Z,,OWNER,,"As part of the preparation for the 1.0 release, test all existing known plugins against the alpha.

For any that break, submit a PR suggesting they pin to a version <1.0 - and include a link to the documentation on how to upgrade the plugin for 1.0.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1704/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1450796965,I_kwDOBm6k_c5WeWel,1894,Initialize CodeMirror during DOMContentLoaded instead of onload,95570,bgrins,closed,0,,,,,0,2022-11-16T03:52:19Z,2022-11-18T07:29:02Z,2022-11-18T07:29:02Z,CONTRIBUTOR,,As per https://github.com/simonw/datasette/pull/1893/files#r1023248927 this should prevent a flash between the textarea being replaced by CodeMirror.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1894/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
309471814,MDU6SXNzdWUzMDk0NzE4MTQ=,189,Ability to sort (and paginate) by column,9599,simonw,closed,0,9599,simonw,,,31,2018-03-28T18:04:51Z,2018-04-15T18:54:22Z,2018-04-09T05:16:02Z,OWNER,,"As requested in https://github.com/simonw/datasette/issues/185#issuecomment-376614973

I've previously avoided this for performance reasons: sort-by-column on a column without an index is likely to perform badly for hundreds of thousands of rows.

That's not a good enough reason to avoid the feature entirely though. A few options:

* Allow sort-by-column by default, give users the option to disable it for specific tables/columns
* Disallow sort-by-column by default, give users option (probably in `metadata.json`) to enable it for specific tables/columns
* Automatically detect if a column either has an index on it OR a table has less than X rows in it

We already have the mechanism in place to cut off SQL queries that take more than X seconds, so if someone DOES try to sort by a column that's too expensive it won't actually hurt anything - but it would be nice to not show people a ""sort"" option which is guaranteed to throw a timeout error.

The vast majority of datasette usage that I've seen so far is on smaller datasets where the performance penalties of sort-by-column are extremely unlikely to show up.

----

Still left to do:

- [x] UI that shows which sort order is currently being applied (in HTML and in JSON)
- [x] UI for applying a sort order (with rel=nofollow to avoid Google crawling it)
- [x] Sort column names should be escaped correctly in generated SQL
- [x] Validation that the selected sort order is a valid column
- [x] Throw error if user attempts to apply _sort AND _sort_desc at the same time
- [x] Ability to disable sorting (or sort only for specific columns) in metadata.json
- [x] Fix ""201 rows where sorted by sortable_with_nulls "" bug
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/189/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
958516743,MDU6SXNzdWU5NTg1MTY3NDM=,306,Configure sphinx.ext.extlinks for issues,9599,simonw,closed,0,,,,,2,2021-08-02T21:19:19Z,2021-08-02T21:39:34Z,2021-08-02T21:29:22Z,OWNER,,As seen in Datasette: https://github.com/simonw/datasette/issues/1227,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/306/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1816857442,I_kwDOBm6k_c5sSwti,2106,`datasette install -e` option,9599,simonw,closed,0,,,,,3,2023-07-22T18:33:42Z,2023-07-26T18:28:33Z,2023-07-22T18:42:54Z,OWNER,,"As seen in LLM and now in `sqlite-utils` too:
- https://github.com/simonw/sqlite-utils/issues/570

Useful for developing plugins, see tutorial at https://llm.datasette.io/en/stable/plugins/tutorial-model-plugin.html",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2106/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1816857105,I_kwDOCGYnMM5sSwoR,570,`sqlite-utils install -e` option,9599,simonw,closed,0,,,,,0,2023-07-22T18:32:23Z,2023-07-22T18:55:59Z,2023-07-22T18:32:56Z,OWNER,,"As seen in LLM.

Needed while working on:
- #567",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/570/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
601358649,MDU6SXNzdWU2MDEzNTg2NDk=,100,"Mechanism for forcing column-type, over-riding auto-detection",9599,simonw,closed,0,,,,,3,2020-04-16T19:12:52Z,2020-04-17T23:53:32Z,2020-04-17T23:53:32Z,OWNER,,"As seen in https://github.com/dogsheep/github-to-sqlite/issues/27#issuecomment-614843406 - there's a problem where you insert a record with a `None` value for a column and that column is created as `TEXT` - but actually you intended it to be an `INT` (as later examples will demonstrate).

Some kind of mechanism for over-riding the detected types of columns would be useful here.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/100/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
959305209,MDU6SXNzdWU5NTkzMDUyMDk=,307,codespell to spell check documentation,9599,simonw,closed,0,,,,,0,2021-08-03T16:48:19Z,2021-08-03T16:48:53Z,2021-08-03T16:48:53Z,OWNER,,As seen in https://github.com/simonw/datasette/issues/1417 and https://til.simonwillison.net/python/codespell,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/307/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1292060682,I_kwDOCGYnMM5NA0gK,450,Add --ignore option to more commands,9599,simonw,closed,0,,,,,9,2022-07-02T13:52:02Z,2022-07-15T22:39:09Z,2022-07-15T22:37:45Z,OWNER,,"As seen in https://sqlite-utils.datasette.io/en/stable/cli-reference.html#add-foreign-key

Could make this TIL trick unnecessary: https://til.simonwillison.net/bash/ignore-errors",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/450/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
944846776,MDU6SXNzdWU5NDQ4NDY3NzY=,297,Option for importing CSV data using the SQLite .import mechanism,9599,simonw,open,0,,,,,23,2021-07-14T22:36:41Z,2023-09-22T20:49:52Z,,OWNER,,"As seen in https://til.simonwillison.net/sqlite/import-csv - `.mode csv` and then `.import school.csv schools` is hugely faster than importing via `sqlite-utils insert` and doing the work in Python - but it can only be implemented by shelling out to the `sqlite3` CLI tool, it's not functionality that is exposed to the Python `sqlite3` module.

An option to use this would be useful - maybe something like this:

    sqlite-utils insert blah.db blah blah.csv --fast",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/297/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1617962395,I_kwDOJHON9s5gcCWb,10,Include schema in README,9599,simonw,closed,0,,,,,0,2023-03-09T20:38:59Z,2023-03-09T20:48:18Z,2023-03-09T20:48:18Z,MEMBER,,As seen in other tools like https://github.com/simonw/git-history,611552758,apple-notes-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/10/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1243704847,I_kwDOCGYnMM5KIW4P,435,Switch to Furo documentation theme,9599,simonw,closed,0,,,,,2,2022-05-20T21:46:39Z,2022-05-20T21:56:10Z,2022-05-20T21:54:43Z,OWNER,,"As seen in:
- https://github.com/simonw/datasette/issues/1746
- https://github.com/simonw/shot-scraper/issues/77",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/435/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
695556681,MDU6SXNzdWU2OTU1NTY2ODE=,19,Figure out incremental re-indexing,9599,simonw,open,0,,,,,2,2020-09-08T05:23:31Z,2020-09-08T05:27:07Z,,MEMBER,,As tables get bigger reindexing everything on a schedule (essentially recreating the entire index from scratch) will start to become a performance bottleneck.,197431109,dogsheep-beta,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-beta/issues/19/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1531991339,I_kwDOBm6k_c5bUFUr,1989,Suggestion: Hiding columns,116795,pax,open,0,,,,,3,2023-01-13T09:33:32Z,2023-03-31T06:18:05Z,,NONE,,As there's the possibility of [hiding tables](https://docs.datasette.io/en/stable/metadata.html#hiding-tables) - I've run into the **need of hiding specific columns** - data that's either not relevant for public or can't be shown due to privacy reasons. ,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1989/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
273267081,MDU6SXNzdWUyNzMyNjcwODE=,70,Paginate views using OFFSET/LIMIT,9599,simonw,closed,0,,,2857392,Ship first public release,0,2017-11-12T21:30:29Z,2017-11-13T21:11:01Z,2017-11-13T21:11:01Z,OWNER,,"As with #69 these should obey a maximum offset setting, which can be over-ridden.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/70/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
267522549,MDU6SXNzdWUyNjc1MjI1NDk=,11,Code that generates compile-time properties about the database ,9599,simonw,closed,0,,,2857392,Ship first public release,1,2017-10-23T02:18:24Z,2017-10-23T16:04:23Z,2017-10-23T16:04:23Z,OWNER,,"At a minimum this will include:

* sha hash of each database file
* list of tables with row counts for each database file",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/11/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1174708375,I_kwDOBm6k_c5GBKCX,1673,Streaming CSV spends a lot of time in `table_column_details`,9599,simonw,open,0,,,,,1,2022-03-20T22:25:28Z,2022-03-20T22:34:06Z,,OWNER,,"At least I think it does. I tried running `py-spy top -p $PID` against a Datasette process that was trying to do:

    datasette covid.db --get '/covid/ny_times_us_counties.csv?_size=10&_stream=on'

While investigating:
- #1355

And spotted this:
```
datasette covid.db --get /covid/ny_times_us_counties.csv?_size=10&_stream=on' (python v3.10.2)
Total Samples 5800
GIL: 71.00%, Active: 98.00%, Threads: 4

  %Own   %Total  OwnTime  TotalTime  Function (filename:line)                                                                                                                                            
  8.00%   8.00%    4.32s     4.38s   sql_operation_in_thread (datasette/database.py:212)
  5.00%   5.00%    3.77s     3.93s   table_column_details (datasette/utils/__init__.py:614)
  6.00%   6.00%    3.72s     3.72s   _worker (concurrent/futures/thread.py:81)
  7.00%   7.00%    2.98s     2.98s   _read_from_self (asyncio/selector_events.py:120)
  5.00%   6.00%    2.35s     2.49s   detect_fts (datasette/utils/__init__.py:571)
  4.00%   4.00%    1.34s     1.34s   _write_to_self (asyncio/selector_events.py:140)
```
Relevant code: https://github.com/simonw/datasette/blob/798f075ef9b98819fdb564f9f79c78975a0f71e8/datasette/utils/__init__.py#L609-L625
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1673/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
275159710,MDU6SXNzdWUyNzUxNTk3MTA=,128,"Every visualization should have an ""embed"" button",9599,simonw,open,0,,,,,0,2017-11-19T13:38:13Z,2019-05-13T18:33:51Z,,OWNER,,"At least for the first round of visualizations, any time you construct one using the UI the result should include an ""embed this"" button that returns source code to copy and paste

These examples should use unpkg.com (or similarl) urls with SRI hashes, eg https://www.srihash.org - and should load data from the datasette JSON API.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/128/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
2028698018,I_kwDOBm6k_c5463mi,2213,feature request: gzip compression of database downloads,536941,fgregg,open,0,,,,,1,2023-12-06T14:35:03Z,2023-12-06T15:05:46Z,,CONTRIBUTOR,,"At the bottom of database pages, datasette gives users the opportunity to download the underlying sqlite database. It would be great if that could be served gzip compressed. 

this is similar to #1213, but for me, i don't need datasette to compress html and json because my CDN layer does it for me, however, cloudflare at least, will not compress a mimetype of ""application""

(see list of mimetype: https://developers.cloudflare.com/speed/optimization/content/brotli/content-compression/)",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2213/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
599776345,MDU6SXNzdWU1OTk3NzYzNDU=,24,Feature idea: github-to-sqlite everything ...,9599,simonw,open,0,,,,,0,2020-04-14T18:34:00Z,2020-04-14T18:34:00Z,,MEMBER,,"At the moment if you want to pull all your repos, issues, issues comments etc you have to do it with a sequence of separate commands.

Consider adding a `everything` or `all` command which fetches everything that the tool knows how to fetch, and is designed to be run on a cron in a way that fetches just new stuff each time.",207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/24/reactions"", ""total_count"": 7, ""+1"": 7, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
754178780,MDU6SXNzdWU3NTQxNzg3ODA=,1121,Table actions cog is misaligned,3243482,abdusco,closed,0,,,,,1,2020-12-01T08:41:25Z,2020-12-03T01:03:19Z,2020-12-03T00:33:36Z,CONTRIBUTOR,,"At the moment it looks like this
https://datasette-graphql-demo.datasette.io/github/repos

![image](https://user-images.githubusercontent.com/3243482/100716533-e6e2d300-33c9-11eb-866e-1e83ba228bf5.png)

Adding a few flex statements fixes the alignment and centers `h1` text and the cog icon vertically.
![image](https://user-images.githubusercontent.com/3243482/100716605-f8c47600-33c9-11eb-8d69-0e37499cf641.png)

",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1121/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
377266351,MDU6SXNzdWUzNzcyNjYzNTE=,373,Views should be shown on root/index page along with tables,416374,gfrmin,closed,0,,,4305096,0.28,1,2018-11-05T06:28:41Z,2019-05-16T00:29:22Z,2019-05-16T00:29:22Z,CONTRIBUTOR,,"At the moment the number of views is given on a datasette ""homepage"", but not links to any views themselves",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/373/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
374953006,MDU6SXNzdWUzNzQ5NTMwMDY=,369,Interface should show same JSON shape options for custom SQL queries,416374,gfrmin,open,0,,,3268330,Datasette 1.0,2,2018-10-29T10:39:15Z,2020-05-30T17:24:06Z,,CONTRIBUTOR,,"At the moment the page returning a custom SQL query shows the JSON and CSV APIs, but not the multiple JSON shapes. However, adding the `_shape` parameter to the JSON API URL manually still works, so perhaps there should be consistency in the interface by having the same ""Advanced Export"" box for custom SQL queries.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/369/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
711627628,MDU6SXNzdWU3MTE2Mjc2Mjg=,981,Action menu for table columns,9599,simonw,closed,0,,,5971510,Datasette 0.50,16,2020-09-30T04:45:38Z,2020-10-08T23:55:00Z,2020-09-30T23:58:17Z,OWNER,,"At the very least I'd like a menu on each table column that lets me select sort-asc v.s. sort-desc without having to click twice.

I'd also like to be able to indicate that a column should be used for faceting (possibly only for columns that are not floating point and do not have a unique index on them).

This needs to be built with accessibility in mind - I don't want screenreaders to read out the contents of a menu as the ""th"" label for any given cell.

Related: #690",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/981/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1279144769,I_kwDOCGYnMM5MPjNB,448,Reading rows from a file => AttributeError: '_io.StringIO' object has no attribute 'readinto',236907,mungewell,closed,0,,,,,5,2022-06-21T21:48:27Z,2023-05-08T22:01:00Z,2023-05-08T22:01:00Z,NONE,,"Attempting to run the example given here (without extra bracket ;-):
https://sqlite-utils.datasette.io/en/stable/python-api.html#reading-rows-from-a-file
```
from sqlite_utils.utils import rows_from_file
import io

rows, format = rows_from_file(io.StringIO(""id,name\n1,Cleo""))
print(list(rows), format)
# Outputs [{'id': '1', 'name': 'Cleo'}] Format.CSV
```

Gives error
```
>""c:\Program Files\Python37\python.exe"" test2.py
Traceback (most recent call last):
  File ""test2.py"", line 4, in <module>
    rows, format = rows_from_file(io.StringIO(""id,name\n1,Cleo""))
  File ""C:\Users\swood\Downloads\sqlite-utils-main-20220621\sqlite-utils-main\sqlite_utils\utils.py"", line 300, in rows_from_file
    first_bytes = buffered.peek(2048).strip()
AttributeError: '_io.StringIO' object has no attribute 'readinto'
```

I am running Python on Windows.
```
>""c:\Program Files\Python37\python.exe""
Python 3.7.4 (tags/v3.7.4:e09359112e, Jul  8 2019, 20:34:20) [MSC v.1916 64 bit (AMD64)] on win32
Type ""help"", ""copyright"", ""credits"" or ""license"" for more information.
```",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/448/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
273192789,MDU6SXNzdWUyNzMxOTI3ODk=,67,Command that builds a local docker container,9599,simonw,closed,0,,,2857392,Ship first public release,2,2017-11-12T02:13:29Z,2017-11-13T16:17:52Z,2017-11-13T16:17:52Z,OWNER,,Be nice to indicate that this isn't just for Now. Shouldn't be too hard either.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/67/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1099584685,I_kwDOCGYnMM5BilSt,381,`sqlite-utils rows` options `--limit` and `--offset`,9599,simonw,closed,0,,,,,2,2022-01-11T20:23:12Z,2022-01-11T23:33:37Z,2022-01-11T23:19:36Z,OWNER,,Because I often want to use it just to preview a few rows from the database. Piping through `| head -n 20` works for JSON and CSV (they stream) but not for `--table`.,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/381/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1237871948,I_kwDOBm6k_c5JyG1M,1743,`datasette.utils.to_css_class()` should be a documented internal,9599,simonw,open,0,,,,,0,2022-05-16T23:57:26Z,2022-05-16T23:57:26Z,,OWNER,,"Because I'm using it in this plugin:
- https://github.com/simonw/datasette-upload-dbs/issues/1",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1743/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
442327592,MDU6SXNzdWU0NDIzMjc1OTI=,456,Installing installs the tests package,7725188,hellerve,closed,0,,,,,3,2019-05-09T16:35:16Z,2020-07-24T20:39:54Z,2020-07-24T20:39:54Z,CONTRIBUTOR,,"Because `setup.py` uses `find_packages` and `tests` is on the top-level, `pip install datasette` will install a top-level package called `tests`, which is probably not desired behavior.

The offending line is here:
https://github.com/simonw/datasette/blob/bfa2ae0d16d39bb82dbe4da4f3fdc3c7f6257418/setup.py#L40

And only `pip uninstall datasette` with a conflicting package would warn you by default; apparently another package had the same problem, which is why I get this message when uninstalling:

```
$ pip uninstall datasette
Uninstalling datasette-0.27:
  Would remove:
    /usr/local/bin/datasette
    /usr/local/lib/python3.7/site-packages/datasette-0.27.dist-info/*
    /usr/local/lib/python3.7/site-packages/datasette/*
    /usr/local/lib/python3.7/site-packages/tests/*
  Would not remove (might be manually added):
    [ .. snip .. ]
Proceed (y/n)? 
```

This should be a relatively simple fix, and I could drop a PR if desired!

Cheers",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/456/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
955316250,MDU6SXNzdWU5NTUzMTYyNTA=,1405,utils.parse_metadata() should be a documented internal function,9599,simonw,closed,0,,,,,3,2021-07-28T23:51:39Z,2021-07-29T23:33:30Z,2021-07-29T23:30:24Z,OWNER,,Because it's used by this plugin: https://github.com/simonw/datasette-remote-metadata,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1405/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
864969683,MDU6SXNzdWU4NjQ5Njk2ODM=,1305,Index view crashes when any database table is not accessible to actor,416374,gfrmin,closed,0,,,,,0,2021-04-22T13:44:22Z,2021-06-02T04:26:29Z,2021-06-02T04:26:29Z,CONTRIBUTOR,,"Because of https://github.com/simonw/datasette/blob/main/datasette/views/index.py#L63, the ```tables``` dict built does not include invisible tables; however, if https://github.com/simonw/datasette/blob/main/datasette/views/index.py#L80 is reached (because table_counts was not successfully initialized, e.g. due to a very large database) then as db.get_all_foreign_keys() returns ALL tables, a KeyError will be raised.

This error can be recreated with the fixtures.db if any table is hidden, e.g. by adding something like                 ```""foreign_key_references"": {
                    ""allow"": {}
                }``` to fixtures-metadata.json and deleting ```or not table_counts``` from https://github.com/simonw/datasette/blob/main/datasette/views/index.py#L77.

I'm not sure how to fix this error; perhaps by testing if the table is in the aforementions ```tables``` dict.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1305/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
435819321,MDU6SXNzdWU0MzU4MTkzMjE=,436,400 Error when trying to register new user via https://publish.datasettes.com/,317694,nniiicc,closed,0,,,,,1,2019-04-22T17:55:00Z,2021-01-04T20:15:42Z,2021-01-04T20:15:41Z,NONE,,"Behavior: When registering a new user via Zeit - confirmation is sent and screen acknowledges registered user... When clicking grant access the next screen is a white 400 error message. 

Replicated: Chrome and Firefox; 2 different email accounts",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/436/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
691475400,MDU6SXNzdWU2OTE0NzU0MDA=,958,Upgrade to latest Black (20.8b1),9599,simonw,closed,0,,,5818042,Datasette 0.49,0,2020-09-02T22:24:19Z,2020-09-11T21:34:24Z,2020-09-02T22:25:10Z,OWNER,,"Black has some changes: https://black.readthedocs.io/en/stable/change_log.html#b0 - in particular:

> - re-implemented support for explicit trailing commas: now it works consistently within any bracket pair, including nested structures (#1288 and duplicates)
> - Black now reindents docstrings when reindenting code around it (#1053)",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/958/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
738128913,MDU6SXNzdWU3MzgxMjg5MTM=,201,.search(columns=) and sqlite-utils search -c ... bug,9599,simonw,closed,0,,,6079500,3.0,1,2020-11-07T01:27:26Z,2020-11-08T16:54:15Z,2020-11-08T16:54:15Z,OWNER,,"Both `table.search(columns=)` and the `sqlite-utils search -c` option do not work as expected - they always return both the `rowid` and the `rank` columns even if those have not been requested.

This should be fixed before the 3.0 non-alpha release.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/201/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
413842611,MDU6SXNzdWU0MTM4NDI2MTE=,14,Utilities for adding indexes,9599,simonw,closed,0,,,,,3,2019-02-24T16:57:28Z,2019-02-24T19:11:28Z,2019-02-24T19:11:28Z,OWNER,,"Both in the Python API and the CLI tool. For the CLI tool this should work:

    $ sqlite-utils create-index mydb.db mytable col1 col2

This will create a compound index across col1 and col2. The name of the index will be automatically chosen unless you use the `--name=...` option.

Support a `--unique` option too.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/14/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1122416919,I_kwDOBm6k_c5C5rkX,1623,/-/patterns returns link: alternate JSON header to 404,9599,simonw,closed,0,,,3268330,Datasette 1.0,2,2022-02-02T21:42:49Z,2022-03-19T04:04:49Z,2022-02-02T21:48:56Z,OWNER,,"Bug from:
- #1620

```
% curl -s -I 'https://latest.datasette.io/-/patterns' | grep link
link: https://latest.datasette.io/-/patterns.json; rel=""alternate""; type=""application/json+datasette""
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1623/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
586486367,MDU6SXNzdWU1ODY0ODYzNjc=,95,Columns with only null values are no longer created in the database,9599,simonw,closed,0,,,,,0,2020-03-23T20:07:42Z,2020-03-23T20:31:15Z,2020-03-23T20:31:15Z,OWNER,,"Bug introduced in #94, and released in `2.4.3`.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/95/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
462117311,MDU6SXNzdWU0NjIxMTczMTE=,531,/database/-/inspect,9599,simonw,open,0,,,,,1,2019-06-28T16:33:41Z,2019-07-08T15:43:57Z,,OWNER,,"Build `/database/-/inspect` which shows tables, columns, column types and foreign keys

It won't show table counts. Or maybe it will include them optionally but only for `-i` databases, in a special area of the JSON reserved for immutable-only inspect details.

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/465#issuecomment-506797086_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/531/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1066288689,I_kwDOBm6k_c4_jkYx,1538,Research pattern for re-registering existing Click tools with register_commands,9599,simonw,closed,0,,,,,3,2021-11-29T17:09:47Z,2021-11-29T17:32:44Z,2021-11-29T17:27:16Z,OWNER,,"Building a Datasette plugin that imports an existing Click CLI tool and re-registers it is proving hard - Click doesn't really want you to do that. I tried this:
```python
from datasette import hookimpl
from git_history.cli import file as git_history_file


@hookimpl
def register_commands(cli):
    cli.command(name=""git-history"")(git_history_file.callback)
```
But when I run this:
```
 % datasette git-history --help     
Usage: datasette git-history [OPTIONS]

  Analyze the history of a specific file and write it to SQLite

Options:
  --help  Show this message and exit.
```
The options are all missing - which means that the command doesn't actually work. Will need to research this pattern separately.

_Originally posted by @simonw in https://github.com/simonw/git-history/issues/21#issuecomment-981835305_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1538/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
465001185,MDU6SXNzdWU0NjUwMDExODU=,549,Send pull request to the repo that the _table.html template will break,9599,simonw,closed,0,,,4471010,Datasette 0.29,1,2019-07-07T22:45:17Z,2019-07-08T03:36:46Z,2019-07-08T03:36:45Z,OWNER,,"Bump this to 0.29 https://github.com/simonw/salaries-datasette/blob/master/requirements/base.txt
And rename https://github.com/simonw/salaries-datasette/blob/master/templates/_rows_and_columns.html to _table.html",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/549/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1504352503,I_kwDOBm6k_c5Zqpj3,1968,Allow to hide some queries in metadata.yml,562352,CharlesNepote,open,0,,,,,0,2022-12-20T10:45:41Z,2022-12-20T10:45:41Z,,NONE,,"By default all queries are displayed.

But there are many cases where it would be interesting to hide the queries by default:
* the website is targeting non-tech people
* the query is veeeeeery long ([eg.](https://mirabelle.openfoodfacts.org/products/energy_calculator))
* reading the query is not important for the users, they only want to see the result

Of course, the user still could have the option to see the query.

It could be an option in the metadata file:
```yml
databases:
  awesome_db:
    tables:
      products:
        hide_sql: true
    queries:
      great_query:
        hide_sql: true
        sql: select * from products where code = :barcode
```

The priority could be:
* no option in the metadata and nothing in the URL: query displayed
* hide_sql in the metadata and nothing in the URL: query displayed as asked in the metadata
* hide_sql in the metadata and &_hide_sql= in the URL: query as asked in the URL

See also: #1824
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1968/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1605959201,I_kwDOBm6k_c5fuP4h,2032,datasette errors when foreign key integrity is enabled,193185,cldellow,open,0,,,,,0,2023-03-02T01:27:51Z,2023-03-02T01:31:58Z,,CONTRIBUTOR,,"By default, [SQLite does not enforce foreign key constraints](https://www.sqlite.org/foreignkeys.html#fk_enable). I typically enable these checks by running:

```sql
PRAGMA foreign_keys = ON;
```

inside of a `prepare_connection` hook.

If a plugin causes the schema to change (eg datasette-scraper creating a new table, or datasette-edit-schema changing a column), then https://github.com/simonw/datasette/blob/0b4a28691468b5c758df74fa1d72a823813c96bf/datasette/utils/internal_db.py#L71-L77 will fail with:

```
FOREIGN KEY constraint failed
```

This could be resolved by either:
- deleting from the `tables` column last
- changing the schema so that the foreign keys have [ON DELETE CASCADE](https://www.sqlite.org/foreignkeys.html#fk_actions)

Let me know if you'd be open to a PR that addresses this -- since foreign key constraints aren't enabled by default, I guess it's questionable whether this is a bug. I think I can workaround this by inspecting the database parameter in `prepare_connection` and trying not to enable fkey checks on the `_internal` database.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2032/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1487757143,I_kwDOCGYnMM5YrV9X,517,Drop support for Python 3.6,9599,simonw,closed,0,,,,,1,2022-12-10T01:23:31Z,2022-12-10T01:36:36Z,2022-12-10T01:36:36Z,OWNER,,"CI has started failing for Python 3.6: https://github.com/simonw/sqlite-utils/actions/runs/3576322798

<img width=""945"" alt=""image"" src=""https://user-images.githubusercontent.com/9599/206821688-e4176cb3-ea82-460d-8e93-626fb4fa0269.png"">

It's fixable by swiching away from `ubuntu-latest` according to:

- https://github.com/actions/setup-python/issues/355#issuecomment-1335042510

But https://endoflife.date/python says that 3.6 end of life was almost 6 years ago, and end of security support nearly 1 year ago.

So I'm OK dropping support entirely - Python 3.6 users will still be able to install version 3.30, just not any releases that come next.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/517/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1099585611,I_kwDOCGYnMM5BilhL,382,`--where` option for `sqlite-rows`,9599,simonw,closed,0,,,,,1,2022-01-11T20:24:23Z,2022-01-11T23:33:14Z,2022-01-11T23:32:47Z,OWNER,,CLI equivalent of `table.rows_where()` - should accept parameters too. Work on this at the same time as #381.,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/382/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1306548397,I_kwDOCGYnMM5N4Fit,454,CLI command for duplicating tables,9599,simonw,closed,0,,,,,1,2022-07-15T21:31:27Z,2022-07-15T21:48:23Z,2022-07-15T21:45:51Z,OWNER,,"CLI equivalent of:
- #449",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/454/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
491219910,MDU6SXNzdWU0OTEyMTk5MTA=,61,importing CSV to SQLite as library,17739,witeshadow,closed,0,,,,,2,2019-09-09T17:12:40Z,2019-11-04T16:25:01Z,2019-11-04T16:25:01Z,NONE,,"CSV can be imported to SQLite when used CLI, but I don't see documentation for when using as library. ",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/61/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1785360409,I_kwDOCGYnMM5qanAZ,563,`--empty-null` option when importing CSV,9599,simonw,closed,0,,,,,1,2023-07-03T05:23:36Z,2023-07-03T05:44:43Z,2023-07-03T05:42:30Z,OWNER,,"CSV files with empty cells in (which come through as the empty string) are common and a bit gross.

Having an option that means ""and if it's an empty string store `null` instead) would be cool.

I brainstormed name options here https://chat.openai.com/share/c947b738-ee7d-419c-af90-bc84e90987da",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/563/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
534629631,MDU6SXNzdWU1MzQ2Mjk2MzE=,650,Add a glossary to the documentation,9599,simonw,open,0,,,,,3,2019-12-09T00:23:45Z,2022-01-13T22:04:56Z,,OWNER,,"Call it `glossary.rst` - it can use a definition list something like this:
```rst
.. _glossary:

Glossary
========

Term
  A definition of the term.

Another term
  Another definition.
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/650/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1099586786,I_kwDOCGYnMM5Bilzi,383,Add documentation page with the output of `--help`,9599,simonw,closed,0,,,,,4,2022-01-11T20:25:58Z,2022-01-11T22:55:05Z,2022-01-11T21:44:05Z,OWNER,,"Can be maintained using `cog` from #373. Similar in purpose to the API reference page, but this is for the CLI.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/383/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
648435885,MDU6SXNzdWU2NDg0MzU4ODU=,878,"New pattern for views that return either JSON or HTML, available for plugins",9599,simonw,open,0,,,3268330,Datasette 1.0,26,2020-06-30T19:26:13Z,2022-03-19T16:19:30Z,,OWNER,,"Can be part of #870 - refactoring existing views to use `register_routes()`.

> I'm going to put the new `check_permissions()` method on `BaseView` as well. If I want that method to be available to plugins I can do so by turning that `BaseView` class into a documented API that plugins are encouraged to use themselves.
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/832#issuecomment-651995453_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/878/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
761706858,MDU6SXNzdWU3NjE3MDY4NTg=,1137,Update README to reflect new datasette.io site,9599,simonw,closed,0,,,,,0,2020-12-10T23:22:06Z,2020-12-10T23:28:50Z,2020-12-10T23:28:50Z,OWNER,,Can finally close #659.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1137/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
611222968,MDU6SXNzdWU2MTEyMjI5Njg=,107,sqlite-utils create-view CLI command,9599,simonw,closed,0,,,,,2,2020-05-02T16:15:13Z,2020-05-03T15:36:58Z,2020-05-03T15:36:37Z,OWNER,,Can go with #27 - `sqlite-utils create-table`.,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/107/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
505674949,MDU6SXNzdWU1MDU2NzQ5NDk=,17,import command should empty all archive-* tables first,9599,simonw,closed,0,,,,,2,2019-10-11T06:58:43Z,2019-10-11T15:40:08Z,2019-10-11T15:40:08Z,MEMBER,,Can have a CLI option for NOT doing that.,206156866,twitter-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/17/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
961008507,MDU6SXNzdWU5NjEwMDg1MDc=,308,Add an interactive tutorial as a Jupyter notebook,9599,simonw,open,0,,,,,2,2021-08-04T20:34:22Z,2021-08-04T21:30:59Z,,OWNER,,Can show people how to open this up in Binder.,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/308/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
734777631,MDU6SXNzdWU3MzQ3Nzc2MzE=,1080,"""View all"" option for facets, to provide a (paginated) list of ALL of the facet counts plus a link to view them",9599,simonw,open,0,,,3268330,Datasette 1.0,7,2020-11-02T19:55:06Z,2022-02-04T06:25:18Z,,OWNER,,Can use `/database/-/...` namespace from #296,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1080/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
345469355,MDU6SXNzdWUzNDU0NjkzNTU=,351,Automatically create a GitHub release linking to release notes for every tagged release,9599,simonw,closed,0,,,,,1,2018-07-28T18:31:12Z,2020-05-28T18:56:16Z,2020-05-28T18:56:15Z,OWNER,,"Can use this API called from Travis: https://developer.github.com/v3/repos/releases/#create-a-release

The release it generates should look like this one: https://github.com/simonw/datasette/releases/tag/0.24",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/351/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
275228834,MDU6SXNzdWUyNzUyMjg4MzQ=,136,"""Reformat SQL"" button next to SQL editor textarea",9599,simonw,closed,0,,,,,0,2017-11-20T03:42:19Z,2019-10-14T03:46:13Z,2019-10-14T03:46:13Z,OWNER,,"Can use this:

https://github.com/zeroturnaround/sql-formatter
https://zeroturnaround.github.io/sql-formatter/
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/136/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
640917326,MDU6SXNzdWU2NDA5MTczMjY=,852,canned_queries() plugin hook,9599,simonw,closed,0,,,5533512,Datasette 0.45,9,2020-06-18T05:24:35Z,2020-06-20T03:08:40Z,2020-06-20T03:08:40Z,OWNER,,"Canned queries are currently baked into `metadata.json` which is read once on startup.

Allowing users to interactively create new canned queries - even if just through a plugin - would make a lot of sense.

Is this a new plugin hook or some other mechanism? Lots to think about here.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/852/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
582517965,MDU6SXNzdWU1ODI1MTc5NjU=,698,Ability for a canned query to write to the database,9599,simonw,closed,0,,,5512395,Datasette 0.44,26,2020-03-16T18:31:59Z,2020-06-06T19:43:49Z,2020-06-06T19:43:48Z,OWNER,,"Canned queries are currently read-only: https://datasette.readthedocs.io/en/0.38/sql_queries.html#canned-queries

Add a `""write"": true` option to their definition in `metadata.json` which turns them into queries that are submitted via POST and send their queries to the write queue.

Then they can be used as a really quick way to define a writable interface and JSON API!",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/698/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
620969465,MDU6SXNzdWU2MjA5Njk0NjU=,767,Allow to specify a URL fragment for canned queries,2657547,rixx,closed,0,,,5471110,Datasette 0.43,2,2020-05-19T13:17:42Z,2020-05-27T21:52:25Z,2020-05-27T21:52:25Z,CONTRIBUTOR,,"Canned queries are very useful to direct users to prepared data and views. I like to use them with charts using datasette-vega a lot, because people get a direct impression at first glance.

datasette-vega doesn't show up by default though, and users have to click through to it. Also, datasette-vega does not always guess the best way to render columns correctly though, so it would be nice if I could specify a URL fragment in my canned queries to make sure people see what I want them to see.

My current workaround is to include a fragement link in ``description_html`` and ask people to reload the page, like [here](https://data.rixx.de/songs/show_by_bpm#g.mark=bar&g.x_column=bpm_floor&g.x_type=ordinal&g.y_column=bpm_count&g.y_type=quantitative), which is a bit hacky.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/767/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1536851861,I_kwDOBm6k_c5bmn-V,1994,Stuck on loading screen,10913053,jackhagley,open,0,,,,,1,2023-01-17T18:33:49Z,2023-01-23T08:21:08Z,,NONE,,"Can’t actually open it!

Downloaded today from the releases tab

Running macOS13.1 

```
bin/python3.9 --version
Python 3.9.6
Took 83ms
bin/python3.9 --version
Python 3.9.6
Took 113ms
bin/pip install datasette>=0.59 datasette-app-support>=0.11.6 datasette-vega>=0.6.2 datasette-cluster-map>=0.17.1 datasette-pretty-json>=0.2.1 datasette-edit-schema>=0.4 datasette-configure-fts>=1.1 datasette-leaflet>=0.2.2 --disable-pip-version-check
Requirement already satisfied: datasette>=0.59 in lib/python3.9/site-packages (0.63)
Requirement already satisfied: datasette-app-support>=0.11.6 in lib/python3.9/site-packages (0.11.6)
Requirement already satisfied: datasette-vega>=0.6.2 in lib/python3.9/site-packages (0.6.2)
Requirement already satisfied: datasette-cluster-map>=0.17.1 in lib/python3.9/site-packages (0.17.2)
Requirement already satisfied: datasette-pretty-json>=0.2.1 in lib/python3.9/site-packages (0.2.2)
Requirement already satisfied: datasette-edit-schema>=0.4 in lib/python3.9/site-packages (0.5.1)
Requirement already satisfied: datasette-configure-fts>=1.1 in lib/python3.9/site-packages (1.1)
Requirement already satisfied: datasette-leaflet>=0.2.2 in lib/python3.9/site-packages (0.2.2)
Requirement already satisfied: click>=7.1.1 in lib/python3.9/site-packages (from datasette>=0.59) (8.1.3)
Requirement already satisfied: hupper>=1.9 in lib/python3.9/site-packages (from datasette>=0.59) (1.10.3)
Requirement already satisfied: pint>=0.9 in lib/python3.9/site-packages (from datasette>=0.59) (0.20.1)
Requirement already satisfied: PyYAML>=5.3 in lib/python3.9/site-packages (from datasette>=0.59) (6.0)
Requirement already satisfied: httpx>=0.20 in lib/python3.9/site-packages (from datasette>=0.59) (0.23.0)
Requirement already satisfied: aiofiles>=0.4 in lib/python3.9/site-packages (from datasette>=0.59) (22.1.0)
Requirement already satisfied: asgi-csrf>=0.9 in lib/python3.9/site-packages (from datasette>=0.59) (0.9)
Requirement already satisfied: asgiref>=3.2.10 in lib/python3.9/site-packages (from datasette>=0.59) (3.5.2)
Requirement already satisfied: uvicorn>=0.11 in lib/python3.9/site-packages (from datasette>=0.59) (0.19.0)
Requirement already satisfied: itsdangerous>=1.1 in lib/python3.9/site-packages (from datasette>=0.59) (2.1.2)
Requirement already satisfied: click-default-group-wheel>=1.2.2 in lib/python3.9/site-packages (from datasette>=0.59) (1.2.2)
Requirement already satisfied: janus>=0.6.2 in lib/python3.9/site-packages (from datasette>=0.59) (1.0.0)
Requirement already satisfied: pluggy>=1.0 in lib/python3.9/site-packages (from datasette>=0.59) (1.0.0)
Requirement already satisfied: Jinja2>=2.10.3 in lib/python3.9/site-packages (from datasette>=0.59) (3.1.2)
Requirement already satisfied: mergedeep>=1.1.1 in lib/python3.9/site-packages (from datasette>=0.59) (1.3.4)
Requirement already satisfied: sqlite-utils in lib/python3.9/site-packages (from datasette-app-support>=0.11.6) (3.30)
Requirement already satisfied: packaging in lib/python3.9/site-packages (from datasette-app-support>=0.11.6) (21.3)
Requirement already satisfied: python-multipart in lib/python3.9/site-packages (from asgi-csrf>=0.9->datasette>=0.59) (0.0.5)
Requirement already satisfied: httpcore<0.16.0,>=0.15.0 in lib/python3.9/site-packages (from httpx>=0.20->datasette>=0.59) (0.15.0)
Requirement already satisfied: certifi in lib/python3.9/site-packages (from httpx>=0.20->datasette>=0.59) (2022.9.24)
Requirement already satisfied: rfc3986[idna2008]<2,>=1.3 in lib/python3.9/site-packages (from httpx>=0.20->datasette>=0.59) (1.5.0)
Requirement already satisfied: sniffio in lib/python3.9/site-packages (from httpx>=0.20->datasette>=0.59) (1.3.0)
Requirement already satisfied: h11<0.13,>=0.11 in lib/python3.9/site-packages (from httpcore<0.16.0,>=0.15.0->httpx>=0.20->datasette>=0.59) (0.12.0)
Requirement already satisfied: anyio==3.* in lib/python3.9/site-packages (from httpcore<0.16.0,>=0.15.0->httpx>=0.20->datasette>=0.59) (3.6.2)
Requirement already satisfied: idna>=2.8 in lib/python3.9/site-packages (from anyio==3.*->httpcore<0.16.0,>=0.15.0->httpx>=0.20->datasette>=0.59) (3.4)
Requirement already satisfied: typing-extensions>=3.7.4.3 in lib/python3.9/site-packages (from janus>=0.6.2->datasette>=0.59) (4.4.0)
Requirement already satisfied: MarkupSafe>=2.0 in lib/python3.9/site-packages (from Jinja2>=2.10.3->datasette>=0.59) (2.1.1)
Requirement already satisfied: tabulate in lib/python3.9/site-packages (from sqlite-utils->datasette-app-support>=0.11.6) (0.9.0)
Requirement already satisfied: python-dateutil in lib/python3.9/site-packages (from sqlite-utils->datasette-app-support>=0.11.6) (2.8.2)
Requirement already satisfied: sqlite-fts4 in lib/python3.9/site-packages (from sqlite-utils->datasette-app-support>=0.11.6) (1.0.3)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in lib/python3.9/site-packages (from packaging->datasette-app-support>=0.11.6) (3.0.9)
Requirement already satisfied: six>=1.5 in lib/python3.9/site-packages (from python-dateutil->sqlite-utils->datasette-app-support>=0.11.6) (1.16.0)
Took 784ms
```
STUCK",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1994/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
675727366,MDU6SXNzdWU2NzU3MjczNjY=,919,"Travis should not build the master branch, only the main branch",9599,simonw,closed,0,,,,,3,2020-08-09T16:18:25Z,2020-08-09T16:26:18Z,2020-08-09T16:19:37Z,OWNER,,"Caused by #849 - since we are mirroring the two branches (to ensure old links to `master` keep working) Travis is building both.

The following in `.travis.yml` should fix that:
```
branches:
  except:
  - master
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/919/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
516370822,MDU6SXNzdWU1MTYzNzA4MjI=,611,Static assets no longer loading for installed plugins,9599,simonw,closed,0,,,,,3,2019-11-01T22:07:00Z,2019-11-01T22:15:55Z,2019-11-01T22:15:55Z,OWNER,,"Caused by fix I made in #606

e.g. `/-/static-plugins/datasette_leaflet_geojson/datasette-leaflet-geojson.js` is a 404, but view-`/-/static-plugins/datasette-leaflet-geojson/datasette-leaflet-geojson.js` works correctly.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/611/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1841343173,I_kwDOBm6k_c5twKrF,2132,Get form fields on query page working again ,9599,simonw,closed,0,,,9700784,Datasette 1.0a3,1,2023-08-08T13:39:05Z,2023-08-08T13:45:10Z,2023-08-08T13:45:09Z,OWNER,,"Caused by:
- #2112

https://latest.datasette.io/fixtures?sql=select+pk1%2C+pk2%2C+pk3%2C+content+from+compound_three_primary_keys+where+%22pk1%22+%3D+%3Ap0+order+by+pk1%2C+pk2%2C+pk3+limit+101&p0=b

<img width=""1100"" alt=""image"" src=""https://github.com/simonw/datasette/assets/9599/7ca20120-923b-4b7d-988d-cdb0e7c30afb"">

The `:p0` form field is missing. Submitting the form results in this error:

<img width=""687"" alt=""image"" src=""https://github.com/simonw/datasette/assets/9599/f900ee16-71fd-42aa-86d7-0058af8457aa"">
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2132/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
275089535,MDU6SXNzdWUyNzUwODk1MzU=,121,?_json=foo&_json=bar query string argument ,9599,simonw,closed,0,,,,,4,2017-11-18T16:09:55Z,2018-05-31T13:48:12Z,2018-05-28T18:11:51Z,OWNER,,"Causes the specified columns in the output to be treated as JSON, and returned deserialized in the .json or .jsono response.

This will be particularly powerful when combined with https://sqlite.org/json1.html",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/121/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
925544070,MDU6SXNzdWU5MjU1NDQwNzA=,287,Update rowid examples in the docs,9599,simonw,closed,0,,,,,0,2021-06-20T08:03:00Z,2021-06-20T18:26:21Z,2021-06-20T18:26:21Z,OWNER,,Changed in #284 - a couple of examples need updating on https://github.com/simonw/sqlite-utils/blob/3.10/docs/cli.rst.,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/287/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
788447787,MDU6SXNzdWU3ODg0NDc3ODc=,1194,?_size= argument is not persisted by hidden form fields in the table filters,9599,simonw,closed,0,,,6346396,Datasette 0.54,3,2021-01-18T17:41:52Z,2021-01-25T03:10:23Z,2021-01-25T03:10:23Z,OWNER,,"Click ""Apply"" on https://covid-19.datasettes.com/covid/ny_times_us_counties?_size=1000&county__exact=San+Francisco&state__exact=California&_sort_desc=date#g.mark=line&g.x_column=date&g.x_type=temporal&g.y_column=cases&g.y_type=quantitative and the `?_size=1000` parameter from the URL will no longer apply on the reloaded page.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1194/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
904071938,MDU6SXNzdWU5MDQwNzE5Mzg=,1345,?_nocol= does not interact well with default facets,9599,simonw,closed,0,,,,,7,2021-05-27T18:39:55Z,2021-05-31T02:40:44Z,2021-05-31T02:31:21Z,OWNER,,"Clicking ""Hide this column"" on `fips` on https://covid-19.datasettes.com/covid/ny_times_us_counties shows this error:

https://covid-19.datasettes.com/covid/ny_times_us_counties?_nocol=fips

> ## Invalid SQL
> no such column: fips

The reason is that https://covid-19.datasettes.com/-/metadata sets up the following:

```json
  ""ny_times_us_counties"": {
      ""sort_desc"": ""date"",
      ""facets"": [
          ""state"",
          ""county"",
          ""fips""
      ],
```
It's setting `fips` as a default facet, which breaks if you attempt to remove the column using `?_nocol`.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1345/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
430103450,MDU6SXNzdWU0MzAxMDM0NTA=,425,Submitting SQL on hide page is broken,9599,simonw,closed,0,,,,,2,2019-04-07T04:21:31Z,2019-04-12T05:12:13Z,2019-04-12T05:00:53Z,OWNER,,Clicking the submit button here doesn't work correctly: https://3a208a4.datasette.io/fixtures?sql=select+%2A+from+compound_three_primary_keys+order+by+pk1%2C+pk2%2C+pk3+limit+101&_hide_sql=1,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/425/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1469796454,I_kwDOBm6k_c5Xm1Bm,1920,Document Datasette.metadata() method,25778,eyeseast,open,0,,,,,0,2022-11-30T15:10:36Z,2022-11-30T15:10:36Z,,CONTRIBUTOR,,"Code is here: https://github.com/simonw/datasette/blob/main/datasette/app.py#L503

This will be the official way to access metadata from plugins.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1920/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
727848625,MDU6SXNzdWU3Mjc4NDg2MjU=,12,"Some workout columns should be float, not text",9599,simonw,open,0,,,,,4,2020-10-23T02:47:02Z,2022-06-23T04:35:02Z,,MEMBER,,"Columns `duration`, `totalDistance` and `totalEnergyBurned` should be converted to float.

https://github.com/dogsheep/healthkit-to-sqlite/blob/71e36e1cf034b96de2a8e6652265d782d3fdf63b/healthkit_to_sqlite/utils.py#L50-L57",197882382,healthkit-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/12/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
657747959,MDU6SXNzdWU2NTc3NDc5NTk=,895,SQL query output should show numeric values in a different colour,9599,simonw,closed,0,,,,,1,2020-07-16T00:28:03Z,2020-09-15T20:40:08Z,2020-09-15T20:40:08Z,OWNER,,"Compare https://latest.datasette.io/fixtures/sortable with https://latest.datasette.io/fixtures?sql=select+pk1%2C+pk2%2C+content%2C+sortable%2C+sortable_with_nulls%2C+sortable_with_nulls_2%2C+text+from+sortable+order+by+pk1%2C+pk2+limit+101

<img width=""1574"" alt=""fixtures__select_pk1__pk2__content__sortable__sortable_with_nulls__sortable_with_nulls_2__text_from_sortable_order_by_pk1__pk2_limit_101_and_fixtures__sortable__201_rows"" src=""https://user-images.githubusercontent.com/9599/87612845-82e09c00-c6c0-11ea-806e-93764ca468c4.png"">
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/895/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1122427321,I_kwDOBm6k_c5C5uG5,1624,Index page `/` has no CORS headers,9599,simonw,open,0,,,,,2,2022-02-02T21:56:10Z,2022-09-28T16:54:22Z,,OWNER,,"Compare the following:
```
% curl -I 'https://latest.datasette.io/fixtures'
HTTP/1.1 200 OK
link: https://latest.datasette.io/fixtures.json; rel=""alternate""; type=""application/json+datasette""
cache-control: max-age=5
referrer-policy: no-referrer
access-control-allow-origin: *
access-control-allow-headers: Authorization
access-control-expose-headers: Link
content-type: text/html; charset=utf-8
x-databases: _memory, _internal, fixtures, extra_database
Date: Wed, 02 Feb 2022 21:55:49 GMT
Server: Google Frontend
Transfer-Encoding: chunked

% curl -I 'https://latest.datasette.io/'        
HTTP/1.1 200 OK
link: https://latest.datasette.io/.json; rel=""alternate""; type=""application/json+datasette""
content-type: text/html; charset=utf-8
x-databases: _memory, _internal, fixtures, extra_database
Date: Wed, 02 Feb 2022 21:55:52 GMT
Server: Google Frontend
Transfer-Encoding: chunked
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1624/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
309558826,MDU6SXNzdWUzMDk1NTg4MjY=,190,Keyset pagination doesn't work correctly for compound primary keys,9599,simonw,closed,0,,,,,7,2018-03-28T22:45:06Z,2018-03-30T06:31:15Z,2018-03-30T06:26:28Z,OWNER,,"Consider https://datasette-issue-190-compound-pks.now.sh/compound-pks-9aafe8f/compound_primary_key

![2018-03-28 at 3 47 pm](https://user-images.githubusercontent.com/9599/38060388-56da86dc-329f-11e8-9f20-5576153ad55c.png)

The next= link is to `d,v`:

https://datasette-issue-190-compound-pks.now.sh/compound-pks-9aafe8f/compound_primary_key?_next=d%2Cv

But that page starts with:

![2018-03-28 at 3 48 pm](https://user-images.githubusercontent.com/9599/38060402-6b0f5984-329f-11e8-85b8-44a666c4ee71.png)

The next key in the sequence should be `d,w`. Also we should return the full a-z of the ones that start with the letter e - in this example we only return `e-w`, `e-x`, `e-y` and `e-z`",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/190/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
448977444,MDU6SXNzdWU0NDg5Nzc0NDQ=,489,Pagination breaks when combined with expanded foreign keys,9599,simonw,closed,0,,,,,1,2019-05-27T19:56:56Z,2019-05-28T02:48:57Z,2019-05-28T02:23:27Z,OWNER,,"Consider https://edb3662.datasette.io/fixtures/roadside_attraction_characteristics?_sort=attraction_id&_size=2

The ""Next page"" link goes here, which returns 0 rows:

https://edb3662.datasette.io/fixtures/roadside_attraction_characteristics?_size=2&_next=%257B%2527value%2527%253A%2B2%252C%2B%2527label%2527%253A%2B%2527Winchester%2BMystery%2BHouse%2527%257D%2C2&_sort=attraction_id

That's because if you double-url-decode that `_next` link you get this:

`_next={'value': 2, 'label': 'Winchester Mystery House'},2`

It should be `_next=2,2`",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/489/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
525993034,MDU6SXNzdWU1MjU5OTMwMzQ=,637,"Custom queries with 0 results should say ""0 results""",9599,simonw,closed,0,,,,,3,2019-11-20T18:28:14Z,2019-11-23T06:17:23Z,2019-11-23T06:07:08Z,OWNER,,"Consider https://latest.datasette.io/fixtures/neighborhood_search?text=foop

<img width=""803"" alt=""fixtures__select_neighborhood__facet_cities_name__state_from_facetable_join_facet_cities_on_facetable_city_id___facet_cities_id_where_neighborhood_like_________text________order_by_neighborhood_"" src=""https://user-images.githubusercontent.com/9599/69266652-6a939e00-0b80-11ea-8b08-960a05a8c8d0.png"">

It's currently not obvious that the query executed and returned 0 results.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/637/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
784628163,MDU6SXNzdWU3ODQ2MjgxNjM=,1185,"""Statement may not contain PRAGMA"" error is not strictly true",9599,simonw,closed,0,,,6346396,Datasette 0.54,3,2021-01-12T22:07:10Z,2021-01-24T21:21:37Z,2021-01-12T22:26:26Z,OWNER,,"Consider https://latest.datasette.io/fixtures?sql=select+%27select%0D%0A%27+%7C%7C+group_concat%28%27++++case+when+%5B%27+%7C%7C+name+%7C%7C+%27%5D+is+not+null+then+%27+%7C%7C+quote%28name+%7C%7C+%27%2C+%27%29+%7C%7C+%27+else+%27%27%27%27+end%27%2C+%27+%7C%7C%0D%0A%27%29+%7C%7C+%27%0D%0A++as+columns%2C%0D%0A++count%28*%29+as+num_rows%0D%0Afrom%0D%0A++%5B%27+%7C%7C+%3Atable+%7C%7C+%27%5D%0D%0Agroup+by%0D%0A++columns%0D%0Aorder+by%0D%0A++num_rows+desc%27+as+query+from+pragma_ytable_info%28%3Atable%29&table=facetable

It says ""Statement may not contain PRAGMA"" - but that's not actually true. Datasette has an allow-list of PRAGMA that are OK - in this case there was a typo in `pragma_ytable_info` which caused the error, but pragma_table_info` would have been OK.

So the error message is misleading.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1185/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
327459829,MDU6SXNzdWUzMjc0NTk4Mjk=,298,URLify URLs in results from custom SQL statements / views,9599,simonw,closed,0,,,,,2,2018-05-29T19:41:07Z,2018-07-24T04:53:20Z,2018-07-24T03:56:50Z,OWNER,,"Consider this custom query:

https://fivethirtyeight.datasettes.com/fivethirtyeight-5de27e3?sql=select+user%2C+%28%27https%3A%2F%2Ftwitter.com%2F%27+%7C%7C+user%29+as+user_url%2C+created_at%2C+text%2C+url+from+%5Btwitter-ratio%2Fsenators%5D+limit+10%3B

```select user, ('https://twitter.com/' || user) as user_url, created_at, text, url from [twitter-ratio/senators] limit 10;```

![2018-05-29 at 12 38 pm](https://user-images.githubusercontent.com/9599/40681177-44a36d5c-633d-11e8-935b-c49dad7ac682.png)

It would be nice if these URLs were turned into links, as happens on the table view page: https://fivethirtyeight.datasettes.com/fivethirtyeight-5de27e3/twitter-ratio%2Fsenators

![2018-05-29 at 12 39 pm](https://user-images.githubusercontent.com/9599/40681206-5c69c47c-633d-11e8-9f3a-08899f8659b8.png)

This currently does not happen because the table view render logic takes a different path through `display_columns_and_rows()` which includes this bit:

https://github.com/simonw/datasette/blob/b0a95da96386ddf99816911e08df86178ffa9a89/datasette/views/table.py#L195-L202",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/298/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1497577017,I_kwDOBm6k_c5ZQzY5,1957,Reconsider row value truncation on query page,9599,simonw,open,0,,,,,1,2022-12-14T23:49:47Z,2022-12-14T23:50:50Z,,OWNER,,"Consider this example: https://ripgrep.datasette.io/repos?sql=select+json_group_array%28full_name%29+from+repos

```sql
select json_group_array(full_name) from repos
```

![CleanShot 2022-12-14 at 15 48 32@2x](https://user-images.githubusercontent.com/9599/207739709-8177f683-f938-49a1-8225-42791fad88fe.png)

My intention here was to get a string of JSON I can copy and paste elsewhere - see: https://til.simonwillison.net/sqlite/compare-before-after-json

The truncation isn't helping here.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1957/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1055469073,I_kwDOBm6k_c4-6S4R,1513,Research: CTEs and union all to calculate facets AND query at the same time,9599,simonw,closed,0,,,,,12,2021-11-16T22:26:45Z,2021-11-16T23:41:46Z,2021-11-16T23:41:46Z,OWNER,,"Consider this page: https://global-power-plants.datasettes.com/global-power-plants/global-power-plants?_search=plant&_facet=owner&_facet=country_long&_facet=primary_fuel

Datasette needs to run the main query for the rows on that page, a count query for the total query, then a separate query for each of those three specified facets.

This is a `_search=` query, so it needs to execute the FTS code once for the rows, again for the count, and then three more times for each of the facets.

Could running that query as a CTE and doing the other queries as part of the same large query produce significant speed improvements?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1513/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
903986178,MDU6SXNzdWU5MDM5ODYxNzg=,1344,Test Datasette Docker images built for different architectures,9599,simonw,open,0,,,,,10,2021-05-27T16:52:29Z,2022-09-06T00:07:58Z,,OWNER,,"Continuing on from #1319 - now that we have the ability to build Datasette's Docker image against multiple architectures we should test that it works.

We can do this with QEMU emulation, see https://twitter.com/nevali/status/1397958044571602945",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1344/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1421544654,I_kwDOBm6k_c5UuwzO,1851,API to insert a single record into an existing table,9599,simonw,closed,0,,,8658075,Datasette 1.0a0,22,2022-10-24T22:24:21Z,2022-11-15T19:59:18Z,2022-10-28T00:59:25Z,OWNER,,Controlled by a new `insert-row` permission.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1851/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
314665147,MDU6SXNzdWUzMTQ2NjUxNDc=,216,Bug: Sort by column with NULL in next_page URL,222245,carlmjohnson,closed,0,,,,,15,2018-04-16T14:03:18Z,2018-04-17T01:45:24Z,2018-04-17T01:45:24Z,NONE,,"Copy-pasting from https://github.com/simonw/datasette/issues/189#issuecomment-381429213, since that issue is closed:

I think I found a bug. I tried to sort by middle initial in my salaries set, and many middle initials are null. The `next_url` gets set by Datasette to:

http://localhost:8001/salaries-d3a5631/2017+Maryland+state+salaries?_next=None%2C391&_sort=middle_initial

But then None is interpreted literally and it tries to find a name with the middle initial ""None"" and ends up skipping ahead to O on page 2.
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/216/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
628156527,MDU6SXNzdWU2MjgxNTY1Mjc=,789,Mechanism for enabling pluggy tracing,9599,simonw,open,0,,,,,2,2020-06-01T05:10:14Z,2020-06-01T05:11:03Z,,OWNER,,"Could be useful for debugging plugins: https://pluggy.readthedocs.io/en/latest/#call-tracing

I tried this out by adding these two lines in `plugins.py`:
```python
pm = pluggy.PluginManager(""datasette"")
pm.add_hookspecs(hookspecs)
# Added these:
pm.trace.root.setwriter(print)
pm.enable_tracing()
```
Output looked something like this:
```
INFO:     127.0.0.1:52724 - ""GET /-/-/static/app.css HTTP/1.1"" 404 Not Found
  actor_from_request [hook]
      datasette: <datasette.app.Datasette object at 0x106277ad0>
      request: <datasette.utils.asgi.Request object at 0x106550a50>

  finish actor_from_request --> [] [hook]

  extra_body_script [hook]
      template: show_json.html
      database: None
      table: None
      view_name: json_data
      datasette: <datasette.app.Datasette object at 0x106277ad0>

  finish extra_body_script --> [] [hook]

  extra_template_vars [hook]
      template: show_json.html
      database: None
      table: None
      view_name: json_data
      request: <datasette.utils.asgi.Request object at 0x1065504d0>
      datasette: <datasette.app.Datasette object at 0x106277ad0>

  finish extra_template_vars --> [] [hook]

  extra_css_urls [hook]
      template: show_json.html
      database: None
      table: None
      datasette: <datasette.app.Datasette object at 0x106277ad0>

  finish extra_css_urls --> [] [hook]

  extra_js_urls [hook]
      template: show_json.html
      database: None
      table: None
      datasette: <datasette.app.Datasette object at 0x106277ad0>

  finish extra_js_urls --> [] [hook]

INFO:     127.0.0.1:52724 - ""GET /-/actor HTTP/1.1"" 200 OK
  actor_from_request [hook]
      datasette: <datasette.app.Datasette object at 0x106277ad0>
      request: <datasette.utils.asgi.Request object at 0x1065500d0>

  finish actor_from_request --> [] [hook]
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/789/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
351845423,MDU6SXNzdWUzNTE4NDU0MjM=,3,Experiment with contentless FTS tables,9599,simonw,closed,0,,,,,1,2018-08-18T19:31:01Z,2019-07-22T20:58:55Z,2019-07-22T20:58:55Z,OWNER,,Could greatly reduce size of resulting database for large datasets: http://cocoamine.net/blog/2015/09/07/contentless-fts4-for-large-immutable-documents/,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/3/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
681334912,MDU6SXNzdWU2ODEzMzQ5MTI=,942,Support column descriptions in metadata.json,9599,simonw,closed,0,,,,,18,2020-08-18T20:52:00Z,2022-01-13T22:21:42Z,2021-08-12T23:53:24Z,OWNER,,"Could look something like this:
```json
{
    ""title"": ""Five Thirty Eight"",
    ""license"": ""CC Attribution 4.0 License"",
    ""license_url"": ""https://creativecommons.org/licenses/by/4.0/"",
    ""source"": ""fivethirtyeight/data on GitHub"",
    ""source_url"": ""https://github.com/fivethirtyeight/data"",
    ""databases"": {
        ""fivethirtyeight"": {
            ""tables"": {
                ""mueller-polls/mueller-approval-polls"": {
                    ""description_html"": ""<p>....</p>"",
                    ""columns"": {
                        ""name_of_column"": ""column_description goes here""
}
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/942/reactions"", ""total_count"": 4, ""+1"": 4, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1359604075,I_kwDOCGYnMM5RCelr,481,"Idea: `sqlite-utils create-table tablename --sql ""select ...""`",9599,simonw,open,0,,,,,0,2022-09-02T01:41:24Z,2022-09-02T01:42:08Z,,OWNER,,"Could offer syntactic sugar for:

```sql
create table foo as select * from bar
```

```
sqlite-utils create-table data.db foo --sql ""select * from bar""
```
https://sqlite-utils.datasette.io/en/stable/cli-reference.html#create-table",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/481/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1076057610,I_kwDOBm6k_c5AI1YK,1546,validating the sql,50336793,jadsongmatos,closed,0,,,,,1,2021-12-09T21:35:57Z,2021-12-18T02:05:17Z,2021-12-18T02:05:16Z,NONE,,Could someone tell me that part of the code is responsible for validating the sql that guarantees that only a table can be read,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1546/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
275755475,MDU6SXNzdWUyNzU3NTU0NzU=,140,Heatmap visualization plugin,9599,simonw,open,0,,,,,2,2017-11-21T15:34:23Z,2019-05-13T18:33:51Z,,OWNER,,Could use https://github.com/scottbedard/svelte-heatmap,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/140/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
777392020,MDU6SXNzdWU3NzczOTIwMjA=,212,Mechanism for maintaining cache of table counts using triggers,9599,simonw,closed,0,,,,,1,2021-01-02T02:58:53Z,2021-01-02T21:40:27Z,2021-01-02T21:40:27Z,OWNER,,"Counting all of the rows in a large table is expensive - this is one of the main causes of performance problems in Datasette when running against large databases.

Carefully constructed SQL triggers could be used to maintain accurate cached counts for a table, by incrementing and decrementing a counter every time a row is inserted or deleted.

`sqlite-utils` already has a mechanism for creating triggers for FTS - the `table.enable_fts(..., create_triggers=True)` method. How about a similar mechanism for setting up triggers to maintain a count of table rows?",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/212/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
837348479,MDU6SXNzdWU4MzczNDg0Nzk=,1269,Don't attempt to run count(*) against virtual tables,9599,simonw,closed,0,,,,,2,2021-03-22T05:57:43Z,2021-03-22T17:40:42Z,2021-03-22T17:40:41Z,OWNER,,"Counting the rows in a virtual table doesn't seem very interesting to me, and it's the cause of at least one crashing bug with SpatiaLite 5.0 on Linux, see https://github.com/simonw/datasette/issues/1268",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1269/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
520508502,MDU6SXNzdWU1MjA1MDg1MDI=,31,"""friends"" command (similar to ""followers"")",9599,simonw,closed,0,,,,,2,2019-11-09T20:20:20Z,2022-09-20T05:05:03Z,2020-02-07T07:03:28Z,MEMBER,,"Current list of commands:
```
  followers          Save followers for specified user (defaults to...
  followers-ids      Populate followers table with IDs of account followers
  friends-ids        Populate followers table with IDs of account friends
```
Obvious omission here is `friends`, which would be powered by `https://api.twitter.com/1.1/friends/list.json`: https://developer.twitter.com/en/docs/accounts-and-users/follow-search-get-users/api-reference/get-friends-list",206156866,twitter-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/31/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
634844634,MDU6SXNzdWU2MzQ4NDQ2MzQ=,817,Drop resource_type from permission_allowed system,9599,simonw,closed,0,,,,,1,2020-06-08T18:41:37Z,2020-06-08T19:00:12Z,2020-06-08T19:00:12Z,OWNER,,"Current signature:

    permission_allowed(datasette, actor, action, resource_type, resource_identifier)

It turns out the `resource_type` is always the same thing for any given action, so it's not actually useful. I'm going to drop it.

New signature will be:

    permission_allowed(datasette, actor, action, resource)

Refs #811.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/817/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
487721884,MDU6SXNzdWU0ODc3MjE4ODQ=,5,Treat Foursquare timestamps as UTC,9599,simonw,closed,0,,,,,0,2019-08-31T02:44:47Z,2019-08-31T02:50:41Z,2019-08-31T02:50:41Z,MEMBER,,"Current test failure is due to timezone differences between my laptop and Circle CI:

https://circleci.com/gh/dogsheep/swarm-to-sqlite/3

```
E         Full diff:
E         - [{'created': '2018-07-01T04:48:19',
E         ?                           ^
E         + [{'created': '2018-07-01T02:48:19',
E         ?                           ^
E         'createdAt': 1530413299,
```

The timestamps I store in `created` should always be UTC.",205429375,swarm-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/5/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
585850715,MDU6SXNzdWU1ODU4NTA3MTU=,19,"Enable full-text search for more stuff (like commits, issues and issue_comments)",9599,simonw,closed,0,,,5225818,1.0,2,2020-03-23T00:19:56Z,2020-03-23T19:06:39Z,2020-03-23T19:06:39Z,MEMBER,,Currently FTS is only enabled for repos and releases.,207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/19/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
278814220,MDU6SXNzdWUyNzg4MTQyMjA=,161,Support WITH query ,388154,wsxiaoys,closed,0,,,,,4,2017-12-03T20:00:40Z,2017-12-08T06:18:12Z,2017-12-04T04:52:41Z,NONE,,"Currently datasettle failed with error message: Statement must begin with SELECT

Example query
```sql
WITH RECURSIVE
  cnt(x) AS (
     SELECT 1
     UNION ALL
     SELECT x+1 FROM cnt
      LIMIT 1000000
  )
SELECT x FROM cnt;
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/161/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
506087267,MDU6SXNzdWU1MDYwODcyNjc=,19,since_id support for home-timeline,9599,simonw,closed,0,,,,,3,2019-10-11T22:48:24Z,2019-10-16T19:13:06Z,2019-10-16T19:12:46Z,MEMBER,,Currently every time you run `home-timeline` we pull all 800 available tweets. We should offer to support `since_id` (which can be provided or can be pulled directly from the database) in order to work more efficiently if this command is executed e.g. on a cron.,206156866,twitter-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/19/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1060631257,I_kwDOBm6k_c4_N_LZ,1528,"Add new `""sql_file""` key to Canned Queries in metadata?",15178711,asg017,open,0,,,,,3,2021-11-22T21:58:01Z,2022-06-10T03:23:08Z,,CONTRIBUTOR,,"Currently for canned queries, you have to inline SQL in your `metadata.yaml` like so:

```yaml
databases:
  fixtures:
    queries:
      neighborhood_search:
        sql: |-
          select neighborhood, facet_cities.name, state
          from facetable
            join facet_cities on facetable.city_id = facet_cities.id
          where neighborhood like '%' || :text || '%'
          order by neighborhood
        title: Search neighborhoods
```

This works fine, but for a few reasons, I usually have my canned queries already written in separate `.sql` files. I'd like to instead re-use those instead of re-writing it. 

So, I'd like to see a new `""sql_file""` key that works like so:

`metadata.yaml`:

```yaml
databases:
  fixtures:
    queries:
      neighborhood_search:
        sql_file: neighborhood_search.sql
        title: Search neighborhoods
```
`neighborhood_search.sql`:
```sql
select neighborhood, facet_cities.name, state
from facetable
join facet_cities on facetable.city_id = facet_cities.id
where neighborhood like '%' || :text || '%'
order by neighborhood
```

Both of these would work in the exact same way, where Datasette would instead open + include `neighborhood_search.sql` on startup. 


A few reasons why I'd like to keep my canned queries SQL separate from metadata.yaml:

- Keeping SQL in standalone SQL files means syntax highlighting and other text editor integrations in my code
- Multiline strings in yaml, while functional, are a tad cumbersome and are hard to edit
- Works well with other tools (can pipe `.sql` files into the `sqlite3` CLI, or use with other SQLite clients easier)
- Typically my canned queries are quite long compared to everything else in my metadata.yaml, so I'd love to separate it where possible

Let me know if this is a feature you'd like to see, I can try to send up a PR if this sounds right!",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1528/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
406055201,MDU6SXNzdWU0MDYwNTUyMDE=,406,Support nullable foreign keys in _labels mode,9599,simonw,closed,0,9599,simonw,,,2,2019-02-03T05:34:20Z,2019-11-02T22:39:28Z,2019-11-02T22:30:27Z,OWNER,,"Currently if there's a null in a foreign key we get ""None"" displayed in the inflated view:

<img width=""412"" alt=""screen shot 2019-02-02 at 9 33 37 pm"" src=""https://user-images.githubusercontent.com/9599/52173123-46221e80-2732-11e9-8dcb-f6f4bc768c33.png"">
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/406/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1486036269,I_kwDOBm6k_c5Ykx0t,1941,Mechanism for supporting key rotation for DATASETTE_SECRET,9599,simonw,open,0,,,,,1,2022-12-09T05:24:53Z,2022-12-09T05:25:20Z,,OWNER,,"Currently if you change `DATASETTE_SECRET` all existing signed tokens - both cookies and API tokens and potentially other things too - will instantly expire.

Adding support for key rotation would allow keys to be rotated on a semi-regular basis without logging everyone out / invalidating every API token instantly.

Can model this on how Django does it: https://github.com/django/django/commit/0dcd549bbe36c060f536ec270d34d9e7d4b8e6c7",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1941/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1823428714,I_kwDOBm6k_c5sr1Bq,2120,Add __all__ to datasette/__init__.py,9599,simonw,open,0,,,,,0,2023-07-27T01:07:10Z,2023-07-27T01:07:10Z,,OWNER,,"Currently looks like this: https://github.com/simonw/datasette/blob/08181823990a71ffa5a1b57b37259198eaa43e06/datasette/__init__.py#L1-L6

Adding `__all__ = [""Permission"", ""Forbidden""...]` would let me get rid of those `# noqa` comments.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2120/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
407174173,MDU6SXNzdWU0MDcxNzQxNzM=,408,"Show metadata info (e.g. license, source) on custom SQL query pages",78356,stefanw,closed,0,,,,,0,2019-02-06T10:43:34Z,2019-10-14T03:53:22Z,2019-10-14T03:53:22Z,NONE,,"Currently metadata info is not displayed on custom SQL pages.

E.g. compare the footer of [this normal table page](https://register-of-members-interests.datasettes.com/regmem-98dc8b7/categories) with the footer [this custom SQL page](https://register-of-members-interests.datasettes.com/regmem-98dc8b7?sql=select+*+from+categories).

This is important in order to adhere to attribution license requirements.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/408/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
953352015,MDU6SXNzdWU5NTMzNTIwMTU=,1404,`register_routes()` hook should take `datasette` argument,9599,simonw,closed,0,,,,,1,2021-07-26T23:00:33Z,2021-07-26T23:27:07Z,2021-07-26T23:26:00Z,OWNER,,Currently that plugin hook takes no arguments at all. This means it's not possible to conditionally register routes based on Datasette plugin configuration.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1404/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1726531350,I_kwDOBm6k_c5m6McW,2079,Datasette should serve Access-Control-Max-Age,9599,simonw,closed,0,,,,,8,2023-05-25T21:50:50Z,2023-05-25T22:56:28Z,2023-05-25T22:08:35Z,OWNER,,"Currently the CORS headers served are:

https://github.com/simonw/datasette/blob/9584879534ff0556e04e4c420262972884cac87b/datasette/utils/__init__.py#L1139-L1143

Serving `Access-Control-Max-Age: 600` would allow browsers to cache that for 10 minutes, avoiding additional CORS pre-flight OPTIONS requests during that time.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2079/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
603624862,MDU6SXNzdWU2MDM2MjQ4NjI=,31,Issue and milestone should have foreign key to repo,9599,simonw,closed,0,,,,,3,2020-04-21T00:46:24Z,2020-04-22T01:20:19Z,2020-04-22T01:20:19Z,MEMBER,,"Currently the `repo` column on those tables is a string `simonw/datasette` rather than an ID referencing a row in `repos`.

_Originally posted by @simonw in https://github.com/dogsheep/github-to-sqlite/issues/29#issuecomment-616883275_",207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/31/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1816997390,I_kwDOCGYnMM5sTS4O,576,Backfill the release notes prior to 0.4,9599,simonw,closed,0,,,,,2,2023-07-23T05:41:42Z,2023-07-23T05:49:51Z,2023-07-23T05:48:21Z,OWNER,,"Currently the changelog starts at 0.4:

https://sqlite-utils.datasette.io/en/3.34/changelog.html#id115

I want the other releases - according to https://pypi.org/project/sqlite-utils/#history there are three missing:

<img width=""663"" alt=""image"" src=""https://github.com/simonw/sqlite-utils/assets/9599/4ebc036b-7bb1-477c-95c1-a2c7e26bcb62"">
",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/576/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
316526433,MDU6SXNzdWUzMTY1MjY0MzM=,234,label_column option in metadata.json,9599,simonw,closed,0,,,,,3,2018-04-21T21:19:08Z,2018-04-22T20:47:12Z,2018-04-22T20:47:12Z,OWNER,,"Currently the column used for displaying a foreign key relationship is automatically detected by `inspect()` by looking for tables that have a primary key column and one other column.

This doesn't work for tables with more than two columns.

Let's allow the table section in `metadata.json` to optionally define a `label_column` which, if present, will be used for those displays.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/234/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1879214365,I_kwDOCGYnMM5wAokd,590,Ability to tell if a Database is an in-memory one,9599,simonw,open,0,,,,,1,2023-09-03T19:50:15Z,2023-09-03T19:50:36Z,,OWNER,,"Currently the constructor accepts `memory=True` or `memory_name=...` and uses those to create a connection, but does not record what those values were:

https://github.com/simonw/sqlite-utils/blob/1260bdc7bfe31c36c272572c6389125f8de6ef71/sqlite_utils/db.py#L307-L349

This makes it hard to tell if a database object is to an in-memory or a file-based database, which is sometimes useful to know.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/590/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
724878151,MDU6SXNzdWU3MjQ4NzgxNTE=,1032,Bring date parsing into Datasette core,9599,simonw,open,0,,,,,8,2020-10-19T18:30:45Z,2020-10-19T19:37:55Z,,OWNER,,"Currently this is mainly handled by a plugin - https://github.com/simonw/datasette-dateutil - but I realise now that this really needs to be core functionality.

See also Twitter thread: https://twitter.com/simonw/status/1318234808653213696",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1032/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
808843401,MDU6SXNzdWU4MDg4NDM0MDE=,1226,--port option should validate port is between 0 and 65535,9599,simonw,closed,0,,,,,4,2021-02-15T22:01:33Z,2021-02-18T18:41:27Z,2021-02-18T18:41:27Z,OWNER,,"Currently throws an ugly error message:
```
(datasette-graphql) datasette-graphql % datasette fivethirtyeight.db -p 80094
INFO:     Started server process [45497]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
Traceback (most recent call last):
  File ""/Users/simon/.local/share/virtualenvs/datasette-graphql-n1OSJCS8/bin/datasette"", line 8, in <module>
    sys.exit(cli())
...
    server = await loop.create_server(
  File ""/Users/simon/.pyenv/versions/3.8.2/lib/python3.8/asyncio/base_events.py"", line 1461, in create_server
    sock.bind(sa)
OverflowError: bind(): port must be 0-65535.
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1226/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1114557284,I_kwDOCGYnMM5Cbstk,390,`sqlite-utils upsert` should require `--pk` more elegantly,9599,simonw,closed,0,,,,,1,2022-01-26T02:20:31Z,2022-01-26T03:20:25Z,2022-01-26T03:19:43Z,OWNER,,"Currently throws an ugly traceback:
```
% echo '[
            {""id"": 1, ""name"": ""Lila""},
            {""id"": 1, ""name"": ""Lila""}
        ]' | sqlite-utils upsert data.db chickens -       
Traceback (most recent call last):
  File ""/Users/simon/.local/share/virtualenvs/sqlite-utils-C4Ilevlm/bin/sqlite-utils"", line 33, in <module>
    sys.exit(load_entry_point('sqlite-utils', 'console_scripts', 'sqlite-utils')())
  File ""/Users/simon/.local/share/virtualenvs/sqlite-utils-C4Ilevlm/lib/python3.8/site-packages/click/core.py"", line 1137, in __call__
    return self.main(*args, **kwargs)
  File ""/Users/simon/.local/share/virtualenvs/sqlite-utils-C4Ilevlm/lib/python3.8/site-packages/click/core.py"", line 1062, in main
    rv = self.invoke(ctx)
  File ""/Users/simon/.local/share/virtualenvs/sqlite-utils-C4Ilevlm/lib/python3.8/site-packages/click/core.py"", line 1668, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File ""/Users/simon/.local/share/virtualenvs/sqlite-utils-C4Ilevlm/lib/python3.8/site-packages/click/core.py"", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File ""/Users/simon/.local/share/virtualenvs/sqlite-utils-C4Ilevlm/lib/python3.8/site-packages/click/core.py"", line 763, in invoke
    return __callback(*args, **kwargs)
  File ""/Users/simon/Dropbox/Development/sqlite-utils/sqlite_utils/cli.py"", line 1104, in upsert
    insert_upsert_implementation(
  File ""/Users/simon/Dropbox/Development/sqlite-utils/sqlite_utils/cli.py"", line 906, in insert_upsert_implementation
    db[table].insert_all(
  File ""/Users/simon/Dropbox/Development/sqlite-utils/sqlite_utils/db.py"", line 2615, in insert_all
    raise PrimaryKeyRequired(""upsert() requires a pk"")
sqlite_utils.db.PrimaryKeyRequired: upsert() requires a pk
```",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/390/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1553425465,I_kwDOCGYnMM5cl2Q5,522,Add COLUMN_TYPE_MAPPING for timedelta,81377,maport,closed,0,,,,,0,2023-01-23T16:49:54Z,2023-11-04T00:49:51Z,2023-11-04T00:49:51Z,NONE,,"Currently trying to create a column with Python type `datetime.timedelta` results in an error:

```
>>> from sqlite_utils import Database
>>> db = Database(""test.db"")
>>> test_tbl = db['test']
>>> test_tbl.insert({'col1': datetime.timedelta()})
Traceback (most recent call last):
  File ""<stdin>"", line 1, in <module>
  File ""/usr/local/lib/python3.10/dist-packages/sqlite_utils/db.py"", line 2979, in insert
    return self.insert_all(
  File ""/usr/local/lib/python3.10/dist-packages/sqlite_utils/db.py"", line 3082, in insert_all
    self.create(
  File ""/usr/local/lib/python3.10/dist-packages/sqlite_utils/db.py"", line 1574, in create
    self.db.create_table(
  File ""/usr/local/lib/python3.10/dist-packages/sqlite_utils/db.py"", line 961, in create_table
    sql = self.create_table_sql(
  File ""/usr/local/lib/python3.10/dist-packages/sqlite_utils/db.py"", line 852, in create_table_sql
    column_type=COLUMN_TYPE_MAPPING[column_type],
KeyError: <class 'datetime.timedelta'>
```

The reason this would be useful is that `MySQLdb` uses `timedelta` for MySQL `TIME` columns:

```
>>> import MySQLdb
>>> conn = MySQLdb.connect(host='database', user='user', passwd='pw')
>>> csr = conn.cursor()
>>> csr.execute(""SELECT CAST('11:20' AS TIME)"")
>>> tuple(csr)
((datetime.timedelta(seconds=40800),),)
```

So currently any attempt to convert a MySQL DB with a `TIME` column using `db-to-sqlite` will result in the above error.

I was rather surprised that `MySQLdb` uses `timedelta` for `TIME` columns but I see that [this column type](https://dev.mysql.com/doc/refman/8.0/en/time.html) is intended for time intervals as well as the time of day so it makes sense. 

",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/522/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
811589344,MDU6SXNzdWU4MTE1ODkzNDQ=,1235,Upgrade Python version used by official Datasette Docker image,9599,simonw,closed,0,,,,,2,2021-02-19T00:47:40Z,2021-02-19T01:48:31Z,2021-02-19T01:48:30Z,OWNER,,"Currently uses 3.7.2:

https://github.com/simonw/datasette/blob/73bed175631a79e13a521eee82f8451dd0477eb3/Dockerfile#L1

There's a security fix for Python which it would be good to ship in this image (even though I'm reasonably confident it doesn't affect Datasette): https://bugs.python.org/issue42938",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1235/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1920416843,I_kwDOCGYnMM5ydzxL,597,sqlite-utils insert-files should be able to convert fields,1737541,grimnight,open,0,,,,,0,2023-09-30T22:20:47Z,2023-09-30T22:20:47Z,,NONE,,"Currently using both `insert-files` and `convert` is needed in order to create sqlar files, it would be more convenient if it could be done with just one command.

```shell
~
❯ cat test.py
import os

class Example:
    def __init__(self, arg1, arg2):
        self.arg1 = arg1

~
❯ sqlite-utils insert-files test.sqlar sqlar test.py -c name:name -c data:content -c mode:mode -c mtime:mtime -c sz:size --pk=name
  [####################################]  100%

~
❯ sqlite-utils convert test.sqlar sqlar data ""zlib.compress(value)"" --import=zlib --where ""name = 'test.py'""
[####################################]  100%

~
❯ cat test.py | sqlite-utils convert test.sqlar sqlar data ""zlib.compress(sys.stdin.buffer.read())"" --import=zlib --import=sys --where ""name = 'test.py'"" # Alternative way
  [####################################]  100%

~
❯ sqlite3 test.sqlar ""SELECT hex(data) FROM sqlar WHERE name = 'test.py';"" | python3 -c ""import sys, zlib; sys.stdout.buffer.write(zlib.decompress(bytes.fromhex(sys.stdin.read())))""
import os

class Example:
    def __init__(self, arg1, arg2):
        self.arg1 = arg1

~
❯ rm test.py

~
❯ sqlar -l test.sqlar
test.py

~
❯ sqlar -x test.sqlar

~
❯ cat test.py
import os

class Example:
    def __init__(self, arg1, arg2):
        self.arg1 = arg1

```",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/597/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
634112607,MDU6SXNzdWU2MzQxMTI2MDc=,812,Ability to customize what happens when a view permission fails,9599,simonw,closed,0,,,5533512,Datasette 0.45,3,2020-06-08T04:26:14Z,2020-07-01T04:17:46Z,2020-07-01T04:17:45Z,OWNER,,"Currently view permission failures raise a `Forbidden` error which is transformed into a 403.

It would be good if this page could offer a way forward - maybe just by linking to (or redirecting to) a login screen. This behaviour will vary based on authentication plugins, so a new plugin hook is probably the best way to do this.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/812/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
626211658,MDU6SXNzdWU2MjYyMTE2NTg=,778,Ability to configure keyset pagination for views and queries,9599,simonw,open,0,,,,,1,2020-05-28T04:48:56Z,2020-10-02T02:26:25Z,,OWNER,,"Currently views offer pagination, but it uses offset/limit - e.g. https://latest.datasette.io/fixtures/paginated_view?_next=100

This means pagination will perform poorly on deeper pages.

If a view is based on a table that has a primary key it should be possible to configure efficient keyset pagination that works the same way that table pagination works.

This may be as simple as configuring a column that can be treated as a ""primary key"" for the purpose of pagination using `metadata.json` - or with a `?_view_pk=colname` querystring argument.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/778/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
725996507,MDU6SXNzdWU3MjU5OTY1MDc=,1036,Make it possible to download BLOB data from the Datasette UI,9599,simonw,closed,0,,,6026070,0.51,16,2020-10-20T22:47:56Z,2021-01-18T17:45:00Z,2020-10-25T00:14:52Z,OWNER,,"Currently you can only extract binary BLOB data as base64-encoded JSON, which is not user friendly at all. It should always be possible for end-users to get the binary data out.

I'm worried about XSS vulnerabilities here, but hopefully sending `Content-Type: application/octet-stream` helps there? Need to research that.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1036/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
508024032,MDU6SXNzdWU1MDgwMjQwMzI=,22,Ability to import from uncompressed archive or from specific files,9599,simonw,closed,0,,,,,0,2019-10-16T18:31:57Z,2019-10-16T18:53:36Z,2019-10-16T18:53:36Z,MEMBER,,"Currently you can only import like this:

    $ twitter-to-sqlite import path-to-twitter.zip

It would be useful if you could import from a folder that was decompressed from that zip:

    $ twitter-to-sqlite import path-to-twitter/

AND from individual files within that folder - since that would allow you to e.g. selectively import certain files:

    $ twitter-to-sqlite import path-to-twitter/favorites.js path-to-twitter/tweets.js",206156866,twitter-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/22/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1097128334,I_kwDOCGYnMM5BZNmO,371,Support mutating row in `--convert` without returning it,9599,simonw,closed,0,,,7558727,3.21,6,2022-01-09T07:38:44Z,2022-01-10T19:27:30Z,2022-01-09T20:06:15Z,OWNER,,"Currently you have to do this:
```
$ sqlite-utils insert dogs.db dogs dogs.json --convert '
row[""is_good""] = 1
return row'
```
Would be neat if this worked too:
```
$ sqlite-utils insert dogs.db dogs dogs.json \
  --convert 'row[""is_good""] = 1'
```",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/371/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
726094754,MDU6SXNzdWU3MjYwOTQ3NTQ=,1037,Add horizontal scrollbar to tables,9599,simonw,closed,0,,,6026070,0.51,3,2020-10-21T03:13:34Z,2020-10-27T20:52:04Z,2020-10-21T03:16:36Z,OWNER,,Currently you have to scroll the entire page sideways if a table is wide. Make the table `overflow-x: auto` instead.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1037/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1123903919,I_kwDOCGYnMM5C_Wmv,397,Support IF NOT EXISTS for table creation,738408,rafguns,closed,0,,,,,3,2022-02-04T07:41:15Z,2022-02-06T01:30:46Z,2022-02-06T01:29:01Z,NONE,,"Currently, I have a bunch of code that looks like this:

```python
subjects = db[""subjects""] if db[""subjects""].exists() else db[""subjects""].create({
    ...
})
```
It would be neat if sqlite-utils could simplify that by supporting `CREATE TABLE IF NOT EXISTS`, so that I'd be able to write, e.g.

```python
subjects = db[""subjects""].create({...}, if_not_exists=True)
```",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/397/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1118585417,I_kwDOCGYnMM5CrEJJ,393,Better documentation for insert-replace,9599,simonw,closed,0,,,,,1,2022-01-30T15:40:23Z,2022-02-03T22:13:24Z,2022-02-03T22:13:24Z,OWNER,,"Currently: https://sqlite-utils.datasette.io/en/stable/python-api.html#insert-replacing-data

> If you want to insert a record or replace an existing record with the same primary key, using the replace=True argument to .insert() or .insert_all():

Should describe the exception you get first, then how to use replace to avoid it.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/393/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
785573793,MDU6SXNzdWU3ODU1NzM3OTM=,1186,"script type=""module"" support",9599,simonw,closed,0,,,6346396,Datasette 0.54,1,2021-01-14T01:17:47Z,2021-01-24T21:21:41Z,2021-01-14T01:50:58Z,OWNER,,"Custom JavaScript can be loaded in `metadata.json` like this:
```json
{
    ""extra_js_urls"": [
        {
            ""url"": ""https://code.jquery.com/jquery-3.2.1.slim.min.js"",
            ""sri"": ""sha256-k2WSCIexGzOj3Euiig+TlR8gA0EmPjuc79OEeY5L45g=""
        }
    ]
}
```
Add a `""module"": true` option which causes the resulting script element to use `<script type=""module"">`",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1186/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1200650491,I_kwDOBm6k_c5HkHj7,1711,Template context powered entirely by the JSON API format,9599,simonw,open,0,,,8755003,Datasette 1.0a-next,1,2022-04-11T22:59:27Z,2022-12-13T05:29:06Z,,OWNER,,Datasette 1.0 will have a stable template context. I'm going to achieve this by refactoring the templates to work only with keys returned by the API (or some of its extras) - then the API documentation will double up as template documentation.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1711/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
459590021,MDU6SXNzdWU0NTk1OTAwMjE=,519,Decide what goes into Datasette 1.0,9599,simonw,closed,0,,,3268330,Datasette 1.0,4,2019-06-23T15:47:41Z,2021-11-15T23:26:11Z,2021-11-15T23:26:11Z,OWNER,,Datasette ASGI #272 is a big part of it... but 1.0 will generally be an indicator that Datasette is a stable platform for developers to write plugins and custom templates against. So lots to think about.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/519/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1342430983,I_kwDOBm6k_c5QA98H,1786,Adjust height of textarea for no JS case,9599,simonw,closed,0,,,,,4,2022-08-18T01:15:15Z,2022-10-27T21:50:12Z,2022-08-18T16:06:09Z,OWNER,,"Datasette Lite: https://lite.datasette.io/?sql=https://gist.githubusercontent.com/simonw/1f8a91123ccefd8844187225b1832d7a/raw/5069075b86aa79358fbab3d4482d1d269077d632/recipes.sql#/data?sql=select+id%2C+name%2C+ingredients%2C+%28%0A++select+json_group_array%28value%29+from+json_each%28ingredients%29%0A++where+value+in+%28select+value+from+json_each%28%3Ap0%29%29%0A%29+as+matching_ingredients%0Afrom+recipes%0Awhere+json_array_length%28matching_ingredients%29+%3E+0%0Aorder+by+json_array_length%28matching_ingredients%29+desc&p0=%5B%22sugar%22%2C+%22cheese%22%5D

![46F8101E-8CE3-4F61-B200-F865E6B5DBCC](https://user-images.githubusercontent.com/9599/185270723-f55513b0-b561-434d-9d7c-4fe5be9756e0.jpeg)


",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1786/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1099723916,I_kwDOBm6k_c5BjHSM,1590,Table+query JSON and CSV links broken when using `base_url` setting,1001306,eelkevdbos,closed,0,,,7571612,Datasette 0.60,11,2022-01-11T23:46:39Z,2022-01-14T01:16:34Z,2022-01-14T01:16:08Z,NONE,,"Datasette appends the prefix found in the `base_url` setting twice if a `base_url` is set.

In the follow asgi example, I'm hosting a custom Datasette instance:

```python
# asgi.py
import pathlib
from asgi_cors import asgi_cors
from channels.routing import URLRouter
from django.urls import re_path
from datasette.app import Datasette

datasette_ = Datasette(
    files=[],
    settings={
        ""base_url"": ""/datasettes/"",
        ""plugins"": {}
    },
    config_dir=pathlib.Path('.'),
)
application = URLRouter([
  re_path(r""^datasettes/.*"", asgi_cors(datasette_.app(), allow_all=True)),
])
```

Running it with:
```shell
$ daphne -p 8002 asgi:application
```

Using a simple query on the `_memory` table: 
```sql
select sqlite_version()
```

http://localhost:8002/datasettes/_memory?sql=select+sqlite_version%28%29

It renders the following upon inspection:
![image](https://user-images.githubusercontent.com/1001306/149038851-aa842950-126a-467c-9a86-fae13bce6221.png)

I am using datasette version `0.59.4`",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1590/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
445868234,MDU6SXNzdWU0NDU4NjgyMzQ=,478,Make it so Docker build doesn't delay PyPI release,9599,simonw,closed,0,,,4471010,Datasette 0.29,3,2019-05-19T21:52:10Z,2019-07-08T03:30:41Z,2019-07-07T20:03:20Z,OWNER,,"Datasette automated releases currently include building a Docker image that has a full custom-compiled version of SQLite and SpatiaLite. This takes ages!

I still want to publish this Docker image (to https://hub.docker.com/r/datasetteproject/datasette/tags ) but I'd like it if this wasn't a blocker on pushing the new package to PyPI. Ideally PyPI publish would happen first.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/478/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
684925907,MDU6SXNzdWU2ODQ5MjU5MDc=,948,Upgrade CodeMirror,9599,simonw,closed,0,,,5818042,Datasette 0.49,8,2020-08-24T19:55:33Z,2020-09-30T20:36:12Z,2020-08-30T18:03:07Z,OWNER,,Datasette currently bundles 5.31.0 (from October 2017) - latest version is 5.57.0 (August 2020). https://codemirror.net/doc/releases.html,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/948/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
973139047,MDU6SXNzdWU5NzMxMzkwNDc=,1439,Rethink how .ext formats (v.s. ?_format=) works before 1.0,9599,simonw,closed,0,,,3268330,Datasette 1.0,48,2021-08-17T23:32:51Z,2022-03-15T20:51:26Z,2022-03-15T20:51:26Z,OWNER,,"Datasette currently has surprising special behaviour for if a table name ends in `.csv` - which can happen when a tool like `csvs-to-sqlite` creates tables that match the filename that they were imported from.

https://latest.datasette.io/fixtures/table%2Fwith%2Fslashes.csv illustrates this behaviour: it links to `.csv` and `.json` that look like this:

- https://latest.datasette.io/fixtures/table%2Fwith%2Fslashes.csv?_format=json
- https://latest.datasette.io/fixtures/table%2Fwith%2Fslashes.csv?_format=csv&_size=max

Where normally Datasette would add the `.csv` or `.json` extension to the path component of the URL (as seen on other pages such as https://latest.datasette.io/fixtures/facet_cities) here the [path_with_format() function](https://github.com/simonw/datasette/blob/adb5b70de5cec3c3dd37184defe606a082c232cf/datasette/utils/__init__.py#L710) notices that there is already a `.` in the path and instead adds `?_format=csv` to the query string instead.

The problem with this mechanism is that it's pretty surprising. Anyone writing external code to Datasette who wants to get back the `.csv` or `.json` version giving the URL to a table page will need to know about and implement this behaviour themselves. That's likely to cause all kinds of bugs in the future.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1439/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
520667773,MDU6SXNzdWU1MjA2Njc3NzM=,620,Mechanism for indicating foreign key relationships in the table and query page URLs,9599,simonw,open,0,,,,,6,2019-11-10T22:26:27Z,2021-04-05T03:57:22Z,,OWNER,,"Datasette currently only inflates foreign keys (into names hyperlinks) if it detects them as foreign key constraints in the underlying database.

It would be useful if you could specify additional ""foreign keys"" using both `metadata.json` and the querystring - similar time how you can pass `?_fts_table=x` https://datasette.readthedocs.io/en/stable/full_text_search.html#configuring-full-text-search-for-a-table-or-view",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/620/reactions"", ""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 1}",,
1563264257,I_kwDOBm6k_c5dLYUB,2010,Row page should default to card view,9599,simonw,open,0,,,3268330,Datasette 1.0,1,2023-01-30T21:49:37Z,2023-01-30T21:52:06Z,,OWNER,,"Datasette currently uses the same table layout on the row pages as it does on the table pages:

https://datasette.io/content/pypi_packages?_sort=name&name__exact=datasette-column-inspect

<img width=""1068"" alt=""image"" src=""https://user-images.githubusercontent.com/9599/215602793-b15dad48-c3e9-4f34-9875-3e79b7aa586d.png"">

https://datasette.io/content/pypi_packages/datasette-column-inspect

<img width=""1068"" alt=""image"" src=""https://user-images.githubusercontent.com/9599/215602883-9d714eec-3908-493e-93d0-c788af47ba21.png"">

If you shrink down to mobile width you get this instead, on both of those pages:

<img width=""613"" alt=""image"" src=""https://user-images.githubusercontent.com/9599/215602964-b95aa089-0269-4647-beae-66fffad332f5.png"">

I think that view, which I think of as the ""card view"", is plain better if you're looking at just a single row - and it (or a variant of it) should be the default presentation on the row page.

",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2010/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
895686039,MDU6SXNzdWU4OTU2ODYwMzk=,1336,Document turning on WAL for live served SQLite databases,9599,simonw,open,0,,,,,1,2021-05-19T17:08:58Z,2022-01-13T21:55:59Z,,OWNER,,"Datasette docs don't talk about WAL yet, which allows you to safely serve reads from a database file while it is accepting writes.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1336/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1399933513,I_kwDOBm6k_c5TcUpJ,1833,Ability to submit long queries by POST,9599,simonw,open,0,,,,,0,2022-10-06T16:03:26Z,2022-10-06T16:18:00Z,,OWNER,,"Datasette doesn't limit URL lengths but some common web proxies do - the one in front of Google Cloud Run for example limits to 8KB total for incoming request headers: https://cloud.google.com/load-balancing/docs/quotas#https-lb-header-limits

This means longer SQL queries can break!

Need an optional mechanism for submitting queries by POST instead.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1833/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
324188953,MDU6SXNzdWUzMjQxODg5NTM=,272,Port Datasette to ASGI,9599,simonw,closed,0,9599,simonw,3268330,Datasette 1.0,42,2018-05-17T21:16:32Z,2019-06-24T04:54:15Z,2019-06-24T03:33:06Z,OWNER,,"Datasette doesn't take much advantage of Sanic, and I'm increasingly having to work around parts of it because of idiosyncrasies that are specific to Datasette - caring about the exact order of querystring arguments for example.

Since Datasette is GET-only our needs from a web framework are actually pretty slim.

This becomes more important as I expand the plugins #14 framework. Am I sure I want the plugin ecosystem to depend on a Sanic if I might move away from it in the future?

If Datasette wasn't all about async/await I would use WSGI, but today it makes more sense to use ASGI. I'd like to be confident that switching to ASGI would still give me the excellent performance that Sanic provides.

https://github.com/django/asgiref/blob/master/specs/asgi.rst",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/272/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
465731062,MDU6SXNzdWU0NjU3MzEwNjI=,555,Static mounts with relative paths not working,3243482,abdusco,closed,0,,,,,0,2019-07-09T11:38:35Z,2019-07-11T16:13:22Z,2019-07-11T16:13:22Z,CONTRIBUTOR,,"Datasette fails to serve files from static mounts that are created using relative paths `datasette --static mystatic:rel/path/to/static/dir`.  
I've explained the problem and the solution in the pull request: https://github.com/simonw/datasette/pull/554",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/555/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1447050738,I_kwDOBm6k_c5WQD3y,1886,"Call for birthday presents: if you're using Datasette, let us know how you're using it here",9599,simonw,open,0,,,,,13,2022-11-13T19:25:51Z,2022-12-18T17:34:20Z,,OWNER,,"Datasette is 5 years old today. To celebrate, I'm asking the community for birthday presents:

https://simonwillison.net/2022/Nov/13/datasette-birthday/

> To celebrate this open source project’s birthday, I’ve decided to try something new: I’m going to ask for birthday presents.
> 
> An aspect of Datastte’s marketing that I’ve so far neglected is social proof. I think it’s time to change that: I know people are using the software to do cool things, but this often happens behind closed doors.
> 
> For Datastte’s birthday, I’m looking for endorsements and case studies and just general demonstrations that show how people are using it do so cool stuff.
> 
> So: if you’ve used Datasette to solve a problem, and you’re willing to publicize it, please give us the gift of your endorsement!
> 
> [...]
> 
> Add a comment to [this issue thread](https://github.com/simonw/datasette/issues/1886) describing what you’re doing. Just a few sentences is fine—though a screenshot or even a link to a live instance would be even better",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1886/reactions"", ""total_count"": 2, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 2, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
316621102,MDU6SXNzdWUzMTY2MjExMDI=,235,Add limit on the size in KB of data returned from a single query,9599,simonw,open,0,,,,,2,2018-04-22T23:01:15Z,2018-04-24T00:30:02Z,,OWNER,,"Datasette limits the number of rows returned to 1,000 and limits the time spent executing a SQL query to 1000ms - and both of these limits can be customized.

It does not have a limit on the size of the response returned. It's possible to compose maliciously large SQL responses in a small number of rows using mechanisms like the `group_concat()` aggregate function. It would be good to avoid malicious SQL creating 100MB+ responses and potentially crashing the server.

I think the easiest place to implement that is here:

https://github.com/simonw/datasette/blob/f3f42957128c1e7ece584d45d9167f2ac003a3b8/datasette/app.py#L175-L190

Currently we use `cursor.fetchmany()` to fetch up to 1,001 rows at once. Instead, we could switch to iterating through `cursor.fetchone()` (or just using `for row in cursor`) and keeping a running tally of the size of the response as we go - maybe just using `rough_response_size += len(str(row))`. If that goes above a certain threshold we can terminate the response with an error, like we do with timelimits.

The bigger challenge here is understanding how well this approach works and what impact it will have on overall Datasette performance. I think I need #33 for this.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/235/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
323681589,MDU6SXNzdWUzMjM2ODE1ODk=,266,Export to CSV,9599,simonw,closed,0,,,,,27,2018-05-16T15:50:24Z,2021-06-17T18:14:24Z,2018-06-18T06:05:25Z,OWNER,,Datasette needs to be able to export data to CSV.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/266/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
476573875,MDU6SXNzdWU0NzY1NzM4NzU=,567,Datasette Edit,9599,simonw,closed,0,,,,,3,2019-08-04T17:09:28Z,2020-02-25T03:40:50Z,2020-02-25T03:40:50Z,OWNER,,"Datasette started out immutable. Then it gained the ability to run against read-only databases that were being modified by other processes. It's time for the next logical progression: the option to allow Datasette (or more likely individual plugins) to write to the database!

This is going to require some careful rethinking of how connection management works.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/567/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1052826038,I_kwDOBm6k_c4-wNm2,1506,Columns beginning with an underscore do not facet correctly,9599,simonw,closed,0,,,,,1,2021-11-14T02:20:32Z,2021-11-14T04:45:21Z,2021-11-14T04:45:21Z,OWNER,,"Datasette treats columns that start with an underscore as querystring parameters it should ignore!

<img width=""914"" alt=""bchydro__item_versions__99_918_rows"" src=""https://user-images.githubusercontent.com/9599/141664864-119990d5-42b2-47f7-91bd-09a74a288af9.png"">

Discovered in https://github.com/simonw/git-history/issues/14#issuecomment-968192464",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1506/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
959999095,MDU6SXNzdWU5NTk5OTkwOTU=,1421,"""Query parameters"" form shows wrong input fields if query contains ""03:31"" style times",6988,j4mie,closed,0,,,,,11,2021-08-04T07:29:04Z,2021-08-09T03:41:07Z,2021-08-09T03:33:02Z,NONE,,"Datasette version `0.58.1`.

I'm guessing this is a bug in the code that looks for `:param`-style query parameters..

<img width=""543"" alt=""image"" src=""https://user-images.githubusercontent.com/6988/128139832-ef9c5291-f3d7-4402-8625-b45d26b4e5bc.png"">
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1421/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
323658641,MDU6SXNzdWUzMjM2NTg2NDE=,262,Add ?_extra= mechanism for requesting extra properties in JSON,9599,simonw,open,0,,,3268330,Datasette 1.0,27,2018-05-16T14:55:42Z,2023-03-29T06:22:22Z,,OWNER,,"Datasette views currently work by creating a set of data that should be returned as JSON, then defining an additional, optional `template_data()` function which is called if the view is being rendered as HTML.

This `template_data()` function calculates extra template context variables which are necessary for the HTML view but should not be included in the JSON.

Example of how that is used today: https://github.com/simonw/datasette/blob/2b79f2bdeb1efa86e0756e741292d625f91cb93d/datasette/views/table.py#L672-L704

With features like Facets in #255 I'm beginning to want to move more items into the `template_data()` - in the case of facets it's the `suggested_facets` array. This saves that feature from being calculated (involving several SQL queries) for the JSON case where it is unlikely to be used.

But... as an API user, I want to still optionally be able to access that information.

Solution: Add a `?_extra=suggested_facets&_extra=table_metadata` argument which can be used to optionally request additional blocks to be added to the JSON API.

Then redefine as many of the current `template_data()` features as extra arguments instead, and teach Datasette to return certain extras by default when rendering templates.

This could allow the JSON representation to be slimmed down further (removing e.g. the `table_definition` and `view_definition` keys) while still making that information available to API users who need it.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/262/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
678760988,MDU6SXNzdWU2Nzg3NjA5ODg=,932,End-user documentation,9599,simonw,open,0,,,3268330,Datasette 1.0,6,2020-08-13T22:04:39Z,2022-03-08T15:20:48Z,,OWNER,,"Datasette's documentation is aimed at people who install and configure it.

What about end users of preconfigured and deployed Datasette instances?

Something that can be linked to from the Datasette UI would be really useful.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/932/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
695359607,MDU6SXNzdWU2OTUzNTk2MDc=,150,Feature for tracing SQL queries,9599,simonw,closed,0,,,,,0,2020-09-07T19:43:08Z,2020-09-07T21:57:01Z,2020-09-07T21:57:01Z,OWNER,,"Debugging `sqlite-utils` when something weird happens (e.g. #149) can be a bit tricky since it runs a bunch of different SQL statements behind the scenes.

An optional ""tracing"" mechanism for seeing what SQL is being executed would be useful.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/150/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
472429048,MDU6SXNzdWU0NzI0MjkwNDg=,9,Too many SQL variables,166463,tholo,closed,0,,,,,4,2019-07-24T18:24:17Z,2019-07-26T10:01:05Z,2019-07-26T10:01:05Z,NONE,,"Decided to try importing my data, and ran into this:

```
Traceback (most recent call last):
  File ""/Users/tholo/Source/health/bin/healthkit-to-sqlite"", line 10, in <module>
    sys.exit(cli())
  File ""/Users/tholo/Source/health/lib/python3.7/site-packages/click/core.py"", line 764, in __call__
    return self.main(*args, **kwargs)
  File ""/Users/tholo/Source/health/lib/python3.7/site-packages/click/core.py"", line 717, in main
    rv = self.invoke(ctx)
  File ""/Users/tholo/Source/health/lib/python3.7/site-packages/click/core.py"", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File ""/Users/tholo/Source/health/lib/python3.7/site-packages/click/core.py"", line 555, in invoke
    return callback(*args, **kwargs)
  File ""/Users/tholo/Source/health/lib/python3.7/site-packages/healthkit_to_sqlite/cli.py"", line 50, in cli
    convert_xml_to_sqlite(fp, db, progress_callback=bar.update)
  File ""/Users/tholo/Source/health/lib/python3.7/site-packages/healthkit_to_sqlite/utils.py"", line 41, in convert_xml_to_sqlite
    write_records(records, db)
  File ""/Users/tholo/Source/health/lib/python3.7/site-packages/healthkit_to_sqlite/utils.py"", line 80, in write_records
    column_order=[""startDate"", ""endDate"", ""value"", ""unit""],
  File ""/Users/tholo/Source/health/lib/python3.7/site-packages/sqlite_utils/db.py"", line 911, in insert_all
    result = self.db.conn.execute(sql, values)
sqlite3.OperationalError: too many SQL variables
```

Added some debug output in sqlite_utils/db.py, which resulted in:

```
                INSERT INTO [rBodyMassIndex] ([creationDate], [endDate], [metadata_HKWasUserEntered], [metadata_Health Mate App Version], [metadata_Modified Date], [metadata_Withings Link], [metadata_Withings User Identifier], [sourceName], [sourceVersion], [startDate], [unit], [value]) VALUES
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ,
                    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                ;
```

with the attached data:

```
['2019-06-27 22:55:10 -0700', '2011-06-22 21:05:53 -0700', '0', '4.4.2', '2011-06-23 04:05:53 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1308801953&type=1', '301293', 'Health Mate', '4040200', '2011-06-22 21:05:53 -0700', 'count', '30.0926', '2019-06-27 22:55:10 -0700', '2011-06-23 09:36:27 -0700', '0', '4.4.2', '2011-06-23 16:36:59 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1308846987&type=1', '301293', 'Health Mate', '4040200', '2011-06-23 09:36:27 -0700', 'count', '30.0926', '2019-06-27 22:55:10 -0700', '2011-06-23 23:54:07 -0700', '0', '4.4.2', '2011-06-24 06:55:19 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1308898447&type=1', '301293', 'Health Mate', '4040200', '2011-06-23 23:54:07 -0700', 'count', '30.679', '2019-06-27 22:55:10 -0700', '2011-06-24 09:13:40 -0700', '0', '4.4.2', '2011-06-24 16:14:35 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1308932020&type=1', '301293', 'Health Mate', '4040200', '2011-06-24 09:13:40 -0700', 'count', '30.3549', '2019-06-27 22:55:10 -0700', '2011-06-25 08:30:08 -0700', '0', '4.4.2', '2011-06-25 15:30:49 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1309015808&type=1', '301293', 'Health Mate', '4040200', '2011-06-25 08:30:08 -0700', 'count', '30.3395', '2019-06-27 22:55:10 -0700', '2011-06-26 07:47:51 -0700', '0', '4.4.2', '2011-06-26 14:48:27 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1309099671&type=1', '301293', 'Health Mate', '4040200', '2011-06-26 07:47:51 -0700', 'count', '30.2315', '2019-06-27 22:55:10 -0700', '2011-06-28 08:48:26 -0700', '0', '4.4.2', '2011-06-28 15:49:13 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1309276106&type=1', '301293', 'Health Mate', '4040200', '2011-06-28 08:48:26 -0700', 'count', '30.0617', '2019-06-27 22:55:10 -0700', '2011-06-29 09:21:16 -0700', '0', '4.4.2', '2011-06-29 16:21:59 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1309364476&type=1', '301293', 'Health Mate', '4040200', '2011-06-29 09:21:16 -0700', 'count', '29.9537', '2019-06-27 22:55:10 -0700', '2011-06-30 08:41:46 -0700', '0', '4.4.2', '2011-06-30 15:42:30 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1309448506&type=1', '301293', 'Health Mate', '4040200', '2011-06-30 08:41:46 -0700', 'count', '29.8302', '2019-06-27 22:55:10 -0700', '2011-07-01 09:05:28 -0700', '0', '4.4.2', '2011-07-01 16:06:24 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1309536328&type=1', '301293', 'Health Mate', '4040200', '2011-07-01 09:05:28 -0700', 'count', '29.8611', '2019-06-27 22:55:10 -0700', '2011-07-02 08:58:50 -0700', '0', '4.4.2', '2011-07-02 15:59:40 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1309622330&type=1', '301293', 'Health Mate', '4040200', '2011-07-02 08:58:50 -0700', 'count', '29.8765', '2019-06-27 22:55:10 -0700', '2011-07-04 09:33:43 -0700', '0', '4.4.2', '2011-07-04 16:34:19 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1309797223&type=1', '301293', 'Health Mate', '4040200', '2011-07-04 09:33:43 -0700', 'count', '30.0309', '2019-06-27 22:55:10 -0700', '2011-07-06 09:40:23 -0700', '0', '4.4.2', '2011-07-06 16:41:02 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1309970423&type=1', '301293', 'Health Mate', '4040200', '2011-07-06 09:40:23 -0700', 'count', '30.1852', '2019-06-27 22:55:10 -0700', '2011-07-08 08:08:48 -0700', '0', '4.4.2', '2011-07-08 15:09:51 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1310137728&type=1', '301293', 'Health Mate', '4040200', '2011-07-08 08:08:48 -0700', 'count', '30.0309', '2019-06-27 22:55:10 -0700', '2011-07-09 08:31:05 -0700', '0', '4.4.2', '2011-07-09 15:31:48 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1310225465&type=1', '301293', 'Health Mate', '4040200', '2011-07-09 08:31:05 -0700', 'count', '29.9537', '2019-06-27 22:55:10 -0700', '2011-07-10 08:14:36 -0700', '0', '4.4.2', '2011-07-10 15:15:12 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1310310876&type=1', '301293', 'Health Mate', '4040200', '2011-07-10 08:14:36 -0700', 'count', '30.0926', '2019-06-27 22:55:10 -0700', '2011-07-12 07:55:21 -0700', '0', '4.4.2', '2011-07-12 14:55:59 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1310482521&type=1', '301293', 'Health Mate', '4040200', '2011-07-12 07:55:21 -0700', 'count', '30.108', '2019-06-27 22:55:10 -0700', '2011-07-13 08:48:05 -0700', '0', '4.4.2', '2011-07-13 15:48:42 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1310572085&type=1', '301293', 'Health Mate', '4040200', '2011-07-13 08:48:05 -0700', 'count', '30', '2019-06-27 22:55:10 -0700', '2011-07-14 09:05:16 -0700', '0', '4.4.2', '2011-07-14 16:05:57 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1310659516&type=1', '301293', 'Health Mate', '4040200', '2011-07-14 09:05:16 -0700', 'count', '29.9074', '2019-06-27 22:55:10 -0700', '2011-07-15 07:09:56 -0700', '0', '4.4.2', '2011-07-15 14:10:35 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1310738996&type=1', '301293', 'Health Mate', '4040200', '2011-07-15 07:09:56 -0700', 'count', '29.9537', '2019-06-27 22:55:10 -0700', '2011-07-16 09:26:04 -0700', '0', '4.4.2', '2011-07-16 16:26:44 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1310833564&type=1', '301293', 'Health Mate', '4040200', '2011-07-16 09:26:04 -0700', 'count', '29.7531', '2019-06-27 22:55:10 -0700', '2011-07-17 09:52:59 -0700', '0', '4.4.2', '2011-07-17 16:53:38 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1310921579&type=1', '301293', 'Health Mate', '4040200', '2011-07-17 09:52:59 -0700', 'count', '29.8765', '2019-06-27 22:55:10 -0700', '2011-07-19 08:56:16 -0700', '0', '4.4.2', '2011-07-19 15:57:03 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1311090976&type=1', '301293', 'Health Mate', '4040200', '2011-07-19 08:56:16 -0700', 'count', '29.7685', '2019-06-27 22:55:10 -0700', '2011-07-21 08:21:20 -0700', '0', '4.4.2', '2011-07-21 15:22:02 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1311261680&type=1', '301293', 'Health Mate', '4040200', '2011-07-21 08:21:20 -0700', 'count', '29.7685', '2019-06-27 22:55:10 -0700', '2011-07-23 08:49:56 -0700', '0', '4.4.2', '2011-07-23 15:50:40 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1311436196&type=1', '301293', 'Health Mate', '4040200', '2011-07-23 08:49:56 -0700', 'count', '29.7222', '2019-06-27 22:55:10 -0700', '2011-07-24 09:17:35 -0700', '0', '4.4.2', '2011-07-24 16:18:14 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1311524255&type=1', '301293', 'Health Mate', '4040200', '2011-07-24 09:17:35 -0700', 'count', '29.5833', '2019-06-27 22:55:10 -0700', '2011-07-25 07:51:55 -0700', '0', '4.4.2', '2011-07-25 14:52:48 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1311605515&type=1', '301293', 'Health Mate', '4040200', '2011-07-25 07:51:55 -0700', 'count', '29.5525', '2019-06-27 22:55:10 -0700', '2011-08-06 10:04:05 -0700', '0', '4.4.2', '2011-08-06 17:04:47 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1312650245&type=1', '301293', 'Health Mate', '4040200', '2011-08-06 10:04:05 -0700', 'count', '29.7377', '2019-06-27 22:55:10 -0700', '2011-08-08 07:52:22 -0700', '0', '4.4.2', '2011-08-08 14:53:03 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1312815142&type=1', '301293', 'Health Mate', '4040200', '2011-08-08 07:52:22 -0700', 'count', '29.6605', '2019-06-27 22:55:10 -0700', '2011-08-10 07:57:30 -0700', '0', '4.4.2', '2011-08-10 14:58:12 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1312988250&type=1', '301293', 'Health Mate', '4040200', '2011-08-10 07:57:30 -0700', 'count', '29.7531', '2019-06-27 22:55:10 -0700', '2011-08-12 07:51:14 -0700', '0', '4.4.2', '2011-08-12 14:51:59 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1313160674&type=1', '301293', 'Health Mate', '4040200', '2011-08-12 07:51:14 -0700', 'count', '29.6914', '2019-06-27 22:55:10 -0700', '2011-08-13 07:45:28 -0700', '0', '4.4.2', '2011-08-13 14:46:08 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1313246728&type=1', '301293', 'Health Mate', '4040200', '2011-08-13 07:45:28 -0700', 'count', '29.5833', '2019-06-27 22:55:10 -0700', '2011-08-17 09:06:20 -0700', '0', '4.4.2', '2011-08-17 16:07:02 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1313597180&type=1', '301293', 'Health Mate', '4040200', '2011-08-17 09:06:20 -0700', 'count', '29.5679', '2019-06-27 22:55:10 -0700', '2011-08-22 08:28:08 -0700', '0', '4.4.2', '2011-08-22 15:28:57 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1314026888&type=1', '301293', 'Health Mate', '4040200', '2011-08-22 08:28:08 -0700', 'count', '29.9846', '2019-06-27 22:55:10 -0700', '2011-08-25 08:59:30 -0700', '0', '4.4.2', '2011-08-25 16:00:15 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1314287970&type=1', '301293', 'Health Mate', '4040200', '2011-08-25 08:59:30 -0700', 'count', '29.9691', '2019-06-27 22:55:10 -0700', '2011-08-30 08:13:59 -0700', '0', '4.4.2', '2011-08-30 15:46:08 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1314717239&type=1', '301293', 'Health Mate', '4040200', '2011-08-30 08:13:59 -0700', 'count', '29.784', '2019-06-27 22:55:10 -0700', '2011-09-12 08:47:51 -0700', '0', '4.4.2', '2011-09-12 15:48:59 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1315842471&type=1', '301293', 'Health Mate', '4040200', '2011-09-12 08:47:51 -0700', 'count', '29.7377', '2019-06-27 22:55:10 -0700', '2011-09-13 09:17:27 -0700', '0', '4.4.2', '2011-09-13 16:48:30 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1315930647&type=1', '301293', 'Health Mate', '4040200', '2011-09-13 09:17:27 -0700', 'count', '29.7531', '2019-06-27 22:55:10 -0700', '2011-10-01 09:12:20 -0700', '0', '4.4.2', '2011-10-01 16:13:00 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1317485540&type=1', '301293', 'Health Mate', '4040200', '2011-10-01 09:12:20 -0700', 'count', '29.8148', '2019-06-27 22:55:10 -0700', '2011-10-11 11:14:11 -0700', '0', '4.4.2', '2011-10-11 18:15:14 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1318356851&type=1', '301293', 'Health Mate', '4040200', '2011-10-11 11:14:11 -0700', 'count', '29.7377', '2019-06-27 22:55:10 -0700', '2011-10-16 09:29:47 -0700', '0', '4.4.2', '2011-10-16 16:30:39 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1318782587&type=1', '301293', 'Health Mate', '4040200', '2011-10-16 09:29:47 -0700', 'count', '29.6914', '2019-06-27 22:55:10 -0700', '2011-10-19 09:21:44 -0700', '0', '4.4.2', '2011-10-19 16:22:25 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1319041304&type=1', '301293', 'Health Mate', '4040200', '2011-10-19 09:21:44 -0700', 'count', '29.7685', '2019-06-27 22:55:10 -0700', '2011-10-24 07:04:22 -0700', '0', '4.4.2', '2011-10-24 14:05:03 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1319465062&type=1', '301293', 'Health Mate', '4040200', '2011-10-24 07:04:22 -0700', 'count', '29.5988', '2019-06-27 22:55:10 -0700', '2011-11-07 09:33:17 -0700', '0', '4.4.2', '2011-11-07 16:33:58 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1320683597&type=1', '301293', 'Health Mate', '4040200', '2011-11-07 09:33:17 -0700', 'count', '29.8611', '2019-06-27 22:55:10 -0700', '2011-11-10 07:59:03 -0700', '0', '4.4.2', '2011-11-10 14:59:48 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1320937143&type=1', '301293', 'Health Mate', '4040200', '2011-11-10 07:59:03 -0700', 'count', '29.9383', '2019-06-27 22:55:10 -0700', '2011-11-13 09:28:31 -0700', '0', '4.4.2', '2011-11-13 16:29:20 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1321201711&type=1', '301293', 'Health Mate', '4040200', '2011-11-13 09:28:31 -0700', 'count', '29.7531', '2019-06-27 22:55:10 -0700', '2011-11-21 08:45:06 -0700', '0', '4.4.2', '2011-11-21 15:46:04 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1321890306&type=1', '301293', 'Health Mate', '4040200', '2011-11-21 08:45:06 -0700', 'count', '29.9691', '2019-06-27 22:55:10 -0700', '2011-11-23 09:55:44 -0700', '0', '4.4.2', '2011-11-23 16:56:18 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1322067344&type=1', '301293', 'Health Mate', '4040200', '2011-11-23 09:55:44 -0700', 'count', '29.8302', '2019-06-27 22:55:10 -0700', '2011-11-29 09:50:44 -0700', '0', '4.4.2', '2011-11-29 16:51:31 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1322585444&type=1', '301293', 'Health Mate', '4040200', '2011-11-29 09:50:44 -0700', 'count', '30.1698', '2019-06-27 22:55:10 -0700', '2011-11-30 11:13:21 -0700', '0', '4.4.2', '2011-11-30 18:14:14 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1322676801&type=1', '301293', 'Health Mate', '4040200', '2011-11-30 11:13:21 -0700', 'count', '30.0617', '2019-06-27 22:55:10 -0700', '2011-12-04 10:24:36 -0700', '0', '4.4.2', '2011-12-04 17:25:24 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1323019476&type=1', '301293', 'Health Mate', '4040200', '2011-12-04 10:24:36 -0700', 'count', '29.9691', '2019-06-27 22:55:10 -0700', '2011-12-10 09:22:18 -0700', '0', '4.4.2', '2011-12-10 16:23:07 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1323534138&type=1', '301293', 'Health Mate', '4040200', '2011-12-10 09:22:18 -0700', 'count', '29.9537', '2019-06-27 22:55:10 -0700', '2011-12-26 10:36:42 -0700', '0', '4.4.2', '2011-12-26 17:37:31 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1324921002&type=1', '301293', 'Health Mate', '4040200', '2011-12-26 10:36:42 -0700', 'count', '30.0926', '2019-06-27 22:55:10 -0700', '2012-01-11 11:24:13 -0700', '0', '4.4.2', '2012-01-11 18:25:04 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1326306253&type=1', '301293', 'Health Mate', '4040200', '2012-01-11 11:24:13 -0700', 'count', '29.8302', '2019-06-27 22:55:10 -0700', '2012-01-15 10:17:09 -0700', '0', '4.4.2', '2012-01-15 17:17:51 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1326647829&type=1', '301293', 'Health Mate', '4040200', '2012-01-15 10:17:09 -0700', 'count', '29.8302', '2019-06-27 22:55:10 -0700', '2012-01-19 09:24:32 -0700', '0', '4.4.2', '2012-01-19 16:25:21 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1326990272&type=1', '301293', 'Health Mate', '4040200', '2012-01-19 09:24:32 -0700', 'count', '29.7994', '2019-06-27 22:55:10 -0700', '2012-01-29 10:26:13 -0700', '0', '4.4.2', '2012-01-29 17:26:52 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1327857973&type=1', '301293', 'Health Mate', '4040200', '2012-01-29 10:26:13 -0700', 'count', '30.0154', '2019-06-27 22:55:10 -0700', '2012-02-03 10:13:28 -0700', '0', '4.4.2', '2012-02-03 17:15:01 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1328289208&type=1', '301293', 'Health Mate', '4040200', '2012-02-03 10:13:28 -0700', 'count', '29.8457', '2019-06-27 22:55:10 -0700', '2012-02-12 09:23:01 -0700', '0', '4.4.2', '2012-02-12 16:23:53 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1329063781&type=1', '301293', 'Health Mate', '4040200', '2012-02-12 09:23:01 -0700', 'count', '30.1235', '2019-06-27 22:55:10 -0700', '2012-03-03 09:26:06 -0700', '0', '4.4.2', '2012-03-03 16:26:54 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1330791966&type=1', '301293', 'Health Mate', '4040200', '2012-03-03 09:26:06 -0700', 'count', '30.0926', '2019-06-27 22:55:10 -0700', '2012-03-11 11:23:15 -0700', '0', '4.4.2', '2012-03-11 18:24:16 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1331490195&type=1', '301293', 'Health Mate', '4040200', '2012-03-11 11:23:15 -0700', 'count', '30.2161', '2019-06-27 22:55:10 -0700', '2012-03-16 09:39:36 -0700', '0', '4.4.2', '2012-03-16 16:40:20 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1331915976&type=1', '301293', 'Health Mate', '4040200', '2012-03-16 09:39:36 -0700', 'count', '30.2778', '2019-06-27 22:55:10 -0700', '2012-03-21 08:33:07 -0700', '0', '4.4.2', '2012-03-21 15:34:00 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1332343987&type=1', '301293', 'Health Mate', '4040200', '2012-03-21 08:33:07 -0700', 'count', '30.1389', '2019-06-27 22:55:10 -0700', '2012-04-11 08:49:34 -0700', '0', '4.4.2', '2012-04-11 15:50:18 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1334159374&type=1', '301293', 'Health Mate', '4040200', '2012-04-11 08:49:34 -0700', 'count', '30.0154', '2019-06-27 22:55:10 -0700', '2012-04-13 08:32:06 -0700', '0', '4.4.2', '2012-04-13 15:32:49 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1334331126&type=1', '301293', 'Health Mate', '4040200', '2012-04-13 08:32:06 -0700', 'count', '29.9383', '2019-06-27 22:55:10 -0700', '2012-04-20 08:21:38 -0700', '0', '4.4.2', '2012-04-20 15:52:45 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1334935298&type=1', '301293', 'Health Mate', '4040200', '2012-04-20 08:21:38 -0700', 'count', '30.2006', '2019-06-27 22:55:10 -0700', '2012-04-25 09:00:01 -0700', '0', '4.4.2', '2012-04-25 16:00:42 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1335369601&type=1', '301293', 'Health Mate', '4040200', '2012-04-25 09:00:01 -0700', 'count', '30.2006', '2019-06-27 22:55:10 -0700', '2012-05-04 11:10:18 -0700', '0', '4.4.2', '2012-05-04 18:10:59 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1336155018&type=1', '301293', 'Health Mate', '4040200', '2012-05-04 11:10:18 -0700', 'count', '30.4321', '2019-06-27 22:55:10 -0700', '2012-05-12 09:35:00 -0700', '0', '4.4.2', '2012-05-12 16:35:43 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1336840500&type=1', '301293', 'Health Mate', '4040200', '2012-05-12 09:35:00 -0700', 'count', '30.1235', '2019-06-27 22:55:10 -0700', '2012-05-22 09:27:53 -0700', '0', '4.4.2', '2012-05-22 16:28:37 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1337704073&type=1', '301293', 'Health Mate', '4040200', '2012-05-22 09:27:53 -0700', 'count', '30.4167', '2019-06-27 22:55:10 -0700', '2012-05-31 09:23:16 -0700', '0', '4.4.2', '2012-05-31 16:24:04 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1338481396&type=1', '301293', 'Health Mate', '4040200', '2012-05-31 09:23:16 -0700', 'count', '30.2006', '2019-06-27 22:55:10 -0700', '2012-06-08 09:29:07 -0700', '0', '4.4.2', '2012-06-08 16:29:52 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1339172947&type=1', '301293', 'Health Mate', '4040200', '2012-06-08 09:29:07 -0700', 'count', '30.5247', '2019-06-27 22:55:10 -0700', '2012-06-21 08:07:33 -0700', '0', '4.4.2', '2012-06-21 15:08:20 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1340291253&type=1', '301293', 'Health Mate', '4040200', '2012-06-21 08:07:33 -0700', 'count', '30.5864', '2019-06-27 22:55:10 -0700', '2012-08-08 10:02:22 -0700', '0', '4.4.2', '2012-08-08 17:03:02 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1344445342&type=1', '301293', 'Health Mate', '4040200', '2012-08-08 10:02:22 -0700', 'count', '30.6636', '2019-06-27 22:55:10 -0700', '2012-08-17 09:11:32 -0700', '0', '4.4.2', '2012-08-17 16:42:05 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1345219892&type=1', '301293', 'Health Mate', '4040200', '2012-08-17 09:11:32 -0700', 'count', '30.8796', '2019-06-27 22:55:10 -0700', '2012-09-10 08:27:21 -0700', '0', '4.4.2', '2012-09-10 15:28:07 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1347290841&type=1', '301293', 'Health Mate', '4040200', '2012-09-10 08:27:21 -0700', 'count', '31.034', '2019-06-27 22:55:10 -0700', '2012-09-17 08:35:33 -0700', '0', '4.4.2', '2012-09-17 15:35:33 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1347896133&type=1', '301293', 'Health Mate', '4040200', '2012-09-17 08:35:33 -0700', 'count', '30.7099', '2019-06-27 22:55:10 -0700', '2012-09-26 08:59:46 -0700', '0', '4.4.2', '2012-09-26 16:13:18 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1348675186&type=1', '301293', 'Health Mate', '4040200', '2012-09-26 08:59:46 -0700', 'count', '30.679', '2019-06-27 22:55:10 -0700', '2012-10-18 08:51:16 -0700', '0', '4.4.2', '2012-10-18 15:51:59 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1350575476&type=1', '301293', 'Health Mate', '4040200', '2012-10-18 08:51:16 -0700', 'count', '30.7716', '2019-06-27 22:55:10 -0700', '2012-11-15 08:54:57 -0700', '0', '4.4.2', '2012-11-15 15:55:58 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1352994897&type=1', '301293', 'Health Mate', '4040200', '2012-11-15 08:54:57 -0700', 'count', '31.0802', '2019-06-27 22:55:10 -0700', '2012-12-17 09:13:40 -0700', '0', '4.4.2', '2012-12-17 16:20:03 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1355760820&type=1', '301293', 'Health Mate', '4040200', '2012-12-17 09:13:40 -0700', 'count', '29.784', '2019-06-27 22:55:10 -0700', '2012-12-19 11:09:55 -0700', '0', '4.4.2', '2012-12-19 18:10:37 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1355940595&type=1', '301293', 'Health Mate', '4040200', '2012-12-19 11:09:55 -0700', 'count', '29.6914', '2019-06-27 22:55:10 -0700', '2012-12-25 10:37:41 -0700', '0', '4.4.2', '2012-12-25 17:38:25 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1356457061&type=1', '301293', 'Health Mate', '4040200', '2012-12-25 10:37:41 -0700', 'count', '29.8765', '2019-06-27 22:55:10 -0700', '2013-01-01 10:44:02 -0700', '0', '4.4.2', '2013-01-01 17:44:46 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1357062242&type=1', '301293', 'Health Mate', '4040200', '2013-01-01 10:44:02 -0700', 'count', '30.0772', '2019-06-27 22:55:10 -0700', '2013-01-15 09:10:46 -0700', '0', '4.4.2', '2013-01-15 16:11:28 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1358266246&type=1', '301293', 'Health Mate', '4040200', '2013-01-15 09:10:46 -0700', 'count', '29.9691', '2019-06-27 22:55:10 -0700', '2013-01-20 11:03:39 -0700', '0', '4.4.2', '2013-01-20 18:04:22 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1358705019&type=1', '301293', 'Health Mate', '4040200', '2013-01-20 11:03:39 -0700', 'count', '30.108', '2019-06-27 22:55:10 -0700', '2013-01-30 08:56:30 -0700', '0', '4.4.2', '2013-01-30 15:57:14 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1359561390&type=1', '301293', 'Health Mate', '4040200', '2013-01-30 08:56:30 -0700', 'count', '30.0926', '2019-06-27 22:55:10 -0700', '2013-02-04 11:02:35 -0700', '0', '4.4.2', '2013-02-04 18:03:25 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1360000955&type=1', '301293', 'Health Mate', '4040200', '2013-02-04 11:02:35 -0700', 'count', '29.8148', '2019-06-27 22:55:10 -0700', '2013-02-07 09:07:06 -0700', '0', '4.4.2', '2013-02-07 16:07:49 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1360253226&type=1', '301293', 'Health Mate', '4040200', '2013-02-07 09:07:06 -0700', 'count', '30.1389', '2019-06-27 22:55:10 -0700', '2013-02-19 08:49:57 -0700', '0', '4.4.2', '2013-02-19 15:50:39 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1361288997&type=1', '301293', 'Health Mate', '4040200', '2013-02-19 08:49:57 -0700', 'count', '30.1235', '2019-06-27 22:55:10 -0700', '2013-03-02 11:20:54 -0700', '0', '4.4.2', '2013-03-02 18:21:38 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1362248454&type=1', '301293', 'Health Mate', '4040200', '2013-03-02 11:20:54 -0700', 'count', '30', '2019-06-27 22:55:10 -0700', '2013-04-23 08:05:30 -0700', '0', '4.4.2', '2013-04-23 15:06:59 +0000', 'withings-bd2://timeline/measure?user                    """"""
id=301293&date=1366729530&type=1', '301293', 'Health Mate', '4040200', '2013-04-23 08:05:30 -0700', 'count', '30.5247', '2019-06-27 22:55:10 -0700', '2013-05-09 09:49:18 -0700', '0', '4.4.2', '2013-05-09 16:50:02 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1368118158&type=1', '301293', 'Health Mate', '4040200', '2013-05-09 09:49:18 -0700', 'count', '30.4167', '2019-06-27 22:55:10 -0700', '2013-06-09 09:28:47 -0700', '0', '4.4.2', '2013-06-09 16:29:30 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1370795327&type=1', '301293', 'Health Mate', '4040200', '2013-06-09 09:28:47 -0700', 'count', '30.8333', '2019-06-27 22:55:10 -0700', '2013-07-09 08:00:17 -0700', '0', '4.4.2', '2013-07-09 15:01:00 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1373382017&type=1', '301293', 'Health Mate', '4040200', '2013-07-09 08:00:17 -0700', 'count', '30.8179', '2019-06-27 22:55:10 -0700', '2013-07-28 09:16:55 -0700', '0', '4.4.2', '2013-07-28 16:17:39 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1375028215&type=1', '301293', 'Health Mate', '4040200', '2013-07-28 09:16:55 -0700', 'count', '30.5556', '2019-06-27 22:55:10 -0700', '2013-09-13 09:22:19 -0700', '0', '4.4.2', '2013-09-13 16:23:08 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1379089339&type=1', '301293', 'Health Mate', '4040200', '2013-09-13 09:22:19 -0700', 'count', '30.9568', '2019-06-27 22:55:10 -0700', '2013-09-24 08:08:23 -0700', '0', '4.4.2', '2013-09-24 15:09:03 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1380035303&type=1', '301293', 'Health Mate', '4040200', '2013-09-24 08:08:23 -0700', 'count', '31.4352', '2019-06-27 22:55:10 -0700', '2013-10-01 08:15:13 -0700', '0', '4.4.2', '2013-10-01 15:15:57 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1380640513&type=1', '301293', 'Health Mate', '4040200', '2013-10-01 08:15:13 -0700', 'count', '31.2037', '2019-06-27 22:55:10 -0700', '2013-10-23 09:31:25 -0700', '0', '4.4.2', '2013-10-23 16:32:13 +0000', 'withings-bd2://timeline/measure?userid=301293&date=1382545885&type=1', '301293', 'Health Mate', '4040200', '2013-10-23 09:31:25 -0700', 'count', '31.8056']
```",197882382,healthkit-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/9/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
613755043,MDU6SXNzdWU2MTM3NTUwNDM=,110,Support decimal.Decimal type,134771,dvhthomas,closed,0,,,,,6,2020-05-07T03:57:19Z,2020-05-11T01:58:20Z,2020-05-11T01:50:11Z,NONE,,"Decimal types in Postgres cause a failure in db.py data type selection
---
I have a Django app using a MoneyField, which uses a `numeric(14,0)` data type in Postgres (https://www.postgresql.org/docs/9.3/datatype-numeric.html). When attempting to export that table I get the following error:

```bash
$ db-to-sqlite --table isaweb_proposal ""postgres://connection"" test.db
....
    column_type=COLUMN_TYPE_MAPPING[column_type],
KeyError: <class 'decimal.Decimal'>
```

Looking at `sql_utils.db.py` at 292-ish it's clear that there is no matching type for what I assume SQLAlchemy interprets as Python decimal.Decimal.

From the [SQLite docs](https://www.sqlite.org/datatype3.html#affinity_name_examples) it looks like DECIMAL in other DBs are considered numeric.

I'm not quite sure if it's as simple as adding a data type to that list or if there are repercussions beyond it.

Thanks for a great tool!",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/110/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
267516329,MDU6SXNzdWUyNjc1MTYzMjk=,6,Better JSON response options,9599,simonw,closed,0,,,2857392,Ship first public release,0,2017-10-23T01:18:47Z,2017-10-24T15:07:58Z,2017-10-24T15:07:58Z,OWNER,,"Default returns this:

    {
        “Columns”: [“id”, “name”, “age”],
        “Rows”: [
             [45, “Simon”, 36]
        ]
    }

.jsono instead returns a list of objects each duplicating the headers in its keys.

They both probably share the same pagination mechanism so it might not be a jsono flat list.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/6/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
708185405,MDU6SXNzdWU3MDgxODU0MDU=,975,Dependabot couldn't authenticate with https://pypi.python.org/simple/,27856297,dependabot-preview[bot],closed,0,,,,,0,2020-09-24T13:44:40Z,2020-09-25T13:34:34Z,2020-09-25T13:34:34Z,CONTRIBUTOR,,"Dependabot couldn't authenticate with https://pypi.python.org/simple/.

You can provide authentication details in your [Dependabot dashboard](https://app.dependabot.com/accounts/simonw) by clicking into the account menu (in the top right) and selecting 'Config variables'.

[View the update logs](https://app.dependabot.com/accounts/simonw/update-logs/48611311).",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/975/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1079422215,I_kwDOCGYnMM5AVq0H,357,pytest-runner is not required,4067843,pgajdos,closed,0,,,,,1,2021-12-14T07:51:24Z,2021-12-16T20:43:19Z,2021-12-16T20:43:13Z,NONE,,Deprecated pytest-runner is not necessary for running the testsuite.,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/357/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1216436131,I_kwDOBm6k_c5IgVej,1721,"Implement plugin hooks: `register_table_extras`, `register_row_extras`, `register_query_extras`",9599,simonw,open,0,,,8755003,Datasette 1.0a-next,0,2022-04-26T20:21:49Z,2022-12-13T05:29:07Z,,OWNER,,"Designed in:
- #1720

Part of:
- #262
- #1709",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1721/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
531502365,MDU6SXNzdWU1MzE1MDIzNjU=,646,Make database level information from metadata.json available in the index.html template,18017473,lagolucas,open,0,,,3268330,Datasette 1.0,3,2019-12-02T19:55:10Z,2022-03-15T20:50:34Z,,NONE,,"Did a search on the issues here and didn't find anything related to what I want.

I want to have information that is on the database level of the JSON like title, source and source_url, and use it on the index page.

I tried some small tweaks on the python and html files, but failed to get that result.

Is there a way? Thanks!",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/646/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
679646710,MDU6SXNzdWU2Nzk2NDY3MTA=,935,"db.execute_write_fn(create_tables, block=True) hangs a thread if connection fails",9599,simonw,closed,0,,,,,3,2020-08-15T21:49:17Z,2020-08-15T22:35:33Z,2020-08-15T22:35:33Z,OWNER,,Discovered in https://github.com/simonw/latest-datasette-with-all-plugins/issues/3#issuecomment-674449757,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/935/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1872043170,I_kwDOBm6k_c5vlRyi,2163,Rename core_X to catalog_X in the internals,9599,simonw,closed,0,,,,,1,2023-08-29T16:45:00Z,2023-08-29T17:01:31Z,2023-08-29T17:01:31Z,OWNER,,"Discussed with Alex this morning. We think the American spelling is fine here (it's shorter than `catalogue`) and that it's a slightly less lazy name than `core_`.

Follows:
- https://github.com/simonw/datasette/issues/2157",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2163/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
318738000,MDU6SXNzdWUzMTg3MzgwMDA=,244,/-/versions page,9599,simonw,closed,0,,,,,1,2018-04-29T18:22:15Z,2018-05-03T14:13:49Z,2018-05-03T14:09:53Z,OWNER,,"Displays the current version of:

* datasette
* Python
* SQLite
* Spatialite (if available)

Installed plugin versions should be shown on /-/plugins",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/244/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
635107393,MDU6SXNzdWU2MzUxMDczOTM=,823,"Documentation is inconsistent about ""id"" as required field on actor",9599,simonw,closed,0,,,5512395,Datasette 0.44,3,2020-06-09T04:47:58Z,2020-06-09T14:58:36Z,2020-06-09T14:58:19Z,OWNER,,"Docs at https://github.com/simonw/datasette/blob/5a6a73e3190cac103906b479d56129413e5ef190/docs/authentication.rst#actors say:

> The only required field in an actor is `""id""`, which must be a string.

But the example here returns `{""token"": token}`:

```python
@hookimpl
def actor_from_request(datasette, request):
    async def inner():
        token = request.args.get(""_token"")
        if not token:
            return None
        # Look up ?_token=xxx in sessions table
        result = await datasette.get_database().execute(
            ""select count(*) from sessions where token = ?"", [token]
        )
        if result.first()[0]:
            return {""token"": token}
        else:
            return None

    return inner
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/823/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
278189708,MDU6SXNzdWUyNzgxODk3MDg=,156,Document CSS hooks and custom templates,9599,simonw,closed,0,,,2949431,Custom templates edition,1,2017-11-30T16:43:15Z,2017-11-30T17:11:34Z,2017-11-30T17:10:58Z,OWNER,,Documentation currently lives in commit messages on https://github.com/simonw/datasette/commit/8ab3a169d42d096f2c7979c6d3d7746618d30f0b and 3cd06729f457d690603b6060dc552b535517ab09,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/156/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
314506033,MDU6SXNzdWUzMTQ1MDYwMzM=,213,Documentation for plugins system,9599,simonw,closed,0,,,,,0,2018-04-16T05:27:07Z,2018-04-16T15:12:48Z,2018-04-16T15:12:48Z,OWNER,,"Documentation for #14 - how to write plugins, how to ship plugins to PyPI and how to use the `--plugins-dir` option added in #211 ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/213/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
665403403,MDU6SXNzdWU2NjU0MDM0MDM=,907,Allow documentation doesn't explain what happens with multiple allow keys,9599,simonw,closed,0,,,5607421,Datasette 0.46,2,2020-07-24T20:34:40Z,2020-07-24T22:53:07Z,2020-07-24T22:53:07Z,OWNER,,"Documentation here: https://datasette.readthedocs.io/en/0.45/authentication.html#defining-permissions-with-allow-blocks

Doesn't explain that with the following ""allow"" block:
```json
{
  ""allow"": {
    ""id"": ""simonw"",
    ""role"": ""staff""
  }
}
```
The rule will match if EITHER the id is simonw OR the role includes staff.

The tests are missing this case too: https://github.com/simonw/datasette/blob/028f193dd6233fa116262ab4b07b13df7dcec9be/tests/test_utils.py#L504

Related to #906",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/907/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1054244712,I_kwDOBm6k_c4-1n9o,1510,Datasette 1.0 documented template context (maybe via API docs),9599,simonw,open,0,,,3268330,Datasette 1.0,3,2021-11-15T23:23:58Z,2023-06-28T02:05:21Z,,OWNER,,Documented context plus protective unit tests. Goal is that custom templates built for 1.x will not break without a 2.x release.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1510/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
451513541,MDU6SXNzdWU0NTE1MTM1NDE=,498,Full text search of all tables at once?,7936571,chrismp,closed,0,,,,,12,2019-06-03T14:24:43Z,2020-05-30T17:26:02Z,2020-05-30T17:26:02Z,NONE,,"Does datasette have a built-in way, in a browser, to do a full-text search of all columns, in all databases and tables, that have full-text search enabled? Is there a plugin that does this?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/498/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
841456306,MDU6SXNzdWU4NDE0NTYzMDY=,1276,"Invalid SQL: ""no such table: pragma_database_list"" on database page",1314318,justinallen,closed,0,,,,,7,2021-03-26T00:03:53Z,2021-03-31T16:27:27Z,2021-03-28T23:52:31Z,NONE,,"Don't think this has been covered here yet. I'm a little stumped with this one and can't tell if it's a bug or I have something misconfigured. 

Oddly, when running locally the usual list of tables populates (i.e. at /charts a list of tables in charts.db). But when on the web server it throws an Invalid SQL error and ""no such table: pragma_database_list"" below. 

All the url endpoints seem to work fine aside from this - individual tables (/charts/chart_one), as well as stored queries (/charts/query_one). 

Not sure if this has anything to do with upgrading to Datasette 0.55, or something to do with our setup, which uses a metadata build script similar to [the one for the 538 server](https://github.com/simonw/fivethirtyeight-datasette/blob/main/make_metadata.py), or something else. 
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1276/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1375792876,I_kwDOBm6k_c5SAO7s,1811,"Drop-down menu with ""REGEXP"" choice",562352,CharlesNepote,open,0,,,,,0,2022-09-16T11:06:18Z,2022-09-16T15:30:31Z,,NONE,,"Drop-down menu below could add ""REGEXP"" choice when REGEXP sqlite extension is installed and used
![image](https://user-images.githubusercontent.com/562352/190675352-810fbdca-0827-4034-8b9f-fd67d5c35afb.png)

Not sure. Close the issue if you don't find it relevant.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1811/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
456569067,MDU6SXNzdWU0NTY1NjkwNjc=,510,Ability to facet by delimiter (e.g. comma separated fields),9599,simonw,open,0,9599,simonw,,,1,2019-06-15T19:34:41Z,2019-07-08T15:44:51Z,,OWNER,,"E.g. if a field contains ""Tags,With,Commas"" be able to facet them in the same way as `_facet_array=` lets you facet `[""Tags"", ""With"", ""Commas""]`",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/510/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
530491074,MDU6SXNzdWU1MzA0OTEwNzQ=,14,Command for importing events,9599,simonw,open,0,,,,,3,2019-11-29T21:28:58Z,2020-04-14T19:38:34Z,,MEMBER,,"Eg from https://api.github.com/users/simonw/events

Docs here: https://developer.github.com/v3/activity/events/#list-events-performed-by-a-user",207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/14/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1524983536,I_kwDOBm6k_c5a5Wbw,1981,Canned query field labels truncated,9599,simonw,open,0,,,,,1,2023-01-09T06:04:24Z,2023-01-09T06:05:44Z,,OWNER,,"Eg here on mobile: https://timezones.datasette.io/timezones/by_point?longitude=-0.1406632&latitude=50.8246776

![107A1894-D1DA-4158-9EA3-40C840DD10E3](https://user-images.githubusercontent.com/9599/211248895-c922ce61-95d3-47ca-9314-dcff7c86afab.jpeg)
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1981/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
275175929,MDU6SXNzdWUyNzUxNzU5Mjk=,132,Row view is not currently expanding foreign keys,9599,simonw,closed,0,,,2919870,Foreign key edition,1,2017-11-19T17:24:25Z,2017-11-23T21:51:51Z,2017-11-23T21:51:30Z,OWNER,,Eg https://sf-trees.now.sh/sf-trees-ebc2ad9/Street_Tree_List/1,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/132/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
275176006,MDU6SXNzdWUyNzUxNzYwMDY=,133,"If view is filtered, search should apply within those filtered rows",9599,simonw,closed,0,,,2919870,Foreign key edition,3,2017-11-19T17:25:36Z,2017-11-24T22:30:32Z,2017-11-24T22:30:15Z,OWNER,,Eg on https://sf-trees.now.sh/sf-trees-ebc2ad9/Street_Tree_List?qSpecies=1,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/133/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1493471221,I_kwDOBm6k_c5ZBI_1,1949,`.json` errors should be returned as JSON,9599,simonw,open,0,,,8755003,Datasette 1.0a-next,10,2022-12-13T06:14:12Z,2022-12-15T00:46:27Z,,OWNER,,"Eg the error in this issue:
- #1945 ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1949/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
847423559,MDU6SXNzdWU4NDc0MjM1NTk=,253,fixtures.db example error in sql-utils blog post,192568,mroswell,closed,0,,,,,2,2021-03-31T22:07:36Z,2021-05-19T03:31:48Z,2021-05-19T03:31:47Z,NONE,,"En route to trying to understand column order transform documentation, I tried the instructions here:
https://simonwillison.net/2020/Sep/23/sqlite-advanced-alter-table/
I get a malformed database schema syntax error.

```
$ wget https://latest.datasette.io/fixtures.db
--2021-03-31 18:00:23--  https://latest.datasette.io/fixtures.db
Resolving latest.datasette.io (latest.datasette.io)... 2607:f8b0:4004:801::2013, 142.250.73.211
Connecting to latest.datasette.io (latest.datasette.io)|2607:f8b0:4004:801::2013|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [application/octet-stream]
Saving to: ‘fixtures.db’

fixtures.db                                                             [ <=>                                                                                                                                                              ] 260.00K  --.-KB/s    in 0.1s

2021-03-31 18:00:23 (2.41 MB/s) - ‘fixtures.db’ saved [266240]

$ sqlite3 fixtures.db '.schema facetable'
Error: malformed database schema (generated_columns) - near ""AS"": syntax error

$ sqlite3 fixtures.db
SQLite version 3.28.0 2019-04-15 14:49:49
Enter "".help"" for usage hints.
sqlite> .schema
Error: malformed database schema (generated_columns) - near ""AS"": syntax error
```
",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/253/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1432013704,I_kwDOBm6k_c5VWsuI,1878,/db/table/-/upsert API,9599,simonw,closed,0,,,8711695, Datasette 1.0a2,8,2022-11-01T20:01:18Z,2022-12-08T01:12:18Z,2022-12-08T01:12:17Z,OWNER,,Equivalent to `sqlite-utils upsert`: https://sqlite-utils.datasette.io/en/stable/python-api.html#upserting-data,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1878/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1470509936,I_kwDOBm6k_c5XpjNw,1924,Docs for replace:true and ignore:true options for insert API,9599,simonw,closed,0,,,7867486,Datasette 1.0a1,4,2022-12-01T01:33:25Z,2022-12-01T18:15:15Z,2022-12-01T02:08:02Z,OWNER,,Equivalent to https://sqlite-utils.datasette.io/en/stable/cli.html#insert-replacing-data,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1924/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
803356942,MDU6SXNzdWU4MDMzNTY5NDI=,1218, /usr/local/opt/python3/bin/python3.6: bad interpreter: No such file or directory,11855322,robmarkcole,open,0,,,,,1,2021-02-08T09:07:00Z,2021-02-23T12:12:17Z,,NONE,,"Error as above, however I do have python3.8 and the readme indicates this is supported.

```
(venv) (base) Robins-MacBook:datasette robin$ ls /usr/local/opt/python3/bin/

.. pip3 python3 python3.8
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1218/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
944903881,MDU6SXNzdWU5NDQ5MDM4ODE=,1396,"""invalid reference format"" publishing Docker image",9599,simonw,closed,0,,,,,9,2021-07-15T01:02:07Z,2021-10-19T08:10:26Z,2021-07-15T19:47:25Z,OWNER,,"Error ocurred at the end of the publish flow for Datasette 0.58: https://github.com/simonw/datasette/runs/3072216421
```
Removing intermediate container cf32b9440907
 ---> dfd6985b2afc
Successfully built dfd6985b2afc
Successfully tagged ***/datasette:0.58
invalid reference format
Error: Process completed with exit code 1.
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1396/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
677272618,MDU6SXNzdWU2NzcyNzI2MTg=,928,Test failures caused by failed attempts to mock pip,9599,simonw,closed,0,,,,,4,2020-08-11T23:53:18Z,2022-02-23T16:19:47Z,2020-08-12T00:07:49Z,OWNER,,"Errors like this one:

https://github.com/simonw/datasette/pull/927/checks?check_run_id=973559696

```
2020-08-11T23:36:39.8801334Z =================================== FAILURES ===================================
2020-08-11T23:36:39.8802411Z _________________________________ test_install _________________________________
2020-08-11T23:36:39.8803242Z 
2020-08-11T23:36:39.8804935Z thing = <module 'pip._internal.cli' from '/opt/hostedtoolcache/Python/3.8.5/x64/lib/python3.8/site-packages/pip/_internal/cli/__init__.py'>
2020-08-11T23:36:39.8806663Z comp = 'main', import_path = 'pip._internal.cli.main'
2020-08-11T23:36:39.8807696Z 
2020-08-11T23:36:39.8808728Z     def _dot_lookup(thing, comp, import_path):
2020-08-11T23:36:39.8810573Z         try:
2020-08-11T23:36:39.8812262Z >           return getattr(thing, comp)
2020-08-11T23:36:39.8817136Z E           AttributeError: module 'pip._internal.cli' has no attribute 'main'
2020-08-11T23:36:39.8843043Z 
2020-08-11T23:36:39.8855951Z /opt/hostedtoolcache/Python/3.8.5/x64/lib/python3.8/unittest/mock.py:1215: AttributeError
2020-08-11T23:36:39.8873372Z 
2020-08-11T23:36:39.8877803Z During handling of the above exception, another exception occurred:
2020-08-11T23:36:39.8906532Z 
2020-08-11T23:36:39.8925767Z     def get_src_prefix():
2020-08-11T23:36:39.8928277Z         # type: () -> str
2020-08-11T23:36:39.8930068Z         if running_under_virtualenv():
2020-08-11T23:36:39.8949721Z             src_prefix = os.path.join(sys.prefix, 'src')
2020-08-11T23:36:39.8951813Z         else:
2020-08-11T23:36:39.8969014Z             # FIXME: keep src in cwd for now (it is not a temporary folder)
2020-08-11T23:36:39.9012110Z             try:
2020-08-11T23:36:39.9013489Z >               src_prefix = os.path.join(os.getcwd(), 'src')
2020-08-11T23:36:39.9014538Z E               FileNotFoundError: [Errno 2] No such file or directory
2020-08-11T23:36:39.9016122Z 
2020-08-11T23:36:39.9017617Z /opt/hostedtoolcache/Python/3.8.5/x64/lib/python3.8/site-packages/pip/_internal/locations.py:50: FileNotFoundError
2020-08-11T23:36:39.9018802Z 
2020-08-11T23:36:39.9020070Z During handling of the above exception, another exception occurred:
2020-08-11T23:36:39.9020930Z 
2020-08-11T23:36:39.9022275Z args = (), keywargs = {}
2020-08-11T23:36:39.9023183Z 
2020-08-11T23:36:39.9024077Z     @wraps(func)
2020-08-11T23:36:39.9024984Z     def patched(*args, **keywargs):
2020-08-11T23:36:39.9028770Z >       with self.decoration_helper(patched,
2020-08-11T23:36:39.9031861Z                                     args,
2020-08-11T23:36:39.9038358Z                                     keywargs) as (newargs, newkeywargs):
2020-08-11T23:36:39.9039654Z 
2020-08-11T23:36:39.9040566Z /opt/hostedtoolcache/Python/3.8.5/x64/lib/python3.8/unittest/mock.py:1322: 
2020-08-11T23:36:39.9041492Z _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/928/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
280014287,MDU6SXNzdWUyODAwMTQyODc=,165,metadata.json support for per-database and per-table information,9599,simonw,closed,0,,,2949431,Custom templates edition,2,2017-12-07T06:15:34Z,2017-12-07T16:48:34Z,2017-12-07T16:47:29Z,OWNER,,"Every database and every table should be able to support the following optional metadata:

	title
	description
	description_html
	license
	license_url
	source
	source_url

If `description_html` is provided it over-rides `description` and will be displayed unescaped.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/165/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
267516650,MDU6SXNzdWUyNjc1MTY2NTA=,7,Framework where by every page is JSON plus a template,9599,simonw,closed,0,,,2857392,Ship first public release,1,2017-10-23T01:22:03Z,2017-10-24T02:27:25Z,2017-10-24T02:27:25Z,OWNER,,"Every single page of my interface should be implemented as a function that returns JSON.

I can then build my jinja templates on top of the exact data that would be returned by the API version.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/7/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1087931918,I_kwDOBm6k_c5A2IYO,1579,`.execute_write(... block=True)` should be the default behaviour,9599,simonw,closed,0,,,7571612,Datasette 0.60,7,2021-12-23T18:54:28Z,2022-01-13T22:28:08Z,2021-12-23T19:18:26Z,OWNER,,"Every single piece of code I've written against the write APIs has used the `block=True` option to wait for the result.

Without that, it instead fires the write into the queue but then continues even before it has finished executing.

`block=True` should clearly be the default behaviour here!",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1579/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
632918799,MDU6SXNzdWU2MzI5MTg3OTk=,808,Permission check for every view in Datasette (plus docs),9599,simonw,closed,0,,,5512395,Datasette 0.44,2,2020-06-07T01:59:23Z,2020-06-07T05:30:49Z,2020-06-07T05:30:49Z,OWNER,,"Every view in Datasette should perform a permission check to see if the current user/actor is allowed to view that page.

This permission check will default to allowed, but having this check will allow plugins to lock down access selectively or even to everything in a Datasette instance.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/808/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
280744309,MDU6SXNzdWUyODA3NDQzMDk=,169,Release v0.14 with templates and static files features,9599,simonw,closed,0,,,2949431,Custom templates edition,1,2017-12-09T18:52:48Z,2017-12-10T02:04:56Z,2017-12-10T02:04:56Z,OWNER,,"Everything in this milestone https://github.com/simonw/datasette/milestone/6 - plus various other fixes: https://github.com/simonw/datasette/compare/0.13...6bdfcf60760c27e29ff34692d06e62b36aeecc56
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/169/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1099897648,I_kwDOCGYnMM5Bjxsw,384,Add examples to every `--help`,9599,simonw,closed,0,,,,,0,2022-01-12T05:31:25Z,2022-01-26T03:15:02Z,2022-01-26T03:15:02Z,OWNER,,Everything on https://sqlite-utils.datasette.io/en/stable/cli-reference.html would benefit from an example.,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/384/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
957345476,MDU6SXNzdWU5NTczNDU0NzY=,1411,Canned query ?sql= is pointlessly echoed in query string starting from hidden mode,9599,simonw,closed,0,,,,,1,2021-08-01T00:17:13Z,2021-08-01T03:27:30Z,2021-08-01T00:58:17Z,OWNER,,"Example: https://latest.datasette.io/fixtures/neighborhood_search?text=cork&_hide_sql=1

Submitting that form again results in this:

https://latest.datasette.io/fixtures/neighborhood_search?sql=%0D%0Aselect+neighborhood%2C+facet_cities.name%2C+state%0D%0Afrom+facetable%0D%0A++++join+facet_cities%0D%0A++++++++on+facetable.city_id+%3D+facet_cities.id%0D%0Awhere+neighborhood+like+%27%25%27+%7C%7C+%3Atext+%7C%7C+%27%25%27%0D%0Aorder+by+neighborhood%3B%0D%0A&_hide_sql=1&text=cork

Because the HTML on https://latest.datasette.io/fixtures/neighborhood_search?text=cork&_hide_sql=1 includes this:

```html
    <h3>Custom SQL query returning 1 row <span class=""show-hide-sql"">(<a href=""/fixtures/neighborhood_search?text=cork"">show</a>)</span></h3>
    
    
        <input type=""hidden"" name=""sql"" value=""
select neighborhood, facet_cities.name, state
from facetable
    join facet_cities
        on facetable.city_id = facet_cities.id
where neighborhood like &#39;%&#39; || :text || &#39;%&#39;
order by neighborhood;
"">
        <input type=""hidden"" name=""_hide_sql"" value=""1"">
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1411/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
546051181,MDU6SXNzdWU1NDYwNTExODE=,16,Exception running first command: IndexError: list index out of range,15092,jayvdb,closed,0,,,,,4,2020-01-07T03:01:58Z,2020-04-14T18:37:21Z,2020-04-14T18:37:21Z,NONE,,"Exception running first command without an existing db or auth.

```py
> mkdir ~/.github/coala
> /usr/bin/github-to-sqlite repos ~/.github/coala coala
Traceback (most recent call last):
  File ""/usr/bin/github-to-sqlite"", line 11, in <module>
    load_entry_point('github-to-sqlite==0.6', 'console_scripts', 'github-to-sqlite')()
  File ""/usr/lib/python3.7/site-packages/click/core.py"", line 764, in __call__
    return self.main(*args, **kwargs)
  File ""/usr/lib/python3.7/site-packages/click/core.py"", line 717, in main
    rv = self.invoke(ctx)
  File ""/usr/lib/python3.7/site-packages/click/core.py"", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File ""/usr/lib/python3.7/site-packages/click/core.py"", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File ""/usr/lib/python3.7/site-packages/click/core.py"", line 555, in invoke
    return callback(*args, **kwargs)
  File ""/usr/lib/python3.7/site-packages/github_to_sqlite/cli.py"", line 163, in repos
    utils.save_repo(db, repo)
  File ""/usr/lib/python3.7/site-packages/github_to_sqlite/utils.py"", line 120, in save_repo
    to_save[""owner""] = save_user(db, to_save[""owner""])
  File ""/usr/lib/python3.7/site-packages/github_to_sqlite/utils.py"", line 61, in save_user
    return db[""users""].upsert(to_save, pk=""id"", alter=True).last_pk
  File ""/usr/lib/python3.7/site-packages/sqlite_utils/db.py"", line 1135, in upsert
    extracts=extracts,
  File ""/usr/lib/python3.7/site-packages/sqlite_utils/db.py"", line 1162, in upsert_all
    upsert=True,
  File ""/usr/lib/python3.7/site-packages/sqlite_utils/db.py"", line 1105, in insert_all
    row = list(self.rows_where(""rowid = ?"", [self.last_rowid]))[0]
IndexError: list index out of range
```",207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/16/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1173023272,I_kwDOCGYnMM5F6uoo,416,Options for how `r.parsedate()` should handle invalid dates,638427,mattkiefer,closed,0,,,,,11,2022-03-17T23:29:55Z,2022-05-03T21:36:49Z,2022-03-21T04:01:39Z,NONE,,"Exceptions are normal expected behavior when typecasting an invalid format. However, r.parsedate() is really just re-formatting strings and keeping the type as text. So it may be better to print-and-pass on exception so the user can see a complete list of invalid values -- while also allowing for the parser to reformat the remaining valid values. 
```
sqlite-utils convert idfpr.db license ""Expiration Date"" ""r.parsedate(value)""
  [#######-----------------------------]   21%  00:01:57Traceback (most recent call last):
  File ""/usr/local/lib/python3.9/dist-packages/sqlite_utils/db.py"", line 2336, in convert_value
    return fn(v)
  File ""<string>"", line 2, in fn
  File ""/usr/local/lib/python3.9/dist-packages/sqlite_utils/recipes.py"", line 8, in parsedate
    parser.parse(value, dayfirst=dayfirst, yearfirst=yearfirst).date().isoformat()
  File ""/usr/lib/python3/dist-packages/dateutil/parser/_parser.py"", line 1374, in parse
    return DEFAULTPARSER.parse(timestr, **kwargs)
  File ""/usr/lib/python3/dist-packages/dateutil/parser/_parser.py"", line 652, in parse
    raise ParserError(""String does not contain a date: %s"", timestr)
dateutil.parser._parser.ParserError: String does not contain a date:   /  /    
```
In this case, I had just one variation of an invalid date: '  /  /  '. But theoretically there could be many values that would have to be fixed one at a time with the current exception handling. ",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/416/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1857851384,I_kwDOCGYnMM5uvI_4,587,New .add_foreign_key() can break if PRAGMA legacy_alter_table=ON and there's an invalid foreign key reference,9599,simonw,closed,0,,,,,3,2023-08-19T20:01:26Z,2023-08-19T20:04:33Z,2023-08-19T20:04:32Z,OWNER,,"Extremely detailed story of how I got to this point:

- https://github.com/simonw/llm/issues/162

Steps to reproduce (only if that pragma is on though):
```bash
python -c '
import sqlite_utils
db = sqlite_utils.Database(memory=True)
db.execute(""""""
CREATE TABLE ""logs"" (
   [id] INTEGER PRIMARY KEY,
   [model] TEXT,
   [prompt] TEXT,
   [system] TEXT,
   [prompt_json] TEXT,
   [options_json] TEXT,
   [response] TEXT,
   [response_json] TEXT,
   [reply_to_id] INTEGER,
   [chat_id] INTEGER REFERENCES [log]([id]),
   [duration_ms] INTEGER,
   [datetime_utc] TEXT
);
"""""")
db[""logs""].add_foreign_key(""reply_to_id"", ""logs"", ""id"")
'
```
This succeeds in some environments, fails in others.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/587/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1616440856,I_kwDOJHON9s5gWO4Y,5,Configure full text search,9599,simonw,open,0,,,,,0,2023-03-09T05:20:46Z,2023-03-09T05:20:46Z,,MEMBER,,"FTS would be useful.

Maybe even extract the plain text from the notes to make that index easier to create, rather than creating it against the HTML. Can use the `plaintext` property for that.",611552758,apple-notes-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/5/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
531583658,MDU6SXNzdWU1MzE1ODM2NTg=,68,Add support for porter stemming in FTS,9599,simonw,closed,0,,,,,1,2019-12-02T22:35:52Z,2020-09-20T04:25:53Z,2020-09-20T04:25:47Z,OWNER,,FTS5 can have porter stemming enabled.,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/68/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
671130371,MDU6SXNzdWU2NzExMzAzNzE=,130,Support tokenize option for FTS,9599,simonw,closed,0,,,,,3,2020-08-01T19:27:22Z,2020-08-01T20:51:28Z,2020-08-01T20:51:14Z,OWNER,,"FTS5 supports things like porter stemming using a `tokenize=` option:

https://www.sqlite.org/fts5.html#tokenizers

Something like this in code:
```
            CREATE VIRTUAL TABLE [{table}_fts] USING {fts_version} (
                {columns},
                tokenize='porter',
                content=[{table}]
            );
```
I tried this out just now and it worked exactly as expected.

So... `db[table].enable_fts(...) should accept a 'tokenize=` argument, and `sqlite-utils enable-fts ...` should support a `--tokenize` option.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/130/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
602533300,MDU6SXNzdWU2MDI1MzMzMDA=,1,Import photo metadata from Apple Photos into SQLite,9599,simonw,open,0,,,5324096,Apple Photos online and securely browsable,8,2020-04-18T19:23:26Z,2020-05-04T02:41:40Z,,MEMBER,,"Faces, albums, locations, that kind of thing.",256834907,dogsheep-photos,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-photos/issues/1/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
447408527,MDU6SXNzdWU0NDc0MDg1Mjc=,483,Option to facet by date using month or year,9599,simonw,open,0,,,,,5,2019-05-23T01:25:29Z,2019-05-29T21:38:27Z,,OWNER,,"Facet by date (from #481) can take datetimes and facet them by the day component.

https://latest.datasette.io/fixtures/facetable?_facet_date=created

I'd like to also be able to facet by month or year.

I'm not sure what the best way to achieve this is. Could be two more Facet classes (YearFacet and MonthFacet) but I think it might be nicer if the existing DateFacet could take an optional argument that changed its behaviour. But... if I do that, do I expose it in the UI somewhere or is it only available to URL-hackers?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/483/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
699947574,MDU6SXNzdWU2OTk5NDc1NzQ=,963,Currently selected array facets are not correctly persisted through hidden form fields,649467,mhalle,closed,0,,,5818042,Datasette 0.49,1,2020-09-12T01:49:17Z,2020-09-12T21:54:29Z,2020-09-12T21:54:09Z,NONE,,"Faceted search uses JSON array elements as facets rather than the arrays. However, if a search is ""Apply""ed (using the Apply button), the array itself rather than its elements used. 

To reproduce:
https://latest.datasette.io/fixtures/facetable?_sort=pk&_facet=created&_facet=tags&_facet_array=tags

Press ""Apply"", which might be done when removing a filter. Notice that the ""tags"" facet values are now arrays, not array elements. It appears the ""&_facet_array=tags"" element of the query string is dropped.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/963/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
814591962,MDU6SXNzdWU4MTQ1OTE5NjI=,1240,Allow facetting on custom queries,7107523,Kabouik,closed,0,,,,,3,2021-02-23T15:52:19Z,2021-02-26T18:19:46Z,2021-02-26T18:18:18Z,NONE,,"Facets are a tremendously useful feature, especially for people peeking at the database for the first time and still having little knowledge about the details of the data. It is of great assistance to discover interesting features to explore futher in advanced queries.

Yet, it seems it's impossible to use facets when running a custom SQL query, be it from the little gear icons in column names, the facet suggestions at the top (hidden when performing a custom query), or by appending a facet code to the URL. 

Is there a technical limitation, or is this something that could be unlocked easily?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1240/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
994390593,MDU6SXNzdWU5OTQzOTA1OTM=,1468,Faceting for custom SQL queries,72577720,MichaelTiemannOSC,closed,0,,,,,2,2021-09-13T02:52:16Z,2021-09-13T04:54:22Z,2021-09-13T04:54:17Z,CONTRIBUTOR,,"Facets are awesome.  But not when I need to join to tidy tables together.  Or even just running explicitly the default SQL query that simply lists all the rows and columns of a table (up to SIZE).  That is to say, when I browse a table, I see facets:

https://latest.datasette.io/fixtures/compound_three_primary_keys

But when I run a custom query, I don't:

https://latest.datasette.io/fixtures?sql=select+pk1%2C+pk2%2C+pk3%2C+content+from+compound_three_primary_keys+order+by+pk1%2C+pk2%2C+pk3+limit+101

Is there an idiom to cause custom SQL to come back with facet suggestions?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1468/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1422973111,I_kwDOBm6k_c5U0Ni3,1854,Flaky test: test_serve_localhost_http,9599,simonw,closed,0,,,,,3,2022-10-25T19:37:35Z,2022-10-25T19:53:02Z,2022-10-25T19:53:02Z,OWNER,,Failing on Python 3.10 at the moment: https://github.com/simonw/datasette/actions/runs/3323629947/jobs/5494340302,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1854/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1413641049,I_kwDOCGYnMM5UQnNZ,501,Tests failing due to updated tabulate library,9599,simonw,closed,0,,,,,4,2022-10-18T18:07:52Z,2022-10-18T18:23:40Z,2022-10-18T18:23:40Z,OWNER,,"Failure here: https://github.com/simonw/sqlite-utils/actions/runs/3275786702/jobs/5391063221

I figured out the problem:

```diff
diff --git a/docs/cli-reference.rst b/docs/cli-reference.rst
index b88e38a..82b4b6c 100644
--- a/docs/cli-reference.rst
+++ b/docs/cli-reference.rst
@@ -112,11 +112,15 @@ See :ref:`cli_query`.
       --tsv                       Output TSV
       --no-headers                Omit CSV headers
       -t, --table                 Output as a formatted table
-      --fmt TEXT                  Table format - one of fancy_grid, fancy_outline,
-                                  github, grid, html, jira, latex, latex_booktabs,
-                                  latex_longtable, latex_raw, mediawiki, moinmoin,
-                                  orgtbl, pipe, plain, presto, pretty, psql, rst,
-                                  simple, textile, tsv, unsafehtml, youtrack
+      --fmt TEXT                  Table format - one of asciidoc, double_grid,
+                                  double_outline, fancy_grid, fancy_outline, github,
+                                  grid, heavy_grid, heavy_outline, html, jira,
+                                  latex, latex_booktabs, latex_longtable, latex_raw,
+                                  mediawiki, mixed_grid, mixed_outline, moinmoin,
+                                  orgtbl, outline, pipe, plain, presto, pretty,
+                                  psql, rounded_grid, rounded_outline, rst, simple,
+                                  simple_grid, simple_outline, textile, tsv,
+                                  unsafehtml, youtrack
       --json-cols                 Detect JSON cols and output them as JSON, not
                                   escaped strings
       -r, --raw                   Raw output, first column of first row
@@ -176,11 +180,15 @@ See :ref:`cli_memory`.
       --tsv                       Output TSV
       --no-headers                Omit CSV headers
       -t, --table                 Output as a formatted table
-      --fmt TEXT                  Table format - one of fancy_grid, fancy_outline,
-                                  github, grid, html, jira, latex, latex_booktabs,
-                                  latex_longtable, latex_raw, mediawiki, moinmoin,
-                                  orgtbl, pipe, plain, presto, pretty, psql, rst,
-                                  simple, textile, tsv, unsafehtml, youtrack
+      --fmt TEXT                  Table format - one of asciidoc, double_grid,
+                                  double_outline, fancy_grid, fancy_outline, github,
+                                  grid, heavy_grid, heavy_outline, html, jira,
+                                  latex, latex_booktabs, latex_longtable, latex_raw,
+                                  mediawiki, mixed_grid, mixed_outline, moinmoin,
+                                  orgtbl, outline, pipe, plain, presto, pretty,
+                                  psql, rounded_grid, rounded_outline, rst, simple,
+                                  simple_grid, simple_outline, textile, tsv,
+                                  unsafehtml, youtrack
       --json-cols                 Detect JSON cols and output them as JSON, not
                                   escaped strings
       -r, --raw                   Raw output, first column of first row
@@ -401,11 +409,14 @@ See :ref:`cli_search`.
       --tsv                  Output TSV
       --no-headers           Omit CSV headers
       -t, --table            Output as a formatted table
-      --fmt TEXT             Table format - one of fancy_grid, fancy_outline,
-                             github, grid, html, jira, latex, latex_booktabs,
-                             latex_longtable, latex_raw, mediawiki, moinmoin,
-                             orgtbl, pipe, plain, presto, pretty, psql, rst, simple,
-                             textile, tsv, unsafehtml, youtrack
+      --fmt TEXT             Table format - one of asciidoc, double_grid,
+                             double_outline, fancy_grid, fancy_outline, github,
+                             grid, heavy_grid, heavy_outline, html, jira, latex,
+                             latex_booktabs, latex_longtable, latex_raw, mediawiki,
+                             mixed_grid, mixed_outline, moinmoin, orgtbl, outline,
+                             pipe, plain, presto, pretty, psql, rounded_grid,
+                             rounded_outline, rst, simple, simple_grid,
+                             simple_outline, textile, tsv, unsafehtml, youtrack
       --json-cols            Detect JSON cols and output them as JSON, not escaped
                              strings
       --load-extension TEXT  Path to SQLite extension, with optional :entrypoint
@@ -651,11 +662,14 @@ See :ref:`cli_tables`.
       --tsv                  Output TSV
       --no-headers           Omit CSV headers
       -t, --table            Output as a formatted table
-      --fmt TEXT             Table format - one of fancy_grid, fancy_outline,
-                             github, grid, html, jira, latex, latex_booktabs,
-                             latex_longtable, latex_raw, mediawiki, moinmoin,
-                             orgtbl, pipe, plain, presto, pretty, psql, rst, simple,
-                             textile, tsv, unsafehtml, youtrack
+      --fmt TEXT             Table format - one of asciidoc, double_grid,
+                             double_outline, fancy_grid, fancy_outline, github,
+                             grid, heavy_grid, heavy_outline, html, jira, latex,
+                             latex_booktabs, latex_longtable, latex_raw, mediawiki,
+                             mixed_grid, mixed_outline, moinmoin, orgtbl, outline,
+                             pipe, plain, presto, pretty, psql, rounded_grid,
+                             rounded_outline, rst, simple, simple_grid,
+                             simple_outline, textile, tsv, unsafehtml, youtrack
       --json-cols            Detect JSON cols and output them as JSON, not escaped
                              strings
       --columns              Include list of columns for each table
@@ -689,11 +703,14 @@ See :ref:`cli_views`.
       --tsv                  Output TSV
       --no-headers           Omit CSV headers
       -t, --table            Output as a formatted table
-      --fmt TEXT             Table format - one of fancy_grid, fancy_outline,
-                             github, grid, html, jira, latex, latex_booktabs,
-                             latex_longtable, latex_raw, mediawiki, moinmoin,
-                             orgtbl, pipe, plain, presto, pretty, psql, rst, simple,
-                             textile, tsv, unsafehtml, youtrack
+      --fmt TEXT             Table format - one of asciidoc, double_grid,
+                             double_outline, fancy_grid, fancy_outline, github,
+                             grid, heavy_grid, heavy_outline, html, jira, latex,
+                             latex_booktabs, latex_longtable, latex_raw, mediawiki,
+                             mixed_grid, mixed_outline, moinmoin, orgtbl, outline,
+                             pipe, plain, presto, pretty, psql, rounded_grid,
+                             rounded_outline, rst, simple, simple_grid,
+                             simple_outline, textile, tsv, unsafehtml, youtrack
       --json-cols            Detect JSON cols and output them as JSON, not escaped
                              strings
       --columns              Include list of columns for each view
@@ -732,11 +749,15 @@ See :ref:`cli_rows`.
       --tsv                       Output TSV
       --no-headers                Omit CSV headers
       -t, --table                 Output as a formatted table
-      --fmt TEXT                  Table format - one of fancy_grid, fancy_outline,
-                                  github, grid, html, jira, latex, latex_booktabs,
-                                  latex_longtable, latex_raw, mediawiki, moinmoin,
-                                  orgtbl, pipe, plain, presto, pretty, psql, rst,
-                                  simple, textile, tsv, unsafehtml, youtrack
+      --fmt TEXT                  Table format - one of asciidoc, double_grid,
+                                  double_outline, fancy_grid, fancy_outline, github,
+                                  grid, heavy_grid, heavy_outline, html, jira,
+                                  latex, latex_booktabs, latex_longtable, latex_raw,
+                                  mediawiki, mixed_grid, mixed_outline, moinmoin,
+                                  orgtbl, outline, pipe, plain, presto, pretty,
+                                  psql, rounded_grid, rounded_outline, rst, simple,
+                                  simple_grid, simple_outline, textile, tsv,
+                                  unsafehtml, youtrack
       --json-cols                 Detect JSON cols and output them as JSON, not
                                   escaped strings
       --load-extension TEXT       Path to SQLite extension, with optional
@@ -768,11 +789,14 @@ See :ref:`cli_triggers`.
       --tsv                  Output TSV
       --no-headers           Omit CSV headers
       -t, --table            Output as a formatted table
-      --fmt TEXT             Table format - one of fancy_grid, fancy_outline,
-                             github, grid, html, jira, latex, latex_booktabs,
-                             latex_longtable, latex_raw, mediawiki, moinmoin,
-                             orgtbl, pipe, plain, presto, pretty, psql, rst, simple,
-                             textile, tsv, unsafehtml, youtrack
+      --fmt TEXT             Table format - one of asciidoc, double_grid,
+                             double_outline, fancy_grid, fancy_outline, github,
+                             grid, heavy_grid, heavy_outline, html, jira, latex,
+                             latex_booktabs, latex_longtable, latex_raw, mediawiki,
+                             mixed_grid, mixed_outline, moinmoin, orgtbl, outline,
+                             pipe, plain, presto, pretty, psql, rounded_grid,
+                             rounded_outline, rst, simple, simple_grid,
+                             simple_outline, textile, tsv, unsafehtml, youtrack
       --json-cols            Detect JSON cols and output them as JSON, not escaped
                              strings
       --load-extension TEXT  Path to SQLite extension, with optional :entrypoint
@@ -804,11 +828,14 @@ See :ref:`cli_indexes`.
       --tsv                  Output TSV
       --no-headers           Omit CSV headers
       -t, --table            Output as a formatted table
-      --fmt TEXT             Table format - one of fancy_grid, fancy_outline,
-                             github, grid, html, jira, latex, latex_booktabs,
-                             latex_longtable, latex_raw, mediawiki, moinmoin,
-                             orgtbl, pipe, plain, presto, pretty, psql, rst, simple,
-                             textile, tsv, unsafehtml, youtrack
+      --fmt TEXT             Table format - one of asciidoc, double_grid,
+                             double_outline, fancy_grid, fancy_outline, github,
+                             grid, heavy_grid, heavy_outline, html, jira, latex,
+                             latex_booktabs, latex_longtable, latex_raw, mediawiki,
+                             mixed_grid, mixed_outline, moinmoin, orgtbl, outline,
+                             pipe, plain, presto, pretty, psql, rounded_grid,
+                             rounded_outline, rst, simple, simple_grid,
+                             simple_outline, textile, tsv, unsafehtml, youtrack
       --json-cols            Detect JSON cols and output them as JSON, not escaped
                              strings
       --load-extension TEXT  Path to SQLite extension, with optional :entrypoint
diff --git a/docs/cli.rst b/docs/cli.rst
index 8bc4176..1d67e88 100644
--- a/docs/cli.rst
+++ b/docs/cli.rst
@@ -187,10 +187,15 @@ Available ``--fmt`` options are:
     cog.out(""\n"" + ""\n"".join('- ``{}``'.format(t) for t in tabulate.tabulate_formats) + ""\n\n"")
 .. ]]]
 
+- ``asciidoc``
+- ``double_grid``
+- ``double_outline``
 - ``fancy_grid``
 - ``fancy_outline``
 - ``github``
 - ``grid``
+- ``heavy_grid``
+- ``heavy_outline``
 - ``html``
 - ``jira``
 - ``latex``
@@ -198,15 +203,22 @@ Available ``--fmt`` options are:
 - ``latex_longtable``
 - ``latex_raw``
 - ``mediawiki``
+- ``mixed_grid``
+- ``mixed_outline``
 - ``moinmoin``
 - ``orgtbl``
+- ``outline``
 - ``pipe``
 - ``plain``
 - ``presto``
 - ``pretty``
 - ``psql``
+- ``rounded_grid``
+- ``rounded_outline``
 - ``rst``
 - ``simple``
+- ``simple_grid``
+- ``simple_outline``
 - ``textile``
 - ``tsv``
 - ``unsafehtml``
```",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/501/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
712368432,MDU6SXNzdWU3MTIzNjg0MzI=,984,Review accessibility of new column action menus,9599,simonw,open,0,,,,,1,2020-09-30T23:56:44Z,2020-10-01T00:01:36Z,,OWNER,,Feature added in #981,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/984/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
721050815,MDU6SXNzdWU3MjEwNTA4MTU=,1019,"""Edit SQL"" button on canned queries",639012,jsfenfen,closed,0,,,6026070,0.51,7,2020-10-14T00:51:39Z,2020-10-23T19:44:06Z,2020-10-14T03:44:23Z,CONTRIBUTOR,,"Feature request: Would it be possible to add an ""edit this query"" button on canned queries? Clicking it would open the canned query as an editable sql query. I think the intent is to have named parameters to allow this, but sometimes you just gotta rewrite it? ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1019/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
505928530,MDU6SXNzdWU1MDU5Mjg1MzA=,18,Command to import home-timeline,9599,simonw,closed,0,,,,,4,2019-10-11T15:47:54Z,2019-10-11T16:51:33Z,2019-10-11T16:51:12Z,MEMBER,,"Feature request: https://twitter.com/johankj/status/1182563563136868352
> Would it be possible to save all tweets in my timeline from the last X days? I would love to see how big a percentage some users are of my daily timeline as a metric on whether I should unfollow them/move them to a list.",206156866,twitter-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/18/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
435531034,MDU6SXNzdWU0MzU1MzEwMzQ=,435,Tracing support for seeing what SQL queries were executed,9599,simonw,closed,0,,,4305096,0.28,4,2019-04-21T17:37:37Z,2019-05-11T20:32:21Z,2019-05-11T19:07:42Z,OWNER,,"Features like faceting, foreign key expansions and now the inspect-less index view mean Datasette can end up executing a surprisingly large number of SQL queries to render a single page.

Past experience with projects like [tikbar](https://github.com/simonw/tikibar) have shown that being able to see what actually went into rendering a page can be critical for optimizing performance and generally understanding how everything works.

Support a tracing mode (probably via a `?_trace=1` querystring) which adds information about what is actually going on to both the HTML and the JSON.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/435/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1339444565,I_kwDOBm6k_c5P1k1V,1783,Better guidance as to what to do after you've installed Datasette,9599,simonw,open,0,,,,,2,2022-08-15T20:11:06Z,2022-08-15T20:14:01Z,,OWNER,,"Feedback [from Discord](https://discord.com/channels/823971286308356157/823971286941302908/1008822978793984060):

> hello, love the project and came for help and to point out a possible gap in the docs. starting with ""getting started"" and ""installation"" every thing looks great, but then there's a giant leap after you have it installed and running. from the user perspective of ""i have a csv of set of csvs that i want to turn into a table(s), what do i do next?"" --- so something like maybe a page for creating your first project should go after ""installation"".

- https://docs.datasette.io/en/0.62/getting_started.html
- https://docs.datasette.io/en/0.62/installation.html",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1783/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1343732788,I_kwDOBm6k_c5QF7w0,1788,Make it more obvious that Datasette publish can publish multiple databases,9599,simonw,closed,0,,,,,0,2022-08-18T22:57:51Z,2022-08-18T23:06:16Z,2022-08-18T23:06:16Z,OWNER,,Feedback initially for `datasette-publish-fly` but it applies to the others too.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1788/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
660355904,MDU6SXNzdWU2NjAzNTU5MDQ=,43,github-to-sqlite tags command for fetching tags,9599,simonw,closed,0,,,,,4,2020-07-18T20:14:12Z,2020-07-18T23:05:56Z,2020-07-18T21:52:15Z,MEMBER,,Fetches paginated data from https://api.github.com/repos/simonw/datasette/tags,207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/43/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
273569477,MDU6SXNzdWUyNzM1Njk0Nzc=,80,Deploy final versions of fivethirtyeight and parlgov datasets (with view pagination),9599,simonw,closed,0,,,2857392,Ship first public release,2,2017-11-13T20:37:46Z,2017-11-13T22:09:46Z,2017-11-13T22:09:46Z,OWNER,,Final versions should be deployed using the first released version of datasette.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/80/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1121121305,I_kwDOBm6k_c5C0vQZ,1618,"Reconsider policy on blocking queries containing the string ""pragma""",770231,strada,open,0,,,,,6,2022-02-01T19:39:46Z,2022-02-02T19:42:03Z,,NONE,,"First of all, thanks for creating this cool project, and also supporting publishing to various hosting services out of the box.

While testing out, I noticed legitimate queries such as 
```
select * from books where title like 'Pragmatic%'
```
or
```
select * from books where title = 'The Pragmatic Programmer'
```
are blocked, due to the regular expression check here:
https://github.com/simonw/datasette/blob/main/datasette/utils/__init__.py#L185

Example as seen from a Datasette instance:
https://fivethirtyeight.datasettes.com/polls?sql=select+*+from+books+where+title+like+%27Pragmatic%25%27%0D%0A

I'd propose a regular expression like 
```
re.compile(f""pragma_(?!({'|'.join(allowed_pragmas)}))""),
```
instead of
```
re.compile(f""pragma(?!_({'|'.join(allowed_pragmas)}))""),
```

I can create a pull request with this change, unless the maintainers think it would allow unwanted queries to be executed.
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1618/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1795187493,I_kwDODLZ_YM5rAGMl,12,Switch to pyproject.toml,9599,simonw,closed,0,,,,,2,2023-07-09T01:06:56Z,2023-07-09T01:19:43Z,2023-07-09T01:19:42Z,MEMBER,,First of my CLI tools to use https://til.simonwillison.net/python/pyproject,213286752,pocket-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/12/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
629473827,MDU6SXNzdWU2Mjk0NzM4Mjc=,5,Set up a demo,26745575,harryvederci,open,0,,,,,1,2020-06-02T19:56:49Z,2020-09-01T06:18:43Z,,NONE,,"First off, thanks for open sourcing this application! This is a suggestion to increase the amount of people that would make use of it: an example in the readme file would help.

Currently, users have to clone the app, install it, authorize through pocket, run a command, an then find out if this application does what they hope it does.

Another possibility is to add a file `example-output.db`, containing one (mock) Pocket article.

Keep up the good work!",213286752,pocket-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/5/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1041778507,I_kwDOCGYnMM4-GEdL,334,Filter by datetime objects using rows_where(),11642379,viseshrp,closed,0,,,,,0,2021-11-02T00:44:08Z,2021-11-13T19:23:21Z,2021-11-13T19:23:21Z,NONE,,"Firstly, thanks for this nice utility. 
It would be nice to have an example in the docs on how to filter by date range using `rows_where()`. 
This doesn't seem to work:
```
table.rows_where('datetime(created) between datetime(""2021-10-31T17:29:59.277428-04:00"") AND datetime(""2021-11-01T03:44:04.544651+00:00"")')
```


I could probably just use `db.query()`, which works for the above, but it would be nice if I could pass in `datetime` objects in `rows_where()`.
Thanks.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/334/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1378495690,I_kwDOBm6k_c5SKizK,1814,Static files not served,4068,frafra,closed,0,,,,,2,2022-09-19T20:38:17Z,2022-09-19T23:35:06Z,2022-09-19T23:34:30Z,NONE,,"Folder structure:

```
bibliography/
bibliography/static-files
bibliography/static-files/styles.css
bibliography/bibliography.db
bibliography/metadata.json
bibliography/settings.json
```

```
$ cat bibliography/settings.json 
{
  ""suggest_facets"": false,
  ""truncate_cells_html"": 1000,
  ""static"": ""assets:static-files/""
}
```

File `/assets/styles.css` is not found (HTTP 404, `Database not found: assets`).

Using datasette revision d0737e4de51ce178e556fc011ccb8cc46bbb6359.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1814/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
267732005,MDU6SXNzdWUyNjc3MzIwMDU=,17,"In development mode, should still pick up new .db files",9599,simonw,closed,0,,,2857392,Ship first public release,1,2017-10-23T16:22:40Z,2017-10-24T02:26:48Z,2017-10-24T02:26:47Z,OWNER,,Follow on from #11 ,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/17/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
605938063,MDU6SXNzdWU2MDU5MzgwNjM=,9,"upload command should be resumable, should only upload photos not already uploaded",9599,simonw,closed,0,,,,,2,2020-04-23T23:31:08Z,2020-04-23T23:39:14Z,2020-04-23T23:39:14Z,MEMBER,,Follow on from #4. ,256834907,dogsheep-photos,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-photos/issues/9/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
421985685,MDU6SXNzdWU0MjE5ODU2ODU=,421,Documentation for ?_hash=1 and Datasette's hashed URL caching,9599,simonw,closed,0,,,4305096,0.28,2,2019-03-17T23:08:36Z,2019-05-19T05:32:37Z,2019-05-19T05:31:27Z,OWNER,,Follow on from #418 - the Datasette documentation needs an entire section (probably a new page) describing exactly how the hash-in-URL caching mechanism works.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/421/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
516310670,MDU6SXNzdWU1MTYzMTA2NzA=,610,Don't suggest array facet if column is only [] empty arrays,9599,simonw,closed,0,,,,,0,2019-11-01T19:42:02Z,2019-11-01T21:46:08Z,2019-11-01T21:46:08Z,OWNER,,Follow on from #562,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/610/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
612287234,MDU6SXNzdWU2MTIyODcyMzQ=,16,"Import machine-learning detected labels (dog, llama etc) from Apple Photos",9599,simonw,open,0,,,,,13,2020-05-05T02:45:43Z,2020-05-05T05:38:16Z,,MEMBER,,"Follow-on from #1. Apple Photos runs some very sophisticated machine learning on-device to figure out if photos are of dogs, llamas and so on. I really want to extract those labels out into my own database.",256834907,dogsheep-photos,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-photos/issues/16/reactions"", ""total_count"": 2, ""+1"": 0, ""-1"": 0, ""laugh"": 1, ""hooray"": 1, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
732905360,MDU6SXNzdWU3MzI5MDUzNjA=,1067,"Table actions menu on view pages, not on query pages",9599,simonw,closed,0,,,6026070,0.51,6,2020-10-30T05:56:39Z,2020-10-31T17:51:31Z,2020-10-31T17:40:14Z,OWNER,,Follow-on from #1066.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1067/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
969840302,MDU6SXNzdWU5Njk4NDAzMDI=,1431,`--help-config` should be called `--help-settings`,9599,simonw,closed,0,,,,,1,2021-08-13T00:46:48Z,2021-08-13T01:01:58Z,2021-08-13T01:01:58Z,OWNER,,Follow-on from #1105 rebranding exercise.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1431/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
276704127,MDU6SXNzdWUyNzY3MDQxMjc=,149,Update custom SQL results to match new table view header,9599,simonw,closed,0,,,2919870,Foreign key edition,1,2017-11-24T22:03:59Z,2017-11-24T22:42:10Z,2017-11-24T22:42:09Z,OWNER,,"Follow-on from #147 - the custom SQL results page should more closely match the design of the table view, which now looks like this:

<img width=""798"" alt=""conventional_power_plants_eu__conventional_power_plants_eu__14_rows_where_company____nuon_"" src=""https://user-images.githubusercontent.com/9599/33224621-4d547f18-d120-11e7-9b33-44ee738a5ea2.png"">
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/149/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
924991194,MDU6SXNzdWU5MjQ5OTExOTQ=,280,Add --encoding option to sqlite-utils memory,9599,simonw,closed,0,,,,,0,2021-06-18T15:03:32Z,2021-06-18T15:29:46Z,2021-06-18T15:29:46Z,OWNER,,Follow-on from #272 - this will work like `--encoding` on `sqlite-utils insert` and will affect all CSV files processed by `sqlite-utils memory`.,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/280/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
377518499,MDU6SXNzdWUzNzc1MTg0OTk=,374,Get Datasette working with Zeit Now v2's 100MB image size limit,9599,simonw,closed,0,,,,,5,2018-11-05T18:08:29Z,2018-12-19T01:35:59Z,2018-12-19T01:35:59Z,OWNER,,"Follow-on from #366

Zeit Now's v2 cloud has a 100MB size limit on Docker images, in order to support much faster wake-ups of new instances.

Fitting Datasette AND the SQLite database it is hosting in here is going to be a challenge.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/374/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
464868844,MDU6SXNzdWU0NjQ4Njg4NDQ=,543,datasette publish option for setting plugin configuration secrets,9599,simonw,closed,0,,,4471010,Datasette 0.29,3,2019-07-06T16:21:23Z,2019-07-08T02:06:34Z,2019-07-08T02:06:34Z,OWNER,,Follow-on from #538 - the `datasette publish` command needs a way of passing secrets which will be made available to plugin configuration but will not be exposed in `/-/metadata.json`.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/543/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
725099777,MDU6SXNzdWU3MjUwOTk3Nzc=,1033,datasette.urls.static_plugins(...) method,9599,simonw,closed,0,,,6026070,0.51,5,2020-10-20T01:16:32Z,2020-10-24T22:58:33Z,2020-10-24T20:03:52Z,OWNER,,"Follow-on from #904. For constructing URLs like this:

    /-/static-plugins/NAME_OF_PLUGIN_PACKAGE/yourfile.js

Should be documented on https://docs.datasette.io/en/latest/writing_plugins.html#static-assets and https://docs.datasette.io/en/latest/internals.html#datasette-urls",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1033/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1175694248,I_kwDOBm6k_c5GE6uo,1677,Remove `check_permission()` from `BaseView`,9599,simonw,closed,0,,,3268330,Datasette 1.0,1,2022-03-21T17:18:18Z,2022-03-21T18:45:04Z,2022-03-21T18:45:03Z,OWNER,,"Follow-on from:
- #1675

Refs:
- #1660",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1677/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
607086780,MDU6SXNzdWU2MDcwODY3ODA=,738,Pass a request object to custom page templates,9599,simonw,closed,0,,,,,1,2020-04-26T18:57:48Z,2020-04-26T19:01:54Z,2020-04-26T19:01:54Z,OWNER,,"Follow-up to #648. I'm not passing a request object to `.render_template()` at the moment, which breaks any other custom plugins using e.g. `extra_template_vars()` that were expecting to be able to access the request.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/738/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
777145954,MDU6SXNzdWU3NzcxNDU5NTQ=,1167,Add Prettier to contributing documentation,9599,simonw,closed,0,,,6346396,Datasette 0.54,3,2020-12-31T22:00:55Z,2021-01-25T02:01:19Z,2021-01-25T01:58:28Z,OWNER,,"Following #1166 - the docs at https://docs.datasette.io/en/stable/contributing.html should include a section about JavaScript, and it should document how to run Prettier.

I run it in VS Code but it can be run on the command-line too:

    npx prettier 'datasette/static/*[!.min].js' --write
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1167/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1058803238,I_kwDOBm6k_c4_HA4m,1520,Pattern for avoiding accidental URL over-rides,9599,simonw,open,0,,,,,1,2021-11-19T18:28:05Z,2021-11-19T18:29:26Z,,OWNER,,"Following #1517 I'm experimenting with a plugin that does this:
```python
@hookimpl
def register_routes():
    return [
        (r""/(?P<db_name>[^/]+)/(?P<table_and_format>[^/]+?)$"", Table().view),
    ]
```
This is supposed to replace the default table page with new code... but there's a problem: `/-/versions` on that instance now returns 404 `Database '-' does not exist`!

Need to figure out a pattern to avoid that happening. Plugins get to add their routes before Datasette's default routes, which is why this is happening here.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1520/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
703970713,MDU6SXNzdWU3MDM5NzA3MTM=,23,Sort option should persist between multiple searches,9599,simonw,closed,0,,,,,0,2020-09-17T23:21:26Z,2020-09-18T22:39:12Z,2020-09-18T22:39:12Z,MEMBER,,Following #21 ,197431109,dogsheep-beta,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-beta/issues/23/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
777540352,MDU6SXNzdWU3Nzc1NDAzNTI=,216,database.triggers_dict introspection property,9599,simonw,closed,0,,,,,1,2021-01-02T23:13:00Z,2021-01-03T04:27:14Z,2021-01-03T04:25:36Z,OWNER,,Following #211,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/216/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
777529979,MDU6SXNzdWU3Nzc1Mjk5Nzk=,213,db.enable_counts() method,9599,simonw,closed,0,,,,,2,2021-01-02T21:44:55Z,2021-01-02T22:04:02Z,2021-01-02T22:04:02Z,OWNER,,"Following #212 it would be useful if there was a utility method for enabling counts for ALL tables in a database:

```python
db.enable_counts()
```

Open question: should this setup triggers for virtual tables such as FTS tables? Could that break things?

",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/213/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
753026388,MDU6SXNzdWU3NTMwMjYzODg=,55,github-to-sqlite workflows does not correctly replace existing records,9599,simonw,closed,0,,,,,0,2020-11-29T21:58:43Z,2020-11-29T23:48:50Z,2020-11-29T23:48:50Z,MEMBER,,Following #54 - see this TODO: https://github.com/dogsheep/github-to-sqlite/blob/1b23ce11953f9f59c0161ea1f99188b55b5ea11c/github_to_sqlite/utils.py#L700,207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/55/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
925677191,MDU6SXNzdWU5MjU2NzcxOTE=,289,Mypy fixes for rows_from_file(),857609,adamchainz,closed,0,,,,,3,2021-06-20T20:34:59Z,2021-06-22T18:44:36Z,2021-06-22T18:13:26Z,NONE,,"Following https://github.com/simonw/sqlite-utils/issues/279#issuecomment-864328927

You had two mypy errors.

The first:

> sqlite_utils/utils.py:157: error: Argument 1 to ""BufferedReader"" has incompatible type ""BinaryIO""; expected ""RawIOBase""

Looking at the `BufferedReader` docs, it seems to expect a `RawIOBase`, and this [has been copied into typeshed](https://github.com/python/typeshed/blob/9ec2f8712480c57353cea097a65d75a2c4ec1846/stdlib/io.pyi#L100). There may be scope to change how `BufferedReader` is documented and typed upstream, but for now it wouldn't be too bad to use a `typing.cast()`:

```
# Detect the format, then call this recursively
buffered = io.BufferedReader(
    cast(io.RawIOBase, fp),  # Undocumented BufferedReader support for BinaryIO
    buffer_size=4096,
)
```

The second error seems to be flagging a legitimate bug in your code:

> sqlite_utils/utils.py:163: error: Argument 1 to ""decode"" of ""bytes"" has incompatible type ""Optional[str]""; expected ""str""

From your type hints, `encoding` may be `None`. In the CSV format block, you use `encoding or ""utf-8-sig""` to set a default, maybe that's desirable in this case too?
",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/289/reactions"", ""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 1, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
516748849,MDU6SXNzdWU1MTY3NDg4NDk=,612,CSV export is broken for tables with null foreign keys,9599,simonw,closed,0,,,,,2,2019-11-02T22:52:47Z,2021-06-17T18:13:20Z,2019-11-02T23:12:53Z,OWNER,,"Following on from #406 - this CSV export appears to be broken:

https://14da705.datasette.io/fixtures/foreign_key_references.csv?_labels=on&_size=max
```csv
pk,foreign_key_with_label,foreign_key_with_label_label,foreign_key_with_no_label,foreign_key_with_no_label_label
1,1,hello,1,1
2,,
```
That second row should have 5 values, but it only has 4.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/612/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
817989436,MDU6SXNzdWU4MTc5ODk0MzY=,242,Async support,25778,eyeseast,open,0,,,,,13,2021-02-27T18:29:38Z,2021-10-28T14:37:56Z,,CONTRIBUTOR,,"Following our conversation last week, want to note this here before I forget.

I've had a couple situations where I'd like to do a bunch of updates in an async event loop, but I run into SQLite's issues with concurrent writes. This feels like something sqlite-utils could help with.

PeeWee ORM has a [SQLite write queue](http://docs.peewee-orm.com/en/latest/peewee/playhouse.html#sqliteq) that might be a good model. It's using threads or gevent, but I _think_ that approach would translate well enough to asyncio. 

Happy to help with this, too.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/242/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
803333769,MDU6SXNzdWU4MDMzMzM3Njk=,32,KeyError: 'Contents' on running upload,11855322,robmarkcole,open,0,,,,,3,2021-02-08T08:36:37Z,2021-07-22T06:40:25Z,,NONE,,"Following the readme, on big sur, and having entered my auth creds via `dogsheep-photos s3-auth`:

```
(venv) (base) Robins-MacBook:datasette robin$ dogsheep-photos upload photos.db     ~/Pictures/Photos\ /Users/robin/Pictures/Library.photoslibrary --dry-run

Fetching existing keys from S3...
Traceback (most recent call last):
  File ""/Users/robin/datasette/venv/bin/dogsheep-photos"", line 8, in <module>
    sys.exit(cli())
  File ""/Users/robin/datasette/venv/lib/python3.8/site-packages/click/core.py"", line 829, in __call__
    return self.main(*args, **kwargs)
  File ""/Users/robin/datasette/venv/lib/python3.8/site-packages/click/core.py"", line 782, in main
    rv = self.invoke(ctx)
  File ""/Users/robin/datasette/venv/lib/python3.8/site-packages/click/core.py"", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File ""/Users/robin/datasette/venv/lib/python3.8/site-packages/click/core.py"", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File ""/Users/robin/datasette/venv/lib/python3.8/site-packages/click/core.py"", line 610, in invoke
    return callback(*args, **kwargs)
  File ""/Users/robin/datasette/venv/lib/python3.8/site-packages/dogsheep_photos/cli.py"", line 96, in upload
    key.split(""."")[0] for key in get_all_keys(client, creds[""photos_s3_bucket""])
  File ""/Users/robin/datasette/venv/lib/python3.8/site-packages/dogsheep_photos/utils.py"", line 46, in get_all_keys
    for row in page[""Contents""]:
KeyError: 'Contents'
```

Possibly since the bucket is in `EU (London) eu-west-2` and this into is not requested?",256834907,dogsheep-photos,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-photos/issues/32/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1121618041,I_kwDOBm6k_c5C2oh5,1620,"Link: rel=""alternate"" to JSON for queries too",9599,simonw,closed,0,,,3268330,Datasette 1.0,3,2022-02-02T08:02:42Z,2022-02-02T21:53:02Z,2022-02-02T21:33:00Z,OWNER,,"Following:
- #1533

I implemented it for tables and rows but I should have done queries as well.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1620/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1420090659,I_kwDOBm6k_c5UpN0j,1848,Private database page should show padlock on every table,9599,simonw,closed,0,,,,,3,2022-10-24T02:28:38Z,2022-10-24T02:50:29Z,2022-10-24T02:42:34Z,OWNER,,"Following:
- #1829

https://latest.datasette.io/_internal looks like this:

<img width=""741"" alt=""image"" src=""https://user-images.githubusercontent.com/9599/197436797-815ab54f-33d9-4f49-981f-628ce4e9013c.png"">

But those queries and tables are private too, and should also show the padlock icon.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1848/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1428630253,I_kwDOBm6k_c5VJyrt,1873,Ensure insert API has good tests for rowid and compound primark key tables,9599,simonw,open,0,,,8755003,Datasette 1.0a-next,11,2022-10-30T06:22:17Z,2022-12-13T05:29:08Z,,OWNER,,"Following:
- #1866

I need to design and implement various edge-cases or primary keys:

- Table without an auto-incrementing primary key
- Table with compound primary keys
- Table with just a `rowid`",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1873/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,reopened
1700936245,I_kwDOCGYnMM5lYjo1,542,Remove `skip_false=True` and `--no-skip-false` in `sqlite-utils` 4.0,9599,simonw,open,0,,,9374594,4.0 backwards incomatible changes,1,2023-05-08T21:04:28Z,2023-05-08T21:07:41Z,,OWNER,,"Following:
- #527

The only reason I didn't remove fix this mis-feature entirely is that it represents a backwards incompatible change. I'll make that change in 4.0.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/542/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
785588942,MDU6SXNzdWU3ODU1ODg5NDI=,1187,"extra_body_script() support for script type=""module""",9599,simonw,closed,0,,,6346396,Datasette 0.54,1,2021-01-14T02:01:47Z,2021-01-24T21:21:44Z,2021-01-14T02:14:39Z,OWNER,,"Follows #1186. The `extra_body_script()` plugin hook should provide a mechanism for specifying that the script should use `<script type=""module"">`.

Relevant docs: https://docs.datasette.io/en/stable/plugin_hooks.html#extra-body-script-template-database-table-columns-view-name-request-datasette",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1187/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
647879783,MDU6SXNzdWU2NDc4Nzk3ODM=,876,Add log out link to the pattern portfolio,9599,simonw,closed,0,,,5533512,Datasette 0.45,1,2020-06-30T05:42:15Z,2020-06-30T23:50:04Z,2020-06-30T23:47:31Z,OWNER,,Follows #875,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/876/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1125576543,I_kwDOBm6k_c5DFu9f,1630,Review datasette.utils and decide which functions should be documented for 1.0,9599,simonw,open,0,,,3268330,Datasette 1.0,0,2022-02-07T06:39:52Z,2022-02-07T06:39:52Z,,OWNER,,"Follows:
- #1176",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1630/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1243715381,I_kwDOCGYnMM5KIZc1,436,"Add ""copy to clipboard"" button to code examples in documentation",9599,simonw,closed,0,,,,,0,2022-05-20T21:53:23Z,2022-05-20T21:57:53Z,2022-05-20T21:57:53Z,OWNER,,"Follows:
- #435

Imitates:
- https://github.com/simonw/datasette/issues/1748

I'll use https://github.com/executablebooks/sphinx-copybutton - here's the Datasette commit: https://github.com/simonw/datasette/commit/1465fea4798599eccfe7e8f012bd8d9adfac3039",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/436/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
756875827,MDU6SXNzdWU3NTY4NzU4Mjc=,1129,Fix footer to the bottom of the page,3243482,abdusco,open,0,,,,,0,2020-12-04T07:28:07Z,2020-12-04T16:04:29Z,,CONTRIBUTOR,,"Footer doesn't stick to the bottom if the body content isn't long enough to reach the end of viewport.

![before & after](https://user-images.githubusercontent.com/3243482/101134785-f6595a80-361b-11eb-81ce-b8b5cb9c5bc2.png)


This can be fixed using flexbox.

```css
body {
  min-height: 100vh;
  display: flex;
  flex-direction: column;
}

.content {
  flex-grow: 1;
}
```
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1129/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
737476423,MDU6SXNzdWU3Mzc0NzY0MjM=,198,Support order by relevance against FTS4,9599,simonw,closed,0,,,,,6,2020-11-06T05:36:31Z,2020-11-06T18:30:44Z,2020-11-06T18:30:44Z,OWNER,,"For #192 and #197 I've decided I want to be able to order by relevance in FTS4 as well as FTS5.

This means I need to port over my work on bm25() from https://github.com/simonw/sqlite-fts4 (since I don't want to add a full dependency).",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/198/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
525254973,MDU6SXNzdWU1MjUyNTQ5NzM=,636,rowid is not included in dropdown filter menus,9599,simonw,closed,0,,,,,3,2019-11-19T20:43:04Z,2019-11-19T23:01:17Z,2019-11-19T23:01:17Z,OWNER,,"For `rowid` tables the `rowid` column isn't shown in the list of filter options:

<img width=""652"" alt=""md__md__53_805_rows_where_where_rowid___1060124_sorted_by_rowid_descending"" src=""https://user-images.githubusercontent.com/9599/69184590-00202680-0aca-11ea-8522-3a4690924b83.png"">

This also means if you link to e.g. `?rowid__gt=1060124` the resulting filter interface will be slightly broken: clicking the ""apply"" button again will lose your filter for example.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/636/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
698791218,MDU6SXNzdWU2OTg3OTEyMTg=,50,"favorites --stop_after=N stops after min(N, 200)",370930,mikepqr,open,0,,,,,2,2020-09-11T03:38:14Z,2020-09-13T05:11:14Z,,CONTRIBUTOR,,"For any number greater than 200, `favorites --stop_after` stops after getting 200 tweets, e.g.
```
$ twitter-to-sqlite favorites tweets.db --stop_after=300
Importing favorites  [####################################]  199
$
```
I don't _think_ this is a limitation of the API (if you omit `--stop_after` you get some very large number, possibly all of them), so I _think_ this is a bug.",206156866,twitter-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/50/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
957731178,MDU6SXNzdWU5NTc3MzExNzg=,304,"`table.convert(..., where=)` and `sqlite-utils convert ... --where=`",9599,simonw,closed,0,,,,,3,2021-08-02T04:27:23Z,2021-08-02T19:00:00Z,2021-08-02T18:58:10Z,OWNER,,"For applying the conversion to a subset of rows selected using the where clause.

Should also take optional arguments, as seen in `db[""dogs""].delete_where(""age < ?"", [3])`.

Follows #302 and #251. This was originally https://github.com/simonw/sqlite-transform/issues/9",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/304/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
503085013,MDU6SXNzdWU1MDMwODUwMTM=,13,statuses-lookup command,9599,simonw,closed,0,,,,,1,2019-10-06T11:00:20Z,2019-10-07T00:33:49Z,2019-10-07T00:31:44Z,MEMBER,,"For bulk retrieving tweets by their ID.

https://developer.twitter.com/en/docs/tweets/post-and-engage/api-reference/get-statuses-lookup

Rate limit is 900/15 minutes (1 call per second) but each call can pull up to 100 IDs, so we can pull 6,000 per minute.

Should support `--SQL` and `--attach` #8 ",206156866,twitter-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/13/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
793027837,MDU6SXNzdWU3OTMwMjc4Mzc=,1205,Rename /:memory: to /_memory,9599,simonw,closed,0,,,3268330,Datasette 1.0,3,2021-01-25T05:04:56Z,2021-01-28T22:55:02Z,2021-01-28T22:51:42Z,OWNER,,"For consistency with `/_internal` - and because then we don't need to escape the `:` characters.

This change would need to be in before Datasette 1.0. I could land it earlier and set up redirects from the old URLs though.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1205/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1352931464,I_kwDOCGYnMM5QpBiI,469,sqlite-utils rows --order option,9599,simonw,closed,0,,,8355157,3.29,1,2022-08-27T03:49:51Z,2022-08-27T04:30:49Z,2022-08-27T04:10:32Z,OWNER,,"For consistency with `search`: https://sqlite-utils.datasette.io/en/stable/cli-reference.html#search

```
  -o, --order TEXT       Order by ('column' or 'column desc')
```

I wanted to run `sqlite-utils rows db.db mytable --order 'rowid desc'` to see the most recently imported rows.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/469/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
327365110,MDU6SXNzdWUzMjczNjUxMTA=,294,inspect should record column types,9599,simonw,open,0,,,,,7,2018-05-29T15:10:41Z,2019-06-28T16:45:28Z,,OWNER,,"For each table we want to know the columns, their order and what type they are.

I'm going to break with SQLite defaults a little on this one and allow datasette to define additional types - to start with just a `geometry` type for columns that are detected as SpatiaLite geometries.

Possible JSON design:

    ""columns"": [{
        ""name"": ""title"",
        ""type"": ""text""
    }, ...]

Refs #276",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/294/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1205687423,I_kwDOCGYnMM5H3VR_,426,CLI docs should link to Python docs and vice versa,9599,simonw,closed,0,9599,simonw,,,1,2022-04-15T16:05:15Z,2023-07-22T22:13:22Z,2023-07-22T22:13:22Z,OWNER,,"For every command/API method there should be a link to the equivalent in the other form factor.

Maybe also link to the API and CLI reference pages too.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/426/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1186696202,I_kwDOBm6k_c5Gu4wK,1696,Show foreign key label when filtering,9599,simonw,open,0,,,,,2,2022-03-30T16:18:54Z,2023-01-29T20:56:20Z,,OWNER,,"For example here:

<img width=""624"" alt=""image"" src=""https://user-images.githubusercontent.com/9599/160882762-a5000ad7-7646-4f60-8d58-23ad9e79b184.png"">

3 corresponds to ""Human Related: Other"" - it would be neat to display this in this area of the page somehow.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1696/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1213683988,I_kwDOBm6k_c5IV1kU,1718,Code examples in the documentation should be formatted with Black,9599,simonw,closed,0,,,,,12,2022-04-24T15:22:50Z,2022-04-24T16:24:14Z,2022-04-24T16:18:03Z,OWNER,,"For example on this page: https://docs.datasette.io/en/stable/writing_plugins.html#packaging-a-plugin

I wonder if there's an easy way for me to enforce this for Sphinx documentation?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1718/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1847201263,I_kwDOBm6k_c5uGg3v,2140,Remove all remaining documentation instances of '$ ',9599,simonw,closed,0,,,,,1,2023-08-11T17:42:13Z,2023-08-11T17:52:25Z,2023-08-11T17:45:00Z,OWNER,,"For example this: https://github.com/simonw/datasette/blob/4535568f2ce907af646304d0ebce2500ebd55677/docs/authentication.rst?plain=1#L33-L35

The problem with that `$ ` prefix is that it prevents users from copying and pasting the raw command.

<img width=""773"" alt=""image"" src=""https://github.com/simonw/datasette/assets/9599/9414437d-1eb9-416b-ae51-0f4f5d665b80"">

https://docs.datasette.io/en/stable/authentication.html#using-the-root-actor",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2140/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
995098231,MDU6SXNzdWU5OTUwOTgyMzE=,1470,?_sort=rowid with _next= returns error,19851673,eigenfoo,closed,0,,,,,4,2021-09-13T16:36:15Z,2021-10-18T19:30:15Z,2021-10-10T01:15:03Z,NONE,,"For example:

- Go to https://cryptics.eigenfoo.xyz/clues/clues?_next=100 (this is the second page of results in a Datasette site)
- Search anything using the FTS search bar. For example, searching for `hello` will take you to https://cryptics.eigenfoo.xyz/clues/clues?_search=hello&_sort=rowid&_next=100
- A `500 Error: list index out of range` is raised.

This is because the search URL includes the `&_next=100` UTM parameter, carried over from where the FTS search was run. However, there isn't a second page in the search results, so a `list index out of range` error is raised. You can confirm that removing this UTM parameter from the URL returns the appropriate search results.

The FTS search request should strip any `_next` UTM parameter.

---

```bash
datasette, version 0.58.1
sqlite-utils, version 3.17
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1470/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1318907685,I_kwDOBm6k_c5OnO8l,1773,500 error if sorted by a column not in the ?_col= list,9599,simonw,closed,0,,,8303187,Datasette 0.62,4,2022-07-27T01:20:27Z,2022-08-14T16:06:25Z,2022-08-14T15:44:05Z,OWNER,,"For example: https://latest.datasette.io/fixtures/sortable?_sort_desc=sortable&_col=sortable_with_nulls

That's `?_sort_desc=sortable&_col=sortable_with_nulls`

<img width=""494"" alt=""image"" src=""https://user-images.githubusercontent.com/9599/181139719-06e3bf66-3fde-4f9a-bfd1-62b326121601.png"">
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1773/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
681086659,MDU6SXNzdWU2ODEwODY2NTk=,47,emojis command,9599,simonw,closed,0,,,,,1,2020-08-18T14:26:26Z,2020-08-18T14:52:13Z,2020-08-18T14:52:13Z,MEMBER,,For fun - it can import https://api.github.com/emojis - maybe with an option to fetch the binary representations in addition to the URLs.,207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/47/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1058196641,I_kwDOCGYnMM4_Esyh,342,Extra options to `lookup()` which get passed to `insert()`,9599,simonw,closed,0,,,,,7,2021-11-19T06:53:03Z,2021-11-19T07:26:54Z,2021-11-19T07:26:54Z,OWNER,,"For https://github.com/simonw/git-history/issues/12 I found myself wanting to pass extra options to `lookup()` to set the column order, primary key etc.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/342/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
348043884,MDU6SXNzdWUzNDgwNDM4ODQ=,357,Plugin hook for loading metadata.json,9599,simonw,open,0,,,,,6,2018-08-06T19:00:01Z,2020-06-21T22:19:58Z,,OWNER,,"For https://github.com/simonw/russian-ira-facebook-ads-datasette/tree/af6d956995e14afd585c35a6a06bb01da32043ba I wrote a script to convert YAML to JSON because YAML is a better format for embedding multi-line HTML descriptions and canned SQL statements.

Example yaml metadata file: https://github.com/simonw/russian-ira-facebook-ads-datasette/blob/af6d956995e14afd585c35a6a06bb01da32043ba/russian-ads-metadata.yaml

It would be useful if Datasette could be fed a YAML file directly:

    datasette -m metadata.yaml

Question is... should this be a native feature (hence adding a YAML dependency) or should it be handled by a `datasette-metadata-yaml` plugin, using a new plugin hook for loading metadata? If so, what would other use-cases for that plugin hook be?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/357/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1097129710,I_kwDOCGYnMM5BZN7u,372,Idea: `suffix` and `stem` file columns,9599,simonw,closed,0,,,7558727,3.21,1,2022-01-09T07:48:53Z,2022-01-10T19:27:34Z,2022-01-09T20:17:00Z,OWNER,,"For https://sqlite-utils.datasette.io/en/stable/cli.html#inserting-data-from-files

Given a file called `dogs.jpg` stem would be `dogs` and ext would be `jpg`.

Need to decide what happens for `dogs.and.cats.jpg.gz`.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/372/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
654405302,MDU6SXNzdWU2NTQ0MDUzMDI=,42,Option for importing just specific repos,9599,simonw,closed,0,,,,,0,2020-07-09T23:20:15Z,2020-07-09T23:25:35Z,2020-07-09T23:25:35Z,MEMBER,,"For if you know which specific repos you care about, as opposed to loading everything owned by the authenticated user.

    github-to-sqlite repos specific.db -r simonw/datasette -r simonw/github-contents
",207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/42/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
660413281,MDU6SXNzdWU2NjA0MTMyODE=,44,Rename tags.repo_id column to tags.repo,9599,simonw,closed,0,,,,,0,2020-07-18T22:13:46Z,2020-07-18T22:15:12Z,2020-07-18T22:15:12Z,MEMBER,,"For improved consistency with other tables. https://observablehq.com/@simonw/datasette-table-diagram

![datasette-table-diagram(1)](https://user-images.githubusercontent.com/9599/87862843-3cca4900-c909-11ea-9c76-58b3f4aca43f.png)
",207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/44/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1078702875,I_kwDOBm6k_c5AS7Mb,1552,Allow to set `facets_array` in metadata (like current `facets`),3556,davidbgk,closed,0,,,7571612,Datasette 0.60,9,2021-12-13T16:00:44Z,2022-01-13T22:26:15Z,2021-12-16T18:47:48Z,CONTRIBUTOR,,"For now, you can set a `facets` value (array) in your metadata file but I couldn't find a way to set a `facets_array` in order to provide default facets for arrays (like tags). My use-case is to access to [that kind of view](https://latest.datasette.io/fixtures/facetable?_facet_array=tags) by default without URL's parameters as with other default facets.

_I'm new to datasette, and I'm willing to help with a PR if that is not already implemented and I missed it!_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1552/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
273127443,MDU6SXNzdWUyNzMxMjc0NDM=,56,Easy way to block search engine crawling in robots.txt,9599,simonw,closed,0,,,,,1,2017-11-11T07:46:07Z,2018-05-28T20:50:25Z,2018-05-28T20:50:24Z,OWNER,,For people who don't want their datasets to be crawled by search engines.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/56/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
320090329,MDU6SXNzdWUzMjAwOTAzMjk=,249,?_size=max argument ,9599,simonw,closed,0,,,,,1,2018-05-03T21:42:04Z,2018-05-04T18:26:30Z,2018-05-04T18:05:04Z,OWNER,,"For plugins that want to load the most data allowable, having `?_size=max` would be useful.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/249/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
959898166,MDU6SXNzdWU5NTk4OTgxNjY=,1420,`datasette publish cloudrun --cpu X` option,9599,simonw,closed,0,,,,,5,2021-08-04T05:04:31Z,2021-08-05T00:54:59Z,2021-08-04T05:33:48Z,OWNER,,"For setting the number of vCPUs - current valid values are 1, 2 or 4: https://cloud.google.com/run/docs/configuring/cpu

Pass that through to `gcloud run deploy --image IMAGE_URL --cpu CPU`",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1420/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
273026602,MDU6SXNzdWUyNzMwMjY2MDI=,52,Solution for temporarily uploading DB so it can be built by docker,9599,simonw,closed,0,,,,,2,2017-11-10T18:55:25Z,2017-12-10T03:02:57Z,2017-12-10T03:02:57Z,OWNER,,For the `datasette publish` command I ideally need a way of uploading the specified DB to somewhere temporary on the internet so that when the Dockerfile is built by the final hosting location it can download that database as part of the build process.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/52/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1296222572,I_kwDOBm6k_c5NQsls,1768,Upgrade to 3.10.6-slim-bullseye Docker base image,9599,simonw,closed,0,,,8303187,Datasette 0.62,5,2022-07-06T18:37:49Z,2022-08-14T15:54:36Z,2022-08-14T15:54:11Z,OWNER,,For the package published to Docker Hub and also the containers used by `datasette package` and `datasette publish cloudrun`.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1768/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1098574572,I_kwDOCGYnMM5Beurs,380,Release notes for 3.21,9599,simonw,closed,0,,,7558727,3.21,1,2022-01-11T02:12:30Z,2022-01-11T02:34:26Z,2022-01-11T02:34:26Z,OWNER,,For these commits: https://github.com/simonw/sqlite-utils/compare/3.20...129141572f249ea290e2a075437e2ebaad215859,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/380/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
534492501,MDU6SXNzdWU1MzQ0OTI1MDE=,648,Mechanism for adding arbitrary pages like /about,9599,simonw,closed,0,,,,,13,2019-12-08T04:55:19Z,2020-05-07T15:21:19Z,2020-04-26T18:46:45Z,OWNER,,"For www.niche-museums.com I solved this by creating an empty `about.db` database file - see https://simonwillison.net/2019/Nov/25/niche-museums/

I want a neater mechanism for this.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/648/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1179998071,I_kwDOBm6k_c5GVVd3,1684,Mechanism for disabling faceting on large tables only,9599,simonw,open,0,,,,,1,2022-03-24T20:06:11Z,2022-03-24T20:13:19Z,,OWNER,,"Forest turned off faceting on https://labordata.bunkum.us/ because it was causing performance problems on some of the huge tables - but it would be nice if it could still be an option on smaller tables such as https://labordata.bunkum.us/voluntary_recognitions-4421085/voluntary_recognitions

One option: a new setting that automatically disables faceting (and facet suggestion) for tables that have either more than X rows or that are so big that the count could not be completed within the time limit.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1684/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1082564912,I_kwDOBm6k_c5AhqEw,1557,`?_nosuggest=1` parameter for disabling facet suggestions on table view,9599,simonw,closed,0,,,7571612,Datasette 0.60,1,2021-12-16T19:21:42Z,2022-01-13T22:26:48Z,2021-12-16T19:24:59Z,OWNER,,"Found I wanted this while I was debugging #625 just to clean up the debug traces, but it makes sense as a partner to `?_nofacet=1` and `?_nocount=1` from #1350 and #1353.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1557/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1468519699,I_kwDOBm6k_c5Xh9UT,1911,`/db/-/create` should support creating tables with compound primary keys,9599,simonw,closed,0,,,8658075,Datasette 1.0a0,2,2022-11-29T18:30:47Z,2022-11-29T18:50:58Z,2022-11-29T18:48:05Z,OWNER,,"Found myself needing this to write the tests for:
- #1864 ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1911/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1816851056,I_kwDOCGYnMM5sSvJw,568,"table.create(..., replace=True)",9599,simonw,closed,0,,,,,7,2023-07-22T18:12:22Z,2023-07-22T19:25:35Z,2023-07-22T19:15:44Z,OWNER,,"Found myself using this pattern to quickly prototype a schema:

```python
import sqlite_utils
db = sqlite_utils.Database(memory=True)

print(db[""answers_chunks""].create({
    ""id"": int,
    ""content"": str,
    ""embedding_type_id"": int,
    ""embedding"": bytes,
    ""embedding_content_md5"": str,
    ""source"": str,
}, pk=""id"", transform=True).schema)
```

<img width=""472"" alt=""image"" src=""https://github.com/simonw/datasette-answers/assets/9599/30237f7c-4964-4190-984c-d824c57939a9"">

Using `replace=True` to drop and then recreate the table would be neat here, and would be consistent with other places that use `replace=True`.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/568/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1087919372,I_kwDOBm6k_c5A2FUM,1578,Confirm if documented nginx proxy config works for row pages with escaped characters in their primary key,9599,simonw,open,0,,,,,4,2021-12-23T18:27:59Z,2021-12-24T21:33:19Z,,OWNER,,"Found this while working on https://github.com/simonw/datasette-tiddlywiki

<img width=""1254"" alt=""image"" src=""https://user-images.githubusercontent.com/9599/147279097-e02f80f3-cc88-4bdd-a26f-03f924c13b5e.png"">

Then clicking on `/tiddlywiki/tiddlers/%24%3A%2FDefaultTiddlers` returns a 404.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1578/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
626001501,MDU6SXNzdWU2MjYwMDE1MDE=,773,All plugin hooks should have unit tests,9599,simonw,closed,0,,,5471110,Datasette 0.43,2,2020-05-27T20:17:41Z,2020-05-28T04:12:11Z,2020-05-28T04:09:25Z,OWNER,,"Four hooks currently missing tests:

- [x] prepare_jinja2_environment
- [x] publish_subcommand
- [x] register_facet_classes
- [x] register_output_renderer

```
$ pytest -k test_plugin_hooks_have_tests -vv
====================================== test session starts ======================================
platform darwin -- Python 3.7.7, pytest-5.2.4, py-1.8.1, pluggy-0.13.1 -- /Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/bin/python
cachedir: .pytest_cache
rootdir: /Users/simon/Dropbox/Development/datasette, inifile: pytest.ini
plugins: asyncio-0.10.0
collected 486 items / 475 deselected / 11 selected                                              

tests/test_plugins.py::test_plugin_hooks_have_tests[asgi_wrapper] XPASS                   [  9%]
tests/test_plugins.py::test_plugin_hooks_have_tests[extra_body_script] XPASS              [ 18%]
tests/test_plugins.py::test_plugin_hooks_have_tests[extra_css_urls] XPASS                 [ 27%]
tests/test_plugins.py::test_plugin_hooks_have_tests[extra_js_urls] XPASS                  [ 36%]
tests/test_plugins.py::test_plugin_hooks_have_tests[extra_template_vars] XPASS            [ 45%]
tests/test_plugins.py::test_plugin_hooks_have_tests[prepare_connection] XPASS             [ 54%]
tests/test_plugins.py::test_plugin_hooks_have_tests[prepare_jinja2_environment] XFAIL     [ 63%]
tests/test_plugins.py::test_plugin_hooks_have_tests[publish_subcommand] XFAIL             [ 72%]
tests/test_plugins.py::test_plugin_hooks_have_tests[register_facet_classes] XFAIL         [ 81%]
tests/test_plugins.py::test_plugin_hooks_have_tests[register_output_renderer] XFAIL       [ 90%]
tests/test_plugins.py::test_plugin_hooks_have_tests[render_cell] XPASS                    [100%]

========================= 475 deselected, 4 xfailed, 7 xpassed in 1.70s =========================

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/771#issuecomment-634915104_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/773/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
470637152,MDU6SXNzdWU0NzA2MzcxNTI=,2,Import workouts,9599,simonw,closed,0,,,,,1,2019-07-20T05:20:21Z,2019-07-20T06:21:41Z,2019-07-20T06:21:41Z,MEMBER,,From #1,197882382,healthkit-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/2/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
470637206,MDU6SXNzdWU0NzA2MzcyMDY=,3,Import ActivitySummary,9599,simonw,closed,0,,,,,0,2019-07-20T05:21:00Z,2019-07-20T05:58:07Z,2019-07-20T05:58:07Z,MEMBER,,"From #1

```python
  'ActivitySummary': {'attr_counts': {'activeEnergyBurned': 980,
                                     'activeEnergyBurnedGoal': 980,
                                     'activeEnergyBurnedUnit': 980,
                                     'appleExerciseTime': 980,
                                     'appleExerciseTimeGoal': 980,
                                     'appleStandHours': 980,
                                     'appleStandHoursGoal': 980,
                                     'dateComponents': 980},
                     'child_counts': {},
                     'count': 980,
                     'parent_counts': {'HealthData': 980}},
```",197882382,healthkit-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/3/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
470640505,MDU6SXNzdWU0NzA2NDA1MDU=,4,Import Records,9599,simonw,closed,0,,,,,1,2019-07-20T06:11:20Z,2019-07-20T06:21:41Z,2019-07-20T06:21:41Z,MEMBER,,"From #1:
```python
 'Record': {'attr_counts': {'creationDate': 2672233,
                            'device': 2665111,
                            'endDate': 2672233,
                            'sourceName': 2672233,
                            'sourceVersion': 2671779,
                            'startDate': 2672233,
                            'type': 2672233,
                            'unit': 2650012,
                            'value': 2672232},
            'child_counts': {'HeartRateVariabilityMetadataList': 2318,
                             'MetadataEntry': 287974},
            'count': 2672233,
            'parent_counts': {'Correlation': 2, 'HealthData': 2672231}},
```",197882382,healthkit-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/4/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
314834783,MDU6SXNzdWUzMTQ4MzQ3ODM=,219,Expose units in the JSON API?,45057,russss,open,0,,,,,0,2018-04-16T22:04:25Z,2018-04-16T22:04:25Z,,CONTRIBUTOR,,"From #203: it would be nice for the JSON API to (optionally) return columns rendered with units in them - if, for example, you're consuming the JSON to render the rows on a map.

I'm not entirely sure how useful this will be though - at the moment my map queries are custom SQL queries (a few have joins in, the rest might be fetching large amounts of data so it makes sense to limit columns fetched). Perhaps the SQL function is a better approach in general.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/219/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
703246031,MDU6SXNzdWU3MDMyNDYwMzE=,51,github-to-sqlite should handle rate limits better,9599,simonw,open,0,,,,,4,2020-09-17T04:01:50Z,2022-10-14T16:34:07Z,,MEMBER,,From #50 - right now it will crash with an error of it hits the rate limit. Since the rate limit information (including reset time) is available in the headers it could automatically sleep and try again instead.,207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/51/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1560662739,I_kwDOBm6k_c5dBdLT,2007,`render_cell()` hook should take an optional `request` argument,9599,simonw,closed,0,,,,,1,2023-01-28T03:13:00Z,2023-08-09T17:15:03Z,2023-01-28T03:34:26Z,OWNER,,From Discord: https://discordapp.com/channels/823971286308356157/996877076982415491/1068227071156965486,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2007/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1615692818,I_kwDOBm6k_c5gTYQS,2035,Potential feature: special support for `?a=1&a=2` on the query page,9599,simonw,open,0,,,3268330,Datasette 1.0,14,2023-03-08T18:05:03Z,2023-03-31T16:09:08Z,,OWNER,,"From a discussion on Discord: https://discord.com/channels/823971286308356157/996877076982415491/1082789517062320138

The key idea is to make it easier for people to implement `where id in (...)` that's populated from query string arguments.

What if you could add `?id=11&id=32&id=62` to the URL and have that made available as a list that can be used in the query?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2035/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1515815014,I_kwDOBm6k_c5aWYBm,1973,render_cell plugin hook's row object is not a sqlite.Row,193185,cldellow,open,0,,,,,4,2023-01-01T20:27:46Z,2023-01-29T00:40:31Z,,CONTRIBUTOR,,"From https://docs.datasette.io/en/stable/plugin_hooks.html#render-cell-row-value-column-table-database-datasette:

> row - sqlite.Row
> The SQLite row object that the value being rendered is part of

This appears to actually be a [CustomRow](https://github.com/simonw/datasette/blob/f0fadc28ddb9f82e5cc1ecaa51e8a342eb6dc528/datasette/utils/__init__.py#L773-L789), but I think that's unrelated to my issue.

I have a table:

```sql
CREATE TABLE IF NOT EXISTS ""dss_job_stats""(
  job_id integer not null references dss_job(id) on delete cascade,
  host text not null,
  // other columns elided as irrelevant
  primary key (job_id, host)
);
```

On datasette 0.63.2, the `render_cell` hook receives a `row` value that looks like:

```
CustomRow([('job_id', {'value': 2, 'label': '2'}), ('host', 'cldellow.com')])
```

I expected the `job_id` value to be `2`, but it's actually `{'value': 2, 'label': '2'}`.

I can work around this, but was wondering if this was intended behaviour?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1973/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
653529088,MDU6SXNzdWU2NTM1MjkwODg=,891,Consider using enable_callback_tracebacks(True),9599,simonw,closed,0,,,5971510,Datasette 0.50,5,2020-07-08T19:07:16Z,2020-10-08T23:54:23Z,2020-09-15T21:59:27Z,OWNER,,"From https://docs.python.org/3/library/sqlite3.html#sqlite3.enable_callback_tracebacks

> `sqlite3.``enable_callback_tracebacks`(*flag*)[¶](https://docs.python.org/3/library/sqlite3.html#sqlite3.enable_callback_tracebacks ""Permalink to this definition"")
> 
> By default you will not get any tracebacks in user-defined functions, aggregates, converters, authorizer callbacks etc. If you want to debug them, you can call this function with *flag* set to `True`. Afterwards, you will get tracebacks from callbacks on `sys.stderr`. Use [`False`](https://docs.python.org/3/library/constants.html#False ""False"") to disable the feature again.

Maybe turn this on for all of Datasette? Are there any disadvantages to doing that?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/891/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
769150394,MDU6SXNzdWU3NjkxNTAzOTQ=,58,Readme HTML has broken internal links,9599,simonw,closed,0,,,,,2,2020-12-16T17:58:11Z,2020-12-16T19:20:14Z,2020-12-16T19:20:14Z,MEMBER,,"From https://github.com/simonw/datasette.io/issues/46
```html
<li><a href=""#filtering-tables"">Filtering tables</a></li>
...
<h3><a id=""user-content-filtering-tables"" class=""anchor"" aria-hidden=""true"" href=""#filtering-tables""><svg class=""octicon octicon-link"" viewBox=""0 0 16 16"" version=""1.1"" width=""16"" height=""16"" aria-hidden=""true""><path fill-rule=""evenodd"" d=""M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z""></path></svg></a>Filtering tables</h3>
```
So this is a bug in GitHub's API, but we need to work around it.",207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/58/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
778380836,MDU6SXNzdWU3NzgzODA4MzY=,4,Feature Request: Gmail,203343,Btibert3,open,0,,,,,5,2021-01-04T21:31:09Z,2021-03-04T20:54:44Z,,NONE,,"From takeout, I only exported my Gmail account.  Ideally I could parse this into sqlite via this tool.",206649770,google-takeout-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/4/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1620515757,I_kwDOBm6k_c5glxut,2039,Subtle bug with `--load-extension` and `--static` flags with absolute Windows paths with`C:\`,15178711,asg017,open,0,,,,,0,2023-03-12T21:18:52Z,2023-03-12T21:18:52Z,,CONTRIBUTOR,,"From the Datasette discord: A user tried running the following command on windows:

```
datasette  --load-extension=""C:\spatialite\mod_spatialite-5.0.1-win-x86\mod_spatialite.dll""
```
This failed with `""The specified module could not be found""`, because the entrypoint option introduced in #1789 splits the input differently. Instead of loading the extension found at `""C:\spatialite\mod_spatialite-5.0.1-win-x86\mod_spatialite.dll""`, it instead tried to load the extension at `""C""` with entrypoint `""\spatialite\mod_spatialite-5.0.1-win-x86\mod_spatialite.dll"". 

This is hard because most absolute windows paths have a colon in them, like `C:\foo.txt` or `D:\bar.txt`.  I'd image the `--static` flag is also vulnerable to this type of bug.


The ""solution"" is to use a relative path instead, but that doesn't feel that great. ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2039/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
267828746,MDU6SXNzdWUyNjc4Mjg3NDY=,24,Implement full URL design,9599,simonw,closed,0,,,2857392,Ship first public release,2,2017-10-23T21:49:05Z,2017-10-24T14:12:00Z,2017-10-24T14:12:00Z,OWNER,,"Full URL design:

    /database-name
    /database-name.json
    /database-name-7sha256
    /database-name-7sha256.json
    /database-name/table-name
    /database-name/table-name.json
    /database-name-7sha256/table-name
    /database-name-7sha256/table-name.json
    /database-name-7sha256/table-name/compound-pk
    /database-name-7sha256/table-name/compound-pk.json
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/24/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
280013907,MDU6SXNzdWUyODAwMTM5MDc=,164,datasette skeleton command for kick-starting database and table metadata,9599,simonw,closed,0,,,2949431,Custom templates edition,3,2017-12-07T06:13:28Z,2021-03-23T02:45:12Z,2017-12-07T06:20:45Z,OWNER,,Generates an example `metadata.json` file populated with all of the databases and tables inspected from the specified databases.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/164/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1212838949,I_kwDOBm6k_c5ISnQl,1716,Configure git blame to ignore Black commit,9599,simonw,closed,0,,,,,1,2022-04-22T21:56:37Z,2022-04-22T22:02:19Z,2022-04-22T22:02:19Z,OWNER,,"GitHub can support this in blame views now too:

https://docs.github.com/en/repositories/working-with-files/using-files/viewing-a-file#ignore-commits-in-the-blame-view",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1716/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1891614971,I_kwDOCGYnMM5wv8D7,594,Represent compound foreign keys in table.foreign_keys output,9599,simonw,open,0,,,,,2,2023-09-12T03:48:24Z,2023-09-12T03:51:13Z,,OWNER,,"Given this schema:
```sql
CREATE TABLE departments (
    campus_name TEXT NOT NULL,
    dept_code TEXT NOT NULL,
    dept_name TEXT,
    PRIMARY KEY (campus_name, dept_code)
);
CREATE TABLE courses (
    course_code TEXT PRIMARY KEY,
    course_name TEXT,
    campus_name TEXT NOT NULL,
    dept_code TEXT NOT NULL,
    FOREIGN KEY (campus_name, dept_code) REFERENCES departments(campus_name, dept_code)
);
```
The output of `db[""courses""].foreign_keys` right now is:
```
[ForeignKey(table='courses', column='campus_name', other_table='departments', other_column='campus_name'),
 ForeignKey(table='courses', column='dept_code', other_table='departments', other_column='dept_code')]
```
Which suggests two normal foreign keys, not one compound foreign key.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/594/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
443034218,MDU6SXNzdWU0NDMwMzQyMTg=,464,Add Glitch to Getting Started docs section,9599,simonw,closed,0,,,4305096,0.28,1,2019-05-11T20:39:39Z,2019-05-16T05:04:35Z,2019-05-16T05:03:46Z,OWNER,,Glitch is by far the easiest way to start trying out Datasette. Add a section to https://datasette.readthedocs.io/en/latest/getting_started.html,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/464/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
398559195,MDU6SXNzdWUzOTg1NTkxOTU=,400,datasette publish cloudrun plugin,10352819,rprimet,closed,0,,,,,1,2019-01-12T14:35:11Z,2019-05-03T16:57:35Z,2019-05-03T16:57:35Z,CONTRIBUTOR,,"Google announced that they may launch a simple service for running Docker containers (previously serverless containers, now called ""cloud run"" -- link to alpha [here](https://services.google.com/fb/forms/serverlesscontainers/)). If/when this happens, it might be a good fit for publishing datasettes? (at least using the current version, manually publishing a datasette seems relatively painless).",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/400/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
763361458,MDU6SXNzdWU3NjMzNjE0NTg=,1142,"""Stream all rows"" is not at all obvious",9599,simonw,open,0,,,,,9,2020-12-12T06:24:57Z,2021-06-17T18:12:31Z,,OWNER,,"Got a question about how to download all rows - the current option isn't at all clear.

<img width=""668"" alt=""loans__ppp_loans__9_511_rows_where_where_search_matches__tech__sorted_by_rowid"" src=""https://user-images.githubusercontent.com/9599/101977057-ac660b00-3bff-11eb-88f4-c93ffd03d3e0.png"">
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1142/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
576582604,MDU6SXNzdWU1NzY1ODI2MDQ=,694,datasette publish cloudrun --memory option,9599,simonw,closed,0,,,,,8,2020-03-05T22:59:57Z,2020-06-23T17:10:51Z,2020-03-05T23:49:41Z,OWNER,,"Got this error deploying large (603MB) database with Cloud Run
```
X Deploying... Cloud Run error: Container failed to start. Failed to start and then listen on the port defined by the PORT environment variable. Logs for this revi
sion might contain more information.                                                                                                                               
  X Creating Revision... Cloud Run error: Container failed to start. Failed to start and then listen on the port defined by the PORT environment variable. Logs for
   this revision might contain more information.                                                                                                                   
  . Routing traffic...                                                                                                                                             
  ✓ Setting IAM Policy...                                                                                                                                          
Deployment failed                                                                                                                                                  
ERROR: (gcloud.run.deploy) Cloud Run error: Container failed to start. Failed to start and then listen on the port defined by the PORT environment variable. Logs for this revision might contain more information.
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/694/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
975158266,MDU6SXNzdWU5NzUxNTgyNjY=,19,table activity_summary has no column named appleMoveTime,9599,simonw,closed,0,,,,,0,2021-08-20T00:46:44Z,2021-08-20T00:54:34Z,2021-08-20T00:54:34Z,MEMBER,,"Got this error today against a fresh export:

    table activity_summary has no column named appleMoveTime
",197882382,healthkit-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/19/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
978743426,MDU6SXNzdWU5Nzg3NDM0MjY=,13,xml.etree.ElementTree.ParseError: not well-formed (invalid token),9599,simonw,closed,0,,,,,4,2021-08-25T05:48:21Z,2021-08-26T18:45:13Z,2021-08-26T18:45:13Z,MEMBER,,"Got this error today:
```
(evernote-to-sqlite) /tmp % evernote-to-sqlite enex evernote.db simonwillison\'s\ notebook.enex 
Importing from ENEX  [######------------------------------]   17%
Traceback (most recent call last):
  File ""/Users/simon/.local/bin/evernote-to-sqlite"", line 8, in <module>
    sys.exit(cli())
  File ""/Users/simon/.local/pipx/venvs/evernote-to-sqlite/lib/python3.9/site-packages/click/core.py"", line 1137, in __call__
    return self.main(*args, **kwargs)
  File ""/Users/simon/.local/pipx/venvs/evernote-to-sqlite/lib/python3.9/site-packages/click/core.py"", line 1062, in main
    rv = self.invoke(ctx)
  File ""/Users/simon/.local/pipx/venvs/evernote-to-sqlite/lib/python3.9/site-packages/click/core.py"", line 1668, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File ""/Users/simon/.local/pipx/venvs/evernote-to-sqlite/lib/python3.9/site-packages/click/core.py"", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File ""/Users/simon/.local/pipx/venvs/evernote-to-sqlite/lib/python3.9/site-packages/click/core.py"", line 763, in invoke
    return __callback(*args, **kwargs)
  File ""/Users/simon/.local/pipx/venvs/evernote-to-sqlite/lib/python3.9/site-packages/evernote_to_sqlite/cli.py"", line 31, in enex
    save_note(db, note)
  File ""/Users/simon/.local/pipx/venvs/evernote-to-sqlite/lib/python3.9/site-packages/evernote_to_sqlite/utils.py"", line 36, in save_note
    content = ET.tostring(ET.fromstring(content_xml)).decode(""utf-8"")
  File ""/usr/local/Cellar/python@3.9/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/xml/etree/ElementTree.py"", line 1347, in XML
    parser.feed(text)
xml.etree.ElementTree.ParseError: not well-formed (invalid token): line 2, column 132
```",303218369,evernote-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/13/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1203842656,I_kwDOCGYnMM5HwS5g,425,`sqlite3.NotSupportedError`: deterministic=True requires SQLite 3.8.3 or higher,9599,simonw,closed,0,,,,,5,2022-04-13T22:16:53Z,2023-04-15T20:14:58Z,2022-04-13T22:48:57Z,OWNER,,"Got this error while investigating:
- #421

Even though I was using the `LD_PRELOAD` trick from https://til.simonwillison.net/sqlite/ld-preload to use a newer version of SQLite.

_Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/421#issuecomment-1098531354_",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/425/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
581339961,MDU6SXNzdWU1ODEzMzk5NjE=,92,.columns_dict doesn't work for all possible column types,9599,simonw,closed,0,,,,,7,2020-03-14T19:30:35Z,2020-03-15T18:37:43Z,2020-03-14T20:04:14Z,OWNER,,"Got this error:
```
  File "".../python3.7/site-packages/sqlite_utils/db.py"", line 462, in <dictcomp>
    for column in self.columns
KeyError: 'REAL'
```
`.columns_dict` uses `REVERSE_COLUMN_TYPE_MAPPING`:
https://github.com/simonw/sqlite-utils/blob/43f1c6ab4e3a6b76531fb6f5447adb83d26f3971/sqlite_utils/db.py#L457-L463
`REVERSE_COLUMN_TYPE_MAPPING` defines `FLOAT` not `REAL`A
https://github.com/simonw/sqlite-utils/blob/43f1c6ab4e3a6b76531fb6f5447adb83d26f3971/sqlite_utils/db.py#L68-L74",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/92/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
811458446,MDU6SXNzdWU4MTE0NTg0NDY=,1233,"""datasette publish cloudrun"" cannot publish files with spaces in their name",9599,simonw,open,0,,,,,1,2021-02-18T21:08:31Z,2021-02-18T21:10:08Z,,OWNER,,"Got this error:
```
Step 6/9 : RUN datasette inspect fixtures.db extra database.db --inspect-file inspect-data.json
 ---> Running in db9da0068592
Usage: datasette inspect [OPTIONS] [FILES]...
Try 'datasette inspect --help' for help.

Error: Invalid value for '[FILES]...': Path 'extra' does not exist.
The command '/bin/sh -c datasette inspect fixtures.db extra database.db --inspect-file inspect-data.json' returned a non-zero code: 2
ERROR
ERROR: build step 0 ""gcr.io/cloud-builders/docker"" failed: step exited with non-zero status: 2
```
While working on the demo for #1232, using this deploy command:
```
GITHUB_SHA=crossdb datasette publish cloudrun fixtures.db 'extra database.db' \
            -m fixtures.json \
            --plugins-dir=plugins \
            --branch=$GITHUB_SHA \
            --version-note=$GITHUB_SHA \
            --extra-options=""--setting template_debug 1 --crossdb"" \
            --install=pysqlite3-binary \
            --service=datasette-latest-crossdb
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1233/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
586567379,MDU6SXNzdWU1ODY1NjczNzk=,22,Handle empty git repositories,9599,simonw,closed,0,,,,,0,2020-03-23T22:49:48Z,2020-03-23T23:13:11Z,2020-03-23T23:13:11Z,MEMBER,,"Got this error:
```
github_to_sqlite.utils.GitHubError: {'message': 'Git Repository is empty.', 'documentation_url': 'https://developer.github.com/v3/repos/commits/#list-commits-on-a-repository'}
```
From https://api.github.com/repos/dogsheep/beta/commits",207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/22/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
755721275,MDU6SXNzdWU3NTU3MjEyNzU=,1123,"Table actions hook are order dependent, should not be",9599,simonw,closed,0,,,,,1,2020-12-03T00:47:12Z,2020-12-03T00:50:17Z,2020-12-03T00:50:17Z,OWNER,,"Got this error: https://github.com/simonw/datasette/runs/1489770800
```
>       assert get_table_actions_links(response_2.text) == [
            {""label"": ""From async"", ""href"": ""/""},
            {""label"": ""Database: fixtures"", ""href"": ""/""},
            {""label"": f""Table: {table_or_view}"", ""href"": ""/""},
        ]
E       AssertionError: assert [{'href': '/'...'From async'}] == [{'href': '/'...: facetable'}]
E         At index 0 diff: {'label': 'Database: fixtures', 'href': '/'} != {'label': 'From async', 'href': '/'}
E         Use -v to get the full diff
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1123/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
602181581,MDU6SXNzdWU2MDIxODE1ODE=,44,"tweet[""source""] can be an empty string",9599,simonw,closed,0,,,,,0,2020-04-17T19:18:26Z,2020-04-17T22:01:44Z,2020-04-17T22:01:44Z,MEMBER,,"Got this excepion:
```
  File ""/Users/simonw/Dropbox/Development/twitter-to-sqlite/twitter_to_sqlite/utils.py"", line 641, in extract_and_save_source
    details = m.groupdict()
AttributeError: 'NoneType' object has no attribute 'groupdict'
```
I traced it back to this tweet: https://twitter.com/osder/status/578712651393576960
```
(Pdb) source_re
re.compile('<a href=""(?P<url>.*?)"".*?>(?P<name>.*?)</a>')
(Pdb) locals()['source']
''
(Pdb) u
> /Users/simonw/Dropbox/Development/twitter-to-sqlite/twitter_to_sqlite/utils.py(393)save_tweets()
-> tweet[""source""] = extract_and_save_source(db, tweet[""source""])
(Pdb) tweet
{'created_at': '2015-03-20T00:20:22+00:00', 'id': 578712651393576960, 'full_text': '@osder', 'truncated': False, 'display_text_range': [0, 6], 'source': '', 'in_reply_to_status_id': 578712521382715392, 'in_reply_to_user_id': 1545741, 'in_reply_to_screen_name': 'osder', 'geo': None, 'coordinates': None, 'place': None, 'contributors': None, 'is_quote_status': False, 'retweet_count': 0, 'favorite_count': 0, 'favorited': False, 'retweeted': False, 'lang': 'und', 'user': 1545741}
```",206156866,twitter-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/44/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1557507274,I_kwDOBm6k_c5c1azK,2005,`extra_template_vars` should be OK to return `None`,9599,simonw,open,0,,,,,1,2023-01-26T01:40:45Z,2023-01-26T01:41:50Z,,OWNER,,"Got this exception and had to make sure it always returned `{}`:

```
  File "".../python3.11/site-packages/datasette/app.py"", line 1049, in render_template
    assert isinstance(extra_vars, dict), ""extra_vars is of type {}"".format(
AssertionError: extra_vars is of type <class 'NoneType'>
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2005/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
957298475,MDU6SXNzdWU5NTcyOTg0NzU=,1407,OSError: AF_UNIX path too long in ds_unix_domain_socket_server,9599,simonw,closed,0,,,,,2,2021-07-31T18:36:06Z,2021-07-31T19:03:44Z,2021-07-31T19:03:44Z,OWNER,,"Got this exception while working on #1406.

```
    @pytest.fixture(scope=""session"")
    def ds_unix_domain_socket_server(tmp_path_factory):
        socket_folder = tmp_path_factory.mktemp(""uds"")
        uds = str(socket_folder / ""datasette.sock"")
        ds_proc = subprocess.Popen(
            [""datasette"", ""--memory"", ""--uds"", uds],
            stdout=subprocess.PIPE,
            stderr=subprocess.STDOUT,
            cwd=tempfile.gettempdir(),
        )
        # Give the server time to start
        time.sleep(1.5)
        # Check it started successfully
>       assert not ds_proc.poll(), ds_proc.stdout.read().decode(""utf-8"")
E       AssertionError: INFO:     Started server process [48453]
E         INFO:     Waiting for application startup.
E         INFO:     Application startup complete.
E         Traceback (most recent call last):
E           File ""/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/bin/datasette"", line 33, in <module>
E             sys.exit(load_entry_point('datasette', 'console_scripts', 'datasette')())
E           File ""/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.8/site-packages/click/core.py"", line 1137, in __call__
E             return self.main(*args, **kwargs)
E           File ""/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.8/site-packages/click/core.py"", line 1062, in main
E             rv = self.invoke(ctx)
E           File ""/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.8/site-packages/click/core.py"", line 1668, in invoke
E             return _process_result(sub_ctx.command.invoke(sub_ctx))
E           File ""/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.8/site-packages/click/core.py"", line 1404, in invoke
E             return ctx.invoke(self.callback, **ctx.params)
E           File ""/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.8/site-packages/click/core.py"", line 763, in invoke
E             return __callback(*args, **kwargs)
E           File ""/Users/simon/Dropbox/Development/datasette/datasette/cli.py"", line 583, in serve
E             uvicorn.run(ds.app(), **uvicorn_kwargs)
E           File ""/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.8/site-packages/uvicorn/main.py"", line 393, in run
E             server.run()
E           File ""/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.8/site-packages/uvicorn/server.py"", line 50, in run
E             loop.run_until_complete(self.serve(sockets=sockets))
E           File ""/Users/simon/.pyenv/versions/3.8.2/lib/python3.8/asyncio/base_events.py"", line 616, in run_until_complete
E             return future.result()
E           File ""/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.8/site-packages/uvicorn/server.py"", line 67, in serve
E             await self.startup(sockets=sockets)
E           File ""/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.8/site-packages/uvicorn/server.py"", line 133, in startup
E             server = await asyncio.start_unix_server(
E           File ""/Users/simon/.pyenv/versions/3.8.2/lib/python3.8/asyncio/streams.py"", line 132, in start_unix_server
E             return await loop.create_unix_server(factory, path, **kwds)
E           File ""/Users/simon/.pyenv/versions/3.8.2/lib/python3.8/asyncio/unix_events.py"", line 296, in create_unix_server
E             sock.bind(path)
E         OSError: AF_UNIX path too long
E         
E       assert not 1
E        +  where 1 = <bound method Popen.poll of <subprocess.Popen object at 0x106924af0>>()
E        +    where <bound method Popen.poll of <subprocess.Popen object at 0x106924af0>> = <subprocess.Popen object at 0x106924af0>.poll
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1407/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
813978858,MDU6SXNzdWU4MTM5Nzg4NTg=,1239,JSON filter fails if column contains spaces,9599,simonw,closed,0,,,,,1,2021-02-23T00:18:07Z,2021-02-23T00:22:53Z,2021-02-23T00:22:53Z,OWNER,,"Got this exception:

`ERROR: conn=<sqlite3.Connection object at 0x10ea68e40>, sql = 'select Address, Affiliation, County, [Has Report], [Latest report notes], [Latest report yes], Latitude, [Location Type], Longitude, Name, id, [Appointment scheduling instructions], [Availability Info], [Latest report] from locations where rowid in (\n            select locations.rowid from locations, json_each(locations.Availability Info) j\n            where j.value = :p0\n        ) and ""Latest report yes"" = :p1 order by id limit 101', params = {'p0': 'Yes: appointment required', 'p1': '1'}: near ""Info"": syntax error`",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1239/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
503190241,MDU6SXNzdWU1MDMxOTAyNDE=,584,Codec error in some CSV exports,9599,simonw,closed,0,,,,,2,2019-10-07T01:15:34Z,2021-06-17T18:13:20Z,2019-10-18T05:23:16Z,OWNER,,"Got this exploring my Swarm checkins:

![448DBFC4-71F8-4846-83C0-BEA511B2157A](https://user-images.githubusercontent.com/9599/66279259-3af53480-e865-11e9-9651-04fd2d895392.jpeg)

`/swarm/stickers.csv?stickerType=messageOnly&_size=max`",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/584/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1386530156,I_kwDOCGYnMM5SpMVs,492,Idea: ability to pass extra variables to `--convert` scripts,9599,simonw,open,0,,,,,1,2022-09-26T18:30:45Z,2022-09-26T18:33:19Z,,OWNER,,"Got this idea from this example in https://jeqo.github.io/notes/2022-09-24-ingest-logs-sqlite/

```bash
sqlite-utils insert /tmp/kafka-logs.db logs server.log.2022-09-24-21 --text --convert ""
import re
r = re.compile(r'^\[(?P<datetime>\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2},\d{3})\] (?P<level>\w+) (?P<log>(.+(\n(?\!\[).+|)+))', re.MULTILINE)
def convert(text):
    rows = [m.groupdict() for m in r.finditer(text)]
    for row in rows:
        row.update({'server': 'localhost'})
        row.update({'component': 'broker'})
    return rows
""
```
And the accompanying note:

> The `row.update` allows to label rows as I’m planning to ingest logs from different hosts and potentially different components.

This made me think: it might be neat if you could inject additional variable values into that script with extra command-line options, to make this kind of reuse easier. Something like this:

```bash
sqlite-utils insert /tmp/kafka-logs.db logs server.log.2022-09-24-21 --text --convert ""
import re
r = re.compile(r'^\[(?P<datetime>\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2},\d{3})\] (?P<level>\w+) (?P<log>(.+(\n(?\!\[).+|)+))', re.MULTILINE)
def convert(text):
    rows = [m.groupdict() for m in r.finditer(text)]
    for row in rows:
        row.update({'server': server})
        row.update({'component': component})
    return rows
"" --var server ""localhost"" --var component ""broker""
```",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/492/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1113384383,I_kwDOBm6k_c5CXOW_,1611,Avoid ever running count(*) against SpatiaLite KNN table,9599,simonw,open,0,,,,,1,2022-01-25T03:32:54Z,2022-02-02T06:45:47Z,,OWNER,,"Got this in a trace:

<img width=""941"" alt=""image"" src=""https://user-images.githubusercontent.com/9599/150906011-5f09dc6d-4def-433a-8546-e4fd94e0edf0.png"">

Looks like running `count(*)` against KNN took 83s! It ignored the time limit. And still only returned a count of 0.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1611/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
703970814,MDU6SXNzdWU3MDM5NzA4MTQ=,24,"the JSON object must be str, bytes or bytearray, not 'Undefined'",9599,simonw,closed,0,,,,,8,2020-09-17T23:21:41Z,2020-09-18T22:33:32Z,2020-09-18T22:33:32Z,MEMBER,,Got this on a search results page.,197431109,dogsheep-beta,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-beta/issues/24/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1662951875,I_kwDOBm6k_c5jHqHD,2057,DeprecationWarning: pkg_resources is deprecated as an API,9599,simonw,closed,0,,,,,25,2023-04-11T17:41:20Z,2023-09-21T22:09:10Z,2023-09-21T22:09:10Z,OWNER,,"Got this running tests against Python 3.11.

```
../../../.local/share/virtualenvs/datasette-big-local-6Yn-280V/lib/python3.11/site-packages/datasette/app.py:14: in <module>
    import pkg_resources
../../../.local/share/virtualenvs/datasette-big-local-6Yn-280V/lib/python3.11/site-packages/pkg_resources/__init__.py:121: in <module>
    warnings.warn(""pkg_resources is deprecated as an API"", DeprecationWarning)
E   DeprecationWarning: pkg_resources is deprecated as an API
```

I ran with `pytest -Werror --pdb -x` to get the debugger for that warning, but it turned out searching the code worked better. It's used in these two places:

https://github.com/simonw/datasette/blob/5890a20c374fb0812d88c9b0ef26a838bfa06c76/datasette/plugins.py#L43-L50

https://github.com/simonw/datasette/blob/5890a20c374fb0812d88c9b0ef26a838bfa06c76/datasette/app.py#L1037",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2057/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1091819089,I_kwDOCGYnMM5BE9ZR,360,MemoryError,559453,nzaar9,closed,0,,,,,1,2022-01-01T13:39:17Z,2022-03-21T04:22:46Z,2022-03-21T04:22:46Z,NONE,,"HI, when dealing with large json file (~170GB) i got the following error 
```
Traceback (most recent call last):
  File ""/usr/local/bin/sqlite-utils"", line 8, in <module>
    sys.exit(cli())
  File ""/usr/lib/python3/dist-packages/click/core.py"", line 1126, in __call__
    return self.main(*args, **kwargs)
  File ""/usr/lib/python3/dist-packages/click/core.py"", line 1051, in main
    rv = self.invoke(ctx)
  File ""/usr/lib/python3/dist-packages/click/core.py"", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File ""/usr/lib/python3/dist-packages/click/core.py"", line 1393, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File ""/usr/lib/python3/dist-packages/click/core.py"", line 752, in invoke
    return __callback(*args, **kwargs)
  File ""/usr/local/lib/python3.9/dist-packages/sqlite_utils/cli.py"", line 1300, in memory
    rows, format_used = rows_from_file(csv_fp, format=format, encoding=encoding)
  File ""/usr/local/lib/python3.9/dist-packages/sqlite_utils/utils.py"", line 185, in rows_from_file
    return rows_from_file(buffered, format=Format.JSON)
  File ""/usr/local/lib/python3.9/dist-packages/sqlite_utils/utils.py"", line 156, in rows_from_file
    decoded = json.load(fp)
  File ""/usr/lib/python3.9/json/__init__.py"", line 293, in load
    return loads(fp.read(),
MemoryError
```",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/360/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1410305897,I_kwDOBm6k_c5UD49p,1845,Reconsider the Datasette first-run experience,9599,simonw,open,0,,,,,3,2022-10-15T22:21:31Z,2022-10-16T08:54:53Z,,OWNER,,"Had a really interesting conversation today about how hard it is to get from ""I installed Datasette"" to ""I've done something useful with it"": https://news.ycombinator.com/item?id=33216789#33218590

Spending some time focusing on that first-run experience feels very worthwhile.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1845/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
616087149,MDU6SXNzdWU2MTYwODcxNDk=,765,publish heroku should default to currently tagged version,9599,simonw,open,0,,,,,1,2020-05-11T18:24:06Z,2020-05-11T18:25:43Z,,OWNER,,"Had a report that deploying to Heroku was using the previously installed version of Datasette, not the latest.

Could be because of this:

https://github.com/simonw/datasette/blob/af6c6c5d6f929f951c0e63bfd1c82e37a071b50f/datasette/publish/heroku.py#L172-L179

Heroku documentation recommends pinning to specific versions https://devcenter.heroku.com/articles/python-pip

So... we could ensure we default to an install value of `[""datasette>=current_tag""]`.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/765/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
901009787,MDU6SXNzdWU5MDEwMDk3ODc=,1340,Research: Cell action menu (like column action but for individual cells),9599,simonw,open,0,,,,,1,2021-05-25T15:49:16Z,2021-05-26T18:59:58Z,,OWNER,,"Had an idea today that it might be useful to select an individual cell and say things like ""show me all other rows with the same value"" - maybe even a set of other menu options against cells as well.

Mocked up a show-on-hover ellipses demo using the CSS inspector:

![idea](https://user-images.githubusercontent.com/9599/119528316-f0744480-bd35-11eb-8eb4-1deea6d60cce.gif)
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1340/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1423336122,I_kwDOBm6k_c5U1mK6,1856,allow_signed_tokens setting for disabling API signed token mechanism,9599,simonw,closed,0,,,8658075,Datasette 1.0a0,3,2022-10-26T02:20:55Z,2022-11-15T19:57:05Z,2022-10-26T02:58:35Z,OWNER,,"Had some design thoughts here: https://github.com/simonw/datasette/issues/1852#issuecomment-1291272280

I liked this option the most:

    --setting allow_create_tokens off",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1856/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
849396758,MDU6SXNzdWU4NDkzOTY3NTg=,1287,Upgrade to Python 3.9.4,9599,simonw,open,0,,,,,5,2021-04-02T18:43:15Z,2021-04-03T22:38:39Z,,OWNER,,Has some security fixes https://pythoninsider.blogspot.com/2021/04/python-393-and-389-are-now-available.html,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1287/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
611326701,MDU6SXNzdWU2MTEzMjY3MDE=,108,Documentation unit tests for CLI commands,9599,simonw,closed,0,,,,,2,2020-05-03T03:58:42Z,2020-05-03T04:13:57Z,2020-05-03T04:13:57Z,OWNER,,Have a test that ensures all CLI commands are documented.,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/108/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
803338729,MDU6SXNzdWU4MDMzMzg3Mjk=,33,photo-to-sqlite: command not found,11855322,robmarkcole,open,0,,,,,4,2021-02-08T08:42:57Z,2021-02-12T15:00:44Z,,NONE,,"Having installed in a venv I get:
```
(venv) (base) Robins-MacBook:datasette robin$ photo-to-sqlite apple-photos photos.db

-bash: photo-to-sqlite: command not found
```",256834907,dogsheep-photos,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-photos/issues/33/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
771324837,MDU6SXNzdWU3NzEzMjQ4Mzc=,53,--since support for favorites,27,anotherjesse,closed,0,,,,,1,2020-12-19T07:08:23Z,2020-12-19T07:47:11Z,2020-12-19T07:47:11Z,NONE,,"Having support for `--since` for updating your favorites would be ideal as the api is both slow and it only returns ~3k most recent favorites.

https://twittercommunity.com/t/cant-get-all-favorite-tweets-by-rest-api/22007/3

The api seems to take an optional `since_id` parameter - https://developer.twitter.com/en/docs/twitter-api/v1/tweets/post-and-engage/api-reference/get-favorites-list",206156866,twitter-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/53/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
892457208,MDU6SXNzdWU4OTI0NTcyMDg=,1327,Support Unicode characters in metadata.json,20846286,GmGniap,closed,0,,,,,2,2021-05-15T14:33:58Z,2021-05-24T19:10:21Z,2021-05-24T19:10:21Z,NONE,,"Hello , when I used Burmese (Unicode) characters in metadata.json like below - 
![image](https://user-images.githubusercontent.com/20846286/118364978-cba70100-b5c0-11eb-967c-7dc3b62478f2.png)

It gave wrong results when I run datasette - 
![image](https://user-images.githubusercontent.com/20846286/118365025-fc873600-b5c0-11eb-97ce-19541b8cc6d8.png)

It would be great & helpful for us if metadata.json can support in Unicode supported Asian Languages. 
Thanks & Regards. ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1327/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
343728754,MDU6SXNzdWUzNDM3Mjg3NTQ=,346,Logo design for DATASETTE,35750428,ggabogarcia,closed,0,,,,,0,2018-07-23T17:40:17Z,2018-08-02T02:31:59Z,2018-08-02T02:31:59Z,NONE,,"Hello :) , I'm a graphic designer, I'm interested in collaborating with open source projects, besides this helps me expand my portfolio. I would like to design a logo for your project. I will be happy to collaborate with you :). ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/346/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
473307794,MDU6SXNzdWU0NzMzMDc3OTQ=,565,Conflict between datasette and uvicorn click versions,440503,jonheslop,closed,0,,,,,1,2019-07-26T11:13:40Z,2020-10-02T00:09:55Z,2020-10-02T00:09:55Z,NONE,,"Hello Datasette is awesome thanks so much!

I not very familiar with Python but I think there is a problem with datasette docker builds 

I keep getting this error

```
ERROR: uvicorn 0.8.4 has requirement click==7.*, but you'll have click 6.0 which is incompatible.
ERROR: datasette 0.29.2 has requirement click~=7.0, but you'll have click 6.0 which is incompatible.
```

The full log from the docker build is here - https://gist.github.com/jonheslop/e01cd322e761cfaf34f0cb83f86411b0

Just in case it’s helpful this is my setup - https://github.com/dotwatcher/dotwatcher-data",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/565/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
408376825,MDU6SXNzdWU0MDgzNzY4MjU=,409,Zeit API v1 does not work for new users - need to migrate to v2,209967,michaelmcandrew,closed,0,,,,,3,2019-02-09T00:50:33Z,2020-04-06T15:44:46Z,2020-04-06T15:44:46Z,NONE,,"Hello there,

This looks like a great tool. Thanks. 

Unfortunately, I hit the following error:

```
michael@hazel ~/src/cc-datasette/data/out datasette publish now cc-datasette.db
> WARN! You are using an old version of the Now Platform. More: https://zeit.co/docs/v1-upgrade
> Deploying /tmp/tmpjtrxwsyf/datasette under michaelmcandrew
> Using project datasette
> Error! You tried to create a Now 1.0 deployment. Please use Now 2.0 instead: https://zeit.co/upgrade
```
I'm guessing you might not hit this because you are not a 'new user' of Zeit (https://github.com/zeit/now-cli/issues/1805#issuecomment-452470953).

Would it be a lot of work to upgrade to the new Zeit API, do you think?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/409/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1452572348,I_kwDOBm6k_c5WlH68,1900,datasette package --spatialite throws error during build,419145,rdmurphy,open,0,,,,,11,2022-11-17T02:03:28Z,2022-11-18T08:00:38Z,,NONE,,"Hello! Attempting to use `datasette package` to bundle up a SpatiaLite DB and I'm getting this error during the `docker build`:

```
sqlite3.OperationalError: /usr/lib/x86_64-linux-gnu/mod_spatialite.so.so: cannot open shared object file: No such file or directory
```

Seems to be throwing when this step is ran:

```
ERROR [6/6] RUN datasette inspect results.db --inspect-file inspect-data.json
```

This is with `v0.63.1`.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1900/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1353441389,I_kwDOCGYnMM5Qq-Bt,477,Conda Forge,49702524,thewchan,closed,0,,,,,2,2022-08-28T19:03:08Z,2022-09-07T03:46:55Z,2022-09-07T03:46:55Z,NONE,,"Hello! I have successfully put this package on to Conda Forge, and I have extending the invitation for the owner/maintainers of this package to be maintainers on Conda Forge as well. Let me know if you are interested! Thanks.
https://github.com/conda-forge/sqlite-utils-feedstock",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/477/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
982803408,MDU6SXNzdWU5ODI4MDM0MDg=,1454,Feature Request: Publish to IPFS,1560788,blitmap,open,0,,,,,0,2021-08-30T13:36:18Z,2021-08-30T13:36:18Z,,NONE,,"Hello,

I am a huge fan of this being used for exploring data.  I think it has a lot of flexibility not found in other tools.

I'm not sure if what I'm asking for is possible:  Can this be extended to publish to IPFS?

IPFS is an attractive hosting option for decentralized journalism.

Food for thought ~",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1454/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
837208901,MDU6SXNzdWU4MzcyMDg5MDE=,1267,Update Datasette alternativeto listening with details,921217,RayBB,closed,0,,,,,1,2021-03-21T23:20:20Z,2021-03-22T04:37:26Z,2021-03-22T04:37:26Z,NONE,,"Hello,

I recently learned about Datasette from an old hackernews post. It seems like an awesome project and I actually have use case I might be trying out in the coming months.

Alas, to get a better understanding of your project I looked it up on alternativeto to see what it is similar too. I promise it's not spam, it's reputable enough to have a [Wikipedia](https://en.wikipedia.org/wiki/AlternativeTo) page. There was no listing on the website so I went ahead and created a listing that is now approved.

I encourage anyone who likes this project and hopes to spread the word to help update the listing by:
1. Adding to the list of software it compares to
2. Uploading screenshots
3. Writing a review
4. Adding ""features""

I know this may seem spammy but I promise I have no affiliation with alternativeto I'm just a happy user and know it's a popular site for discovering software.

Here is the listing for datasette:
https://alternativeto.net/software/datasette/about/

Cheers",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1267/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
660827546,MDU6SXNzdWU2NjA4Mjc1NDY=,899,How to setup a request limit per user,133845,Krazybug,closed,0,,,,,1,2020-07-19T13:08:25Z,2020-07-31T23:54:42Z,2020-07-31T23:54:42Z,NONE,,"Hello,

Until now I'm using  datasette without any authentication system but I would like to setup a configuration  or limiting the number of requests per user (eventually by IP or with a cookie mechanism) and eventually allowing me to ban specific users/IPs.

Is there a plugin available for this use case ? 
If not what are your insights regarding this UC ?

Should I write a plugin ? Should I deploy datasette behind a reverse proxy to manage this ?
 ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/899/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1646068413,I_kwDOBm6k_c5iHQK9,2048,Test failures encountered while packaging for GNU Guix,8332263,Apteryks,open,0,,,,,0,2023-03-29T15:36:54Z,2023-03-29T15:36:54Z,,NONE,,"Hello,

While reviewing a packaged submitted to Guix to add `datasette`, the test suite produces the following errors:
```
=================================== FAILURES ===================================
_________________________ test_row_strange_table_name __________________________
[gw21] linux -- Python 3.9.9 /gnu/store/slsh0qjv5j68xda2bb6h8gsxwyi1j25a-python-wrapper-3.9.9/bin/python

app_client = <datasette.utils.testing.TestClient object at 0x7fffef099be0>

    def test_row_strange_table_name(app_client):
        response = app_client.get(
            ""/fixtures/table~2Fwith~2Fslashes~2Ecsv/3.json?_shape=objects""
        )
>       assert response.status == 200
E       assert 400 == 200
E        +  where 400 = <datasette.utils.testing.TestResponse object at 0x7fffef236a30>.status

/tmp/guix-build-datasette-0.64.2.drv-0/source/tests/test_api.py:701: AssertionError
----------------------------- Captured stderr call -----------------------------
ERROR: conn=<sqlite3.Connection object at 0x7fffeedfe5d0>, sql = 'select rowid, * from [table%7E2Fwith%7E2Fslashes%7E2Ecsv] where ""rowid""=:p0', params = {'p0': '3'}: no such table: table%7E2Fwith%7E2Fslashes%7E2Ecsv
_______________ test_database_page_for_database_with_dot_in_name _______________
[gw15] linux -- Python 3.9.9 /gnu/store/slsh0qjv5j68xda2bb6h8gsxwyi1j25a-python-wrapper-3.9.9/bin/python

app_client_with_dot = <datasette.utils.testing.TestClient object at 0x7fffef3416a0>

    def test_database_page_for_database_with_dot_in_name(app_client_with_dot):
        response = app_client_with_dot.get(""/fixtures~2Edot.json"")
>       assert response.status == 200
E       assert 302 == 200
E        +  where 302 = <datasette.utils.testing.TestResponse object at 0x7fffef089fa0>.status

/tmp/guix-build-datasette-0.64.2.drv-0/source/tests/test_api.py:633: AssertionError
___________________ test_tilde_encoded_database_names[fo%o] ____________________
[gw6] linux -- Python 3.9.9 /gnu/store/slsh0qjv5j68xda2bb6h8gsxwyi1j25a-python-wrapper-3.9.9/bin/python

db_name = 'fo%o'

    @pytest.mark.asyncio
    @pytest.mark.parametrize(""db_name"", (""foo"", r""fo%o"", ""f~/c.d""))
    async def test_tilde_encoded_database_names(db_name):
        ds = Datasette()
        ds.add_memory_database(db_name)
        response = await ds.client.get(""/.json"")
        assert db_name in response.json().keys()
        path = response.json()[db_name][""path""]
        # And the JSON for that database
        response2 = await ds.client.get(path + "".json"")
>       assert response2.status_code == 200
E       assert 302 == 200
E        +  where 302 = <Response [302 Found]>.status_code

/tmp/guix-build-datasette-0.64.2.drv-0/source/tests/test_api.py:983: AssertionError
__________________ test_tilde_encoded_database_names[f~/c.d] ___________________
[gw7] linux -- Python 3.9.9 /gnu/store/slsh0qjv5j68xda2bb6h8gsxwyi1j25a-python-wrapper-3.9.9/bin/python

db_name = 'f~/c.d'

    @pytest.mark.asyncio
    @pytest.mark.parametrize(""db_name"", (""foo"", r""fo%o"", ""f~/c.d""))
    async def test_tilde_encoded_database_names(db_name):
        ds = Datasette()
        ds.add_memory_database(db_name)
        response = await ds.client.get(""/.json"")
        assert db_name in response.json().keys()
        path = response.json()[db_name][""path""]
        # And the JSON for that database
        response2 = await ds.client.get(path + "".json"")
>       assert response2.status_code == 200
E       assert 302 == 200
E        +  where 302 = <Response [302 Found]>.status_code

/tmp/guix-build-datasette-0.64.2.drv-0/source/tests/test_api.py:983: AssertionError
______________ test_database_with_space_in_name[/searchable.json] ______________
[gw21] linux -- Python 3.9.9 /gnu/store/slsh0qjv5j68xda2bb6h8gsxwyi1j25a-python-wrapper-3.9.9/bin/python

app_client_two_attached_databases = <datasette.utils.testing.TestClient object at 0x7fffef11d730>
path = '/searchable.json'

    @pytest.mark.parametrize(
        ""path"",
        (
            ""/"",
            "".json"",
            ""/searchable"",
            ""/searchable.json"",
            ""/searchable_view"",
            ""/searchable_view.json"",
        ),
    )
    def test_database_with_space_in_name(app_client_two_attached_databases, path):
>       response = app_client_two_attached_databases.get(
            ""/extra~20database"" + path, follow_redirects=True
        )

/tmp/guix-build-datasette-0.64.2.drv-0/source/tests/test_api.py:920: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/gnu/store/mcclmphjgbrgpa0v037a4nlq336482g8-python-asgiref-3.4.1/lib/python3.9/site-packages/asgiref/sync.py:223: in __call__
    return call_result.result()
/gnu/store/65i3nhcwmz0p8rqbg48gaavyky4g4hwk-python-3.9.9/lib/python3.9/concurrent/futures/_base.py:438: in result
    return self.__get_result()
/gnu/store/65i3nhcwmz0p8rqbg48gaavyky4g4hwk-python-3.9.9/lib/python3.9/concurrent/futures/_base.py:390: in __get_result
    raise self._exception
/gnu/store/mcclmphjgbrgpa0v037a4nlq336482g8-python-asgiref-3.4.1/lib/python3.9/site-packages/asgiref/sync.py:292: in main_wrap
    result = await self.awaitable(*args, **kwargs)
/tmp/guix-build-datasette-0.64.2.drv-0/source/datasette/utils/testing.py:66: in get
    return await self._request(
/tmp/guix-build-datasette-0.64.2.drv-0/source/datasette/utils/testing.py:156: in _request
    httpx_response = await self.ds.client.request(
/tmp/guix-build-datasette-0.64.2.drv-0/source/datasette/app.py:1602: in request
    return await client.request(
/gnu/store/bj5lb299rfb4cbbq5kczq9imdk9a7y64-python-httpx-0.23.0/lib/python3.9/site-packages/httpx/_client.py:1527: in request
    return await self.send(request, auth=auth, follow_redirects=follow_redirects)
/gnu/store/bj5lb299rfb4cbbq5kczq9imdk9a7y64-python-httpx-0.23.0/lib/python3.9/site-packages/httpx/_client.py:1614: in send
    response = await self._send_handling_auth(
/gnu/store/bj5lb299rfb4cbbq5kczq9imdk9a7y64-python-httpx-0.23.0/lib/python3.9/site-packages/httpx/_client.py:1642: in _send_handling_auth
    response = await self._send_handling_redirects(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <httpx.AsyncClient object at 0x7fffef11d940>
request = <Request('GET', 'http://localhost/extra%7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E20database/searchable%7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E2Ejson')>
follow_redirects = True
history = [<Response [302 Found]>, <Response [302 Found]>, <Response [302 Found]>, <Response [302 Found]>, <Response [302 Found]>, <Response [302 Found]>, ...]

    async def _send_handling_redirects(
        self,
        request: Request,
        follow_redirects: bool,
        history: typing.List[Response],
    ) -> Response:
        while True:
            if len(history) > self.max_redirects:
>               raise TooManyRedirects(
                    ""Exceeded maximum allowed redirects."", request=request
                )
E               httpx.TooManyRedirects: Exceeded maximum allowed redirects.

/gnu/store/bj5lb299rfb4cbbq5kczq9imdk9a7y64-python-httpx-0.23.0/lib/python3.9/site-packages/httpx/_client.py:1672: TooManyRedirects
___________________ test_database_with_space_in_name[.json] ____________________
[gw19] linux -- Python 3.9.9 /gnu/store/slsh0qjv5j68xda2bb6h8gsxwyi1j25a-python-wrapper-3.9.9/bin/python

app_client_two_attached_databases = <datasette.utils.testing.TestClient object at 0x7fffef085a90>
path = '.json'

    @pytest.mark.parametrize(
        ""path"",
        (
            ""/"",
            "".json"",
            ""/searchable"",
            ""/searchable.json"",
            ""/searchable_view"",
            ""/searchable_view.json"",
        ),
    )
    def test_database_with_space_in_name(app_client_two_attached_databases, path):
>       response = app_client_two_attached_databases.get(
            ""/extra~20database"" + path, follow_redirects=True
        )

/tmp/guix-build-datasette-0.64.2.drv-0/source/tests/test_api.py:920: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/gnu/store/mcclmphjgbrgpa0v037a4nlq336482g8-python-asgiref-3.4.1/lib/python3.9/site-packages/asgiref/sync.py:223: in __call__
    return call_result.result()
/gnu/store/65i3nhcwmz0p8rqbg48gaavyky4g4hwk-python-3.9.9/lib/python3.9/concurrent/futures/_base.py:438: in result
    return self.__get_result()
/gnu/store/65i3nhcwmz0p8rqbg48gaavyky4g4hwk-python-3.9.9/lib/python3.9/concurrent/futures/_base.py:390: in __get_result
    raise self._exception
/gnu/store/mcclmphjgbrgpa0v037a4nlq336482g8-python-asgiref-3.4.1/lib/python3.9/site-packages/asgiref/sync.py:292: in main_wrap
    result = await self.awaitable(*args, **kwargs)
/tmp/guix-build-datasette-0.64.2.drv-0/source/datasette/utils/testing.py:66: in get
    return await self._request(
/tmp/guix-build-datasette-0.64.2.drv-0/source/datasette/utils/testing.py:156: in _request
    httpx_response = await self.ds.client.request(
/tmp/guix-build-datasette-0.64.2.drv-0/source/datasette/app.py:1602: in request
    return await client.request(
/gnu/store/bj5lb299rfb4cbbq5kczq9imdk9a7y64-python-httpx-0.23.0/lib/python3.9/site-packages/httpx/_client.py:1527: in request
    return await self.send(request, auth=auth, follow_redirects=follow_redirects)
/gnu/store/bj5lb299rfb4cbbq5kczq9imdk9a7y64-python-httpx-0.23.0/lib/python3.9/site-packages/httpx/_client.py:1614: in send
    response = await self._send_handling_auth(
/gnu/store/bj5lb299rfb4cbbq5kczq9imdk9a7y64-python-httpx-0.23.0/lib/python3.9/site-packages/httpx/_client.py:1642: in _send_handling_auth
    response = await self._send_handling_redirects(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <httpx.AsyncClient object at 0x7fffecd99ca0>
request = <Request('GET', 'http://localhost/extra%7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E20database%7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E2Ejson')>
follow_redirects = True
history = [<Response [302 Found]>, <Response [302 Found]>, <Response [302 Found]>, <Response [302 Found]>, <Response [302 Found]>, <Response [302 Found]>, ...]

    async def _send_handling_redirects(
        self,
        request: Request,
        follow_redirects: bool,
        history: typing.List[Response],
    ) -> Response:
        while True:
            if len(history) > self.max_redirects:
>               raise TooManyRedirects(
                    ""Exceeded maximum allowed redirects."", request=request
                )
E               httpx.TooManyRedirects: Exceeded maximum allowed redirects.

/gnu/store/bj5lb299rfb4cbbq5kczq9imdk9a7y64-python-httpx-0.23.0/lib/python3.9/site-packages/httpx/_client.py:1672: TooManyRedirects
______________ test_database_with_space_in_name[/searchable_view] ______________
[gw22] linux -- Python 3.9.9 /gnu/store/slsh0qjv5j68xda2bb6h8gsxwyi1j25a-python-wrapper-3.9.9/bin/python

app_client_two_attached_databases = <datasette.utils.testing.TestClient object at 0x7fffeeab4c70>
path = '/searchable_view'

    @pytest.mark.parametrize(
        ""path"",
        (
            ""/"",
            "".json"",
            ""/searchable"",
            ""/searchable.json"",
            ""/searchable_view"",
            ""/searchable_view.json"",
        ),
    )
    def test_database_with_space_in_name(app_client_two_attached_databases, path):
>       response = app_client_two_attached_databases.get(
            ""/extra~20database"" + path, follow_redirects=True
        )

/tmp/guix-build-datasette-0.64.2.drv-0/source/tests/test_api.py:920: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/gnu/store/mcclmphjgbrgpa0v037a4nlq336482g8-python-asgiref-3.4.1/lib/python3.9/site-packages/asgiref/sync.py:223: in __call__
    return call_result.result()
/gnu/store/65i3nhcwmz0p8rqbg48gaavyky4g4hwk-python-3.9.9/lib/python3.9/concurrent/futures/_base.py:438: in result
    return self.__get_result()
/gnu/store/65i3nhcwmz0p8rqbg48gaavyky4g4hwk-python-3.9.9/lib/python3.9/concurrent/futures/_base.py:390: in __get_result
    raise self._exception
/gnu/store/mcclmphjgbrgpa0v037a4nlq336482g8-python-asgiref-3.4.1/lib/python3.9/site-packages/asgiref/sync.py:292: in main_wrap
    result = await self.awaitable(*args, **kwargs)
/tmp/guix-build-datasette-0.64.2.drv-0/source/datasette/utils/testing.py:66: in get
    return await self._request(
/tmp/guix-build-datasette-0.64.2.drv-0/source/datasette/utils/testing.py:156: in _request
    httpx_response = await self.ds.client.request(
/tmp/guix-build-datasette-0.64.2.drv-0/source/datasette/app.py:1602: in request
    return await client.request(
/gnu/store/bj5lb299rfb4cbbq5kczq9imdk9a7y64-python-httpx-0.23.0/lib/python3.9/site-packages/httpx/_client.py:1527: in request
    return await self.send(request, auth=auth, follow_redirects=follow_redirects)
/gnu/store/bj5lb299rfb4cbbq5kczq9imdk9a7y64-python-httpx-0.23.0/lib/python3.9/site-packages/httpx/_client.py:1614: in send
    response = await self._send_handling_auth(
/gnu/store/bj5lb299rfb4cbbq5kczq9imdk9a7y64-python-httpx-0.23.0/lib/python3.9/site-packages/httpx/_client.py:1642: in _send_handling_auth
    response = await self._send_handling_redirects(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <httpx.AsyncClient object at 0x7fffec5b3580>
request = <Request('GET', 'http://localhost/extra%7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E20database/searchable_view')>
follow_redirects = True
history = [<Response [302 Found]>, <Response [302 Found]>, <Response [302 Found]>, <Response [302 Found]>, <Response [302 Found]>, <Response [302 Found]>, ...]

    async def _send_handling_redirects(
        self,
        request: Request,
        follow_redirects: bool,
        history: typing.List[Response],
    ) -> Response:
        while True:
            if len(history) > self.max_redirects:
>               raise TooManyRedirects(
                    ""Exceeded maximum allowed redirects."", request=request
                )
E               httpx.TooManyRedirects: Exceeded maximum allowed redirects.

/gnu/store/bj5lb299rfb4cbbq5kczq9imdk9a7y64-python-httpx-0.23.0/lib/python3.9/site-packages/httpx/_client.py:1672: TooManyRedirects
_____________________ test_database_with_space_in_name[/] ______________________
[gw18] linux -- Python 3.9.9 /gnu/store/slsh0qjv5j68xda2bb6h8gsxwyi1j25a-python-wrapper-3.9.9/bin/python

app_client_two_attached_databases = <datasette.utils.testing.TestClient object at 0x7fffef085be0>
path = '/'

    @pytest.mark.parametrize(
        ""path"",
        (
            ""/"",
            "".json"",
            ""/searchable"",
            ""/searchable.json"",
            ""/searchable_view"",
            ""/searchable_view.json"",
        ),
    )
    def test_database_with_space_in_name(app_client_two_attached_databases, path):
>       response = app_client_two_attached_databases.get(
            ""/extra~20database"" + path, follow_redirects=True
        )

/tmp/guix-build-datasette-0.64.2.drv-0/source/tests/test_api.py:920: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/gnu/store/mcclmphjgbrgpa0v037a4nlq336482g8-python-asgiref-3.4.1/lib/python3.9/site-packages/asgiref/sync.py:223: in __call__
    return call_result.result()
/gnu/store/65i3nhcwmz0p8rqbg48gaavyky4g4hwk-python-3.9.9/lib/python3.9/concurrent/futures/_base.py:438: in result
    return self.__get_result()
/gnu/store/65i3nhcwmz0p8rqbg48gaavyky4g4hwk-python-3.9.9/lib/python3.9/concurrent/futures/_base.py:390: in __get_result
    raise self._exception
/gnu/store/mcclmphjgbrgpa0v037a4nlq336482g8-python-asgiref-3.4.1/lib/python3.9/site-packages/asgiref/sync.py:292: in main_wrap
    result = await self.awaitable(*args, **kwargs)
/tmp/guix-build-datasette-0.64.2.drv-0/source/datasette/utils/testing.py:66: in get
    return await self._request(
/tmp/guix-build-datasette-0.64.2.drv-0/source/datasette/utils/testing.py:156: in _request
    httpx_response = await self.ds.client.request(
/tmp/guix-build-datasette-0.64.2.drv-0/source/datasette/app.py:1602: in request
    return await client.request(
/gnu/store/bj5lb299rfb4cbbq5kczq9imdk9a7y64-python-httpx-0.23.0/lib/python3.9/site-packages/httpx/_client.py:1527: in request
    return await self.send(request, auth=auth, follow_redirects=follow_redirects)
/gnu/store/bj5lb299rfb4cbbq5kczq9imdk9a7y64-python-httpx-0.23.0/lib/python3.9/site-packages/httpx/_client.py:1614: in send
    response = await self._send_handling_auth(
/gnu/store/bj5lb299rfb4cbbq5kczq9imdk9a7y64-python-httpx-0.23.0/lib/python3.9/site-packages/httpx/_client.py:1642: in _send_handling_auth
    response = await self._send_handling_redirects(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <httpx.AsyncClient object at 0x7fffec5f1370>
request = <Request('GET', 'http://localhost/extra%7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E20database/')>
follow_redirects = True
history = [<Response [302 Found]>, <Response [302 Found]>, <Response [302 Found]>, <Response [302 Found]>, <Response [302 Found]>, <Response [302 Found]>, ...]

    async def _send_handling_redirects(
        self,
        request: Request,
        follow_redirects: bool,
        history: typing.List[Response],
    ) -> Response:
        while True:
            if len(history) > self.max_redirects:
>               raise TooManyRedirects(
                    ""Exceeded maximum allowed redirects."", request=request
                )
E               httpx.TooManyRedirects: Exceeded maximum allowed redirects.

/gnu/store/bj5lb299rfb4cbbq5kczq9imdk9a7y64-python-httpx-0.23.0/lib/python3.9/site-packages/httpx/_client.py:1672: TooManyRedirects
________________ test_database_with_space_in_name[/searchable] _________________
[gw20] linux -- Python 3.9.9 /gnu/store/slsh0qjv5j68xda2bb6h8gsxwyi1j25a-python-wrapper-3.9.9/bin/python

app_client_two_attached_databases = <datasette.utils.testing.TestClient object at 0x7fffef099f10>
path = '/searchable'

    @pytest.mark.parametrize(
        ""path"",
        (
            ""/"",
            "".json"",
            ""/searchable"",
            ""/searchable.json"",
            ""/searchable_view"",
            ""/searchable_view.json"",
        ),
    )
    def test_database_with_space_in_name(app_client_two_attached_databases, path):
>       response = app_client_two_attached_databases.get(
            ""/extra~20database"" + path, follow_redirects=True
        )

/tmp/guix-build-datasette-0.64.2.drv-0/source/tests/test_api.py:920: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/gnu/store/mcclmphjgbrgpa0v037a4nlq336482g8-python-asgiref-3.4.1/lib/python3.9/site-packages/asgiref/sync.py:223: in __call__
    return call_result.result()
/gnu/store/65i3nhcwmz0p8rqbg48gaavyky4g4hwk-python-3.9.9/lib/python3.9/concurrent/futures/_base.py:438: in result
    return self.__get_result()
/gnu/store/65i3nhcwmz0p8rqbg48gaavyky4g4hwk-python-3.9.9/lib/python3.9/concurrent/futures/_base.py:390: in __get_result
    raise self._exception
/gnu/store/mcclmphjgbrgpa0v037a4nlq336482g8-python-asgiref-3.4.1/lib/python3.9/site-packages/asgiref/sync.py:292: in main_wrap
    result = await self.awaitable(*args, **kwargs)
/tmp/guix-build-datasette-0.64.2.drv-0/source/datasette/utils/testing.py:66: in get
    return await self._request(
/tmp/guix-build-datasette-0.64.2.drv-0/source/datasette/utils/testing.py:156: in _request
    httpx_response = await self.ds.client.request(
/tmp/guix-build-datasette-0.64.2.drv-0/source/datasette/app.py:1602: in request
    return await client.request(
/gnu/store/bj5lb299rfb4cbbq5kczq9imdk9a7y64-python-httpx-0.23.0/lib/python3.9/site-packages/httpx/_client.py:1527: in request
    return await self.send(request, auth=auth, follow_redirects=follow_redirects)
/gnu/store/bj5lb299rfb4cbbq5kczq9imdk9a7y64-python-httpx-0.23.0/lib/python3.9/site-packages/httpx/_client.py:1614: in send
    response = await self._send_handling_auth(
/gnu/store/bj5lb299rfb4cbbq5kczq9imdk9a7y64-python-httpx-0.23.0/lib/python3.9/site-packages/httpx/_client.py:1642: in _send_handling_auth
    response = await self._send_handling_redirects(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <httpx.AsyncClient object at 0x7fffecd8c790>
request = <Request('GET', 'http://localhost/extra%7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E20database/searchable')>
follow_redirects = True
history = [<Response [302 Found]>, <Response [302 Found]>, <Response [302 Found]>, <Response [302 Found]>, <Response [302 Found]>, <Response [302 Found]>, ...]

    async def _send_handling_redirects(
        self,
        request: Request,
        follow_redirects: bool,
        history: typing.List[Response],
    ) -> Response:
        while True:
            if len(history) > self.max_redirects:
>               raise TooManyRedirects(
                    ""Exceeded maximum allowed redirects."", request=request
                )
E               httpx.TooManyRedirects: Exceeded maximum allowed redirects.

/gnu/store/bj5lb299rfb4cbbq5kczq9imdk9a7y64-python-httpx-0.23.0/lib/python3.9/site-packages/httpx/_client.py:1672: TooManyRedirects
___________ test_database_with_space_in_name[/searchable_view.json] ____________
[gw23] linux -- Python 3.9.9 /gnu/store/slsh0qjv5j68xda2bb6h8gsxwyi1j25a-python-wrapper-3.9.9/bin/python

app_client_two_attached_databases = <datasette.utils.testing.TestClient object at 0x7fffef341520>
path = '/searchable_view.json'

    @pytest.mark.parametrize(
        ""path"",
        (
            ""/"",
            "".json"",
            ""/searchable"",
            ""/searchable.json"",
            ""/searchable_view"",
            ""/searchable_view.json"",
        ),
    )
    def test_database_with_space_in_name(app_client_two_attached_databases, path):
>       response = app_client_two_attached_databases.get(
            ""/extra~20database"" + path, follow_redirects=True
        )

/tmp/guix-build-datasette-0.64.2.drv-0/source/tests/test_api.py:920: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/gnu/store/mcclmphjgbrgpa0v037a4nlq336482g8-python-asgiref-3.4.1/lib/python3.9/site-packages/asgiref/sync.py:223: in __call__
    return call_result.result()
/gnu/store/65i3nhcwmz0p8rqbg48gaavyky4g4hwk-python-3.9.9/lib/python3.9/concurrent/futures/_base.py:438: in result
    return self.__get_result()
/gnu/store/65i3nhcwmz0p8rqbg48gaavyky4g4hwk-python-3.9.9/lib/python3.9/concurrent/futures/_base.py:390: in __get_result
    raise self._exception
/gnu/store/mcclmphjgbrgpa0v037a4nlq336482g8-python-asgiref-3.4.1/lib/python3.9/site-packages/asgiref/sync.py:292: in main_wrap
    result = await self.awaitable(*args, **kwargs)
/tmp/guix-build-datasette-0.64.2.drv-0/source/datasette/utils/testing.py:66: in get
    return await self._request(
/tmp/guix-build-datasette-0.64.2.drv-0/source/datasette/utils/testing.py:156: in _request
    httpx_response = await self.ds.client.request(
/tmp/guix-build-datasette-0.64.2.drv-0/source/datasette/app.py:1602: in request
    return await client.request(
/gnu/store/bj5lb299rfb4cbbq5kczq9imdk9a7y64-python-httpx-0.23.0/lib/python3.9/site-packages/httpx/_client.py:1527: in request
    return await self.send(request, auth=auth, follow_redirects=follow_redirects)
/gnu/store/bj5lb299rfb4cbbq5kczq9imdk9a7y64-python-httpx-0.23.0/lib/python3.9/site-packages/httpx/_client.py:1614: in send
    response = await self._send_handling_auth(
/gnu/store/bj5lb299rfb4cbbq5kczq9imdk9a7y64-python-httpx-0.23.0/lib/python3.9/site-packages/httpx/_client.py:1642: in _send_handling_auth
    response = await self._send_handling_redirects(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <httpx.AsyncClient object at 0x7fffef085460>
request = <Request('GET', 'http://localhost/extra%7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E20database/searchable_view%7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E7E2Ejson')>
follow_redirects = True
history = [<Response [302 Found]>, <Response [302 Found]>, <Response [302 Found]>, <Response [302 Found]>, <Response [302 Found]>, <Response [302 Found]>, ...]

    async def _send_handling_redirects(
        self,
        request: Request,
        follow_redirects: bool,
        history: typing.List[Response],
    ) -> Response:
        while True:
            if len(history) > self.max_redirects:
>               raise TooManyRedirects(
                    ""Exceeded maximum allowed redirects."", request=request
                )
E               httpx.TooManyRedirects: Exceeded maximum allowed redirects.

/gnu/store/bj5lb299rfb4cbbq5kczq9imdk9a7y64-python-httpx-0.23.0/lib/python3.9/site-packages/httpx/_client.py:1672: TooManyRedirects
________________ test_weird_database_names[database (1).sqlite] ________________
[gw7] linux -- Python 3.9.9 /gnu/store/slsh0qjv5j68xda2bb6h8gsxwyi1j25a-python-wrapper-3.9.9/bin/python

tmpdir = local('/tmp/guix-build-datasette-0.64.2.drv-0/pytest-of-nixbld/pytest-0/popen-gw7/test_weird_database_names_data0')
filename = 'database (1).sqlite'

    @pytest.mark.parametrize(
        ""filename"", [""test-database (1).sqlite"", ""database (1).sqlite""]
    )
    def test_weird_database_names(tmpdir, filename):
        # https://github.com/simonw/datasette/issues/1181
        runner = CliRunner()
        db_path = str(tmpdir / filename)
        sqlite3.connect(db_path).execute(""vacuum"")
        result1 = runner.invoke(cli, [db_path, ""--get"", ""/""])
        assert result1.exit_code == 0, result1.output
        filename_no_stem = filename.rsplit(""."", 1)[0]
        expected_link = '<a href=""/{}"">{}</a>'.format(
            tilde_encode(filename_no_stem), filename_no_stem
        )
        assert expected_link in result1.output
        # Now try hitting that database page
        result2 = runner.invoke(
            cli, [db_path, ""--get"", ""/{}"".format(tilde_encode(filename_no_stem))]
        )
>       assert result2.exit_code == 0, result2.output
E       AssertionError: 
E         
E       assert 1 == 0
E        +  where 1 = <Result SystemExit(1)>.exit_code

/tmp/guix-build-datasette-0.64.2.drv-0/source/tests/test_cli.py:321: AssertionError
_____________ test_weird_database_names[test-database (1).sqlite] ______________
[gw6] linux -- Python 3.9.9 /gnu/store/slsh0qjv5j68xda2bb6h8gsxwyi1j25a-python-wrapper-3.9.9/bin/python

tmpdir = local('/tmp/guix-build-datasette-0.64.2.drv-0/pytest-of-nixbld/pytest-0/popen-gw6/test_weird_database_names_test0')
filename = 'test-database (1).sqlite'

    @pytest.mark.parametrize(
        ""filename"", [""test-database (1).sqlite"", ""database (1).sqlite""]
    )
    def test_weird_database_names(tmpdir, filename):
        # https://github.com/simonw/datasette/issues/1181
        runner = CliRunner()
        db_path = str(tmpdir / filename)
        sqlite3.connect(db_path).execute(""vacuum"")
        result1 = runner.invoke(cli, [db_path, ""--get"", ""/""])
        assert result1.exit_code == 0, result1.output
        filename_no_stem = filename.rsplit(""."", 1)[0]
        expected_link = '<a href=""/{}"">{}</a>'.format(
            tilde_encode(filename_no_stem), filename_no_stem
        )
        assert expected_link in result1.output
        # Now try hitting that database page
        result2 = runner.invoke(
            cli, [db_path, ""--get"", ""/{}"".format(tilde_encode(filename_no_stem))]
        )
>       assert result2.exit_code == 0, result2.output
E       AssertionError: 
E         
E       assert 1 == 0
E        +  where 1 = <Result SystemExit(1)>.exit_code

/tmp/guix-build-datasette-0.64.2.drv-0/source/tests/test_cli.py:321: AssertionError
_ test_row_html_compound_primary_key[/fixtures/compound_primary_key/a~2Fb,~2Ec~2Dd-expected1] _
[gw11] linux -- Python 3.9.9 /gnu/store/slsh0qjv5j68xda2bb6h8gsxwyi1j25a-python-wrapper-3.9.9/bin/python

app_client = <datasette.utils.testing.TestClient object at 0x7fffec2d37f0>
path = '/fixtures/compound_primary_key/a~2Fb,~2Ec~2Dd'
expected = [['<td class=""col-pk1 type-str"">a/b</td>', '<td class=""col-pk2 type-str"">.c-d</td>', '<td class=""col-content type-str"">c</td>']]

    @pytest.mark.parametrize(
        ""path,expected"",
        (
            (
                ""/fixtures/compound_primary_key/a,b"",
                [
                    [
                        '<td class=""col-pk1 type-str"">a</td>',
                        '<td class=""col-pk2 type-str"">b</td>',
                        '<td class=""col-content type-str"">c</td>',
                    ]
                ],
            ),
            (
                ""/fixtures/compound_primary_key/a~2Fb,~2Ec~2Dd"",
                [
                    [
                        '<td class=""col-pk1 type-str"">a/b</td>',
                        '<td class=""col-pk2 type-str"">.c-d</td>',
                        '<td class=""col-content type-str"">c</td>',
                    ]
                ],
            ),
        ),
    )
    def test_row_html_compound_primary_key(app_client, path, expected):
        response = app_client.get(path)
>       assert response.status == 200
E       assert 302 == 200
E        +  where 302 = <datasette.utils.testing.TestResponse object at 0x7fffd47c23a0>.status

/tmp/guix-build-datasette-0.64.2.drv-0/source/tests/test_html.py:370: AssertionError
_ test_css_classes_on_body[/fixtures/table~2Fwith~2Fslashes~2Ecsv-expected_classes5] _
[gw3] linux -- Python 3.9.9 /gnu/store/slsh0qjv5j68xda2bb6h8gsxwyi1j25a-python-wrapper-3.9.9/bin/python

app_client = <datasette.utils.testing.TestClient object at 0x7fffd4743fa0>
path = '/fixtures/table~2Fwith~2Fslashes~2Ecsv'
expected_classes = ['table', 'db-fixtures', 'table-tablewithslashescsv-fa7563']

    @pytest.mark.parametrize(
        ""path,expected_classes"",
        [
            (""/"", [""index""]),
            (""/fixtures"", [""db"", ""db-fixtures""]),
            (""/fixtures?sql=select+1"", [""query"", ""db-fixtures""]),
            (
                ""/fixtures/simple_primary_key"",
                [""table"", ""db-fixtures"", ""table-simple_primary_key""],
            ),
            (
                ""/fixtures/neighborhood_search"",
                [""query"", ""db-fixtures"", ""query-neighborhood_search""],
            ),
            (
                ""/fixtures/table~2Fwith~2Fslashes~2Ecsv"",
                [""table"", ""db-fixtures"", ""table-tablewithslashescsv-fa7563""],
            ),
            (
                ""/fixtures/simple_primary_key/1"",
                [""row"", ""db-fixtures"", ""table-simple_primary_key""],
            ),
        ],
    )
    def test_css_classes_on_body(app_client, path, expected_classes):
        response = app_client.get(path)
>       assert response.status == 200
E       assert 302 == 200
E        +  where 302 = <datasette.utils.testing.TestResponse object at 0x7fffd4628dc0>.status

/tmp/guix-build-datasette-0.64.2.drv-0/source/tests/test_html.py:238: AssertionError
_ test_templates_considered[/fixtures/table~2Fwith~2Fslashes~2Ecsv-table-fixtures-tablewithslashescsv-fa7563.html, *table.html] _
[gw3] linux -- Python 3.9.9 /gnu/store/slsh0qjv5j68xda2bb6h8gsxwyi1j25a-python-wrapper-3.9.9/bin/python

app_client = <datasette.utils.testing.TestClient object at 0x7fffd4743fa0>
path = '/fixtures/table~2Fwith~2Fslashes~2Ecsv'
expected_considered = 'table-fixtures-tablewithslashescsv-fa7563.html, *table.html'

    @pytest.mark.parametrize(
        ""path,expected_considered"",
        [
            (""/"", ""*index.html""),
            (""/fixtures"", ""database-fixtures.html, *database.html""),
            (
                ""/fixtures/simple_primary_key"",
                ""table-fixtures-simple_primary_key.html, *table.html"",
            ),
            (
                ""/fixtures/table~2Fwith~2Fslashes~2Ecsv"",
                ""table-fixtures-tablewithslashescsv-fa7563.html, *table.html"",
            ),
            (
                ""/fixtures/simple_primary_key/1"",
                ""row-fixtures-simple_primary_key.html, *row.html"",
            ),
        ],
    )
    def test_templates_considered(app_client, path, expected_considered):
        response = app_client.get(path)
>       assert response.status == 200
E       assert 302 == 200
E        +  where 302 = <datasette.utils.testing.TestResponse object at 0x7fffd44f2d60>.status

/tmp/guix-build-datasette-0.64.2.drv-0/source/tests/test_html.py:264: AssertionError
_ test_alternate_url_json[/fixtures/table~2Fwith~2Fslashes~2Ecsv-http://localhost/fixtures/table~2Fwith~2Fslashes~2Ecsv.json] _
[gw21] linux -- Python 3.9.9 /gnu/store/slsh0qjv5j68xda2bb6h8gsxwyi1j25a-python-wrapper-3.9.9/bin/python

app_client = <datasette.utils.testing.TestClient object at 0x7fffecd9fac0>
path = '/fixtures/table~2Fwith~2Fslashes~2Ecsv'
expected = 'http://localhost/fixtures/table~2Fwith~2Fslashes~2Ecsv.json'

    @pytest.mark.parametrize(
        ""path,expected"",
        (
            # Instance index page
            (""/"", ""http://localhost/.json""),
            # Table page
            (""/fixtures/facetable"", ""http://localhost/fixtures/facetable.json""),
            (
                ""/fixtures/table~2Fwith~2Fslashes~2Ecsv"",
                ""http://localhost/fixtures/table~2Fwith~2Fslashes~2Ecsv.json"",
            ),
            # Row page
            (
                ""/fixtures/no_primary_key/1"",
                ""http://localhost/fixtures/no_primary_key/1.json"",
            ),
            # Database index page
            (
                ""/fixtures"",
                ""http://localhost/fixtures.json"",
            ),
            # Custom query page
            (
                ""/fixtures?sql=select+*+from+facetable"",
                ""http://localhost/fixtures.json?sql=select+*+from+facetable"",
            ),
            # Canned query page
            (
                ""/fixtures/neighborhood_search?text=town"",
                ""http://localhost/fixtures/neighborhood_search.json?text=town"",
            ),
            # /-/ pages
            (
                ""/-/plugins"",
                ""http://localhost/-/plugins.json"",
            ),
        ),
    )
    def test_alternate_url_json(app_client, path, expected):
        response = app_client.get(path)
>       assert response.status == 200
E       assert 302 == 200
E        +  where 302 = <datasette.utils.testing.TestResponse object at 0x7fffd4650b80>.status

/tmp/guix-build-datasette-0.64.2.drv-0/source/tests/test_html.py:948: AssertionError
_ test_edit_sql_link_on_canned_queries[/fixtures/~F0~9D~90~9C~F0~9D~90~A2~F0~9D~90~AD~F0~9D~90~A2~F0~9D~90~9E~F0~9D~90~AC-/fixtures?sql=select+id%2C+name+from+facet_cities+order+by+id+limit+1%3B] _
[gw18] linux -- Python 3.9.9 /gnu/store/slsh0qjv5j68xda2bb6h8gsxwyi1j25a-python-wrapper-3.9.9/bin/python

app_client = <datasette.utils.testing.TestClient object at 0x7fffec5952e0>
path = '/fixtures/~F0~9D~90~9C~F0~9D~90~A2~F0~9D~90~AD~F0~9D~90~A2~F0~9D~90~9E~F0~9D~90~AC'
expected = '/fixtures?sql=select+id%2C+name+from+facet_cities+order+by+id+limit+1%3B'

    @pytest.mark.parametrize(
        ""path,expected"",
        [
            (
                ""/fixtures/neighborhood_search"",
                ""/fixtures?sql=%0Aselect+_neighborhood%2C+facet_cities.name%2C+state%0Afrom+facetable%0A++++join+facet_cities%0A++++++++on+facetable._city_id+%3D+facet_cities.id%0Awhere+_neighborhood+like+%27%25%27+%7C%7C+%3Atext+%7C%7C+%27%25%27%0Aorder+by+_neighborhood%3B%0A&amp;text="",
            ),
            (
                ""/fixtures/neighborhood_search?text=ber"",
                ""/fixtures?sql=%0Aselect+_neighborhood%2C+facet_cities.name%2C+state%0Afrom+facetable%0A++++join+facet_cities%0A++++++++on+facetable._city_id+%3D+facet_cities.id%0Awhere+_neighborhood+like+%27%25%27+%7C%7C+%3Atext+%7C%7C+%27%25%27%0Aorder+by+_neighborhood%3B%0A&amp;text=ber"",
            ),
            (""/fixtures/pragma_cache_size"", None),
            (
                # /fixtures/𝐜𝐢𝐭𝐢𝐞𝐬
                ""/fixtures/~F0~9D~90~9C~F0~9D~90~A2~F0~9D~90~AD~F0~9D~90~A2~F0~9D~90~9E~F0~9D~90~AC"",
                ""/fixtures?sql=select+id%2C+name+from+facet_cities+order+by+id+limit+1%3B"",
            ),
            (""/fixtures/magic_parameters"", None),
        ],
    )
    def test_edit_sql_link_on_canned_queries(app_client, path, expected):
        response = app_client.get(path)
>       assert response.status == 200
E       assert 302 == 200
E        +  where 302 = <datasette.utils.testing.TestResponse object at 0x7fffec4a6f70>.status

/tmp/guix-build-datasette-0.64.2.drv-0/source/tests/test_html.py:841: AssertionError
_______________________ test_table_with_slashes_in_name ________________________
[gw9] linux -- Python 3.9.9 /gnu/store/slsh0qjv5j68xda2bb6h8gsxwyi1j25a-python-wrapper-3.9.9/bin/python

app_client = <datasette.utils.testing.TestClient object at 0x7fffec0860a0>

    def test_table_with_slashes_in_name(app_client):
        response = app_client.get(
            ""/fixtures/table~2Fwith~2Fslashes~2Ecsv.json?_shape=objects""
        )
>       assert response.status == 200
E       assert 302 == 200
E        +  where 302 = <datasette.utils.testing.TestResponse object at 0x7fffec086fa0>.status

/tmp/guix-build-datasette-0.64.2.drv-0/source/tests/test_table_api.py:141: AssertionError
__________________ test_custom_query_with_unicode_characters ___________________
[gw8] linux -- Python 3.9.9 /gnu/store/slsh0qjv5j68xda2bb6h8gsxwyi1j25a-python-wrapper-3.9.9/bin/python

app_client = <datasette.utils.testing.TestClient object at 0x7fffec1cda90>

    def test_custom_query_with_unicode_characters(app_client):
        # /fixtures/𝐜𝐢𝐭𝐢𝐞𝐬.json
        response = app_client.get(
            ""/fixtures/~F0~9D~90~9C~F0~9D~90~A2~F0~9D~90~AD~F0~9D~90~A2~F0~9D~90~9E~F0~9D~90~AC.json?_shape=array""
        )
>       assert [{""id"": 1, ""name"": ""San Francisco""}] == response.json

/tmp/guix-build-datasette-0.64.2.drv-0/source/tests/test_table_api.py:1042: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/tmp/guix-build-datasette-0.64.2.drv-0/source/datasette/utils/testing.py:40: in json
    return json.loads(self.text)
/gnu/store/65i3nhcwmz0p8rqbg48gaavyky4g4hwk-python-3.9.9/lib/python3.9/json/__init__.py:346: in loads
    return _default_decoder.decode(s)
/gnu/store/65i3nhcwmz0p8rqbg48gaavyky4g4hwk-python-3.9.9/lib/python3.9/json/decoder.py:337: in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <json.decoder.JSONDecoder object at 0x7ffff7479760>, s = '', idx = 0

    def raw_decode(self, s, idx=0):
        """"""Decode a JSON document from ``s`` (a ``str`` beginning with
        a JSON document) and return a 2-tuple of the Python
        representation and the index in ``s`` where the document ended.
    
        This can be used to decode a JSON document from a string that may
        have extraneous data at the end.
    
        """"""
        try:
            obj, end = self.scan_once(s, idx)
        except StopIteration as err:
>           raise JSONDecodeError(""Expecting value"", s, err.value) from None
E           json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

/gnu/store/65i3nhcwmz0p8rqbg48gaavyky4g4hwk-python-3.9.9/lib/python3.9/json/decoder.py:355: JSONDecodeError
_ test_searchable[/fixtures/searchable.json?_search=te*+AND+do*&_searchmode=raw-expected_rows3] _
[gw13] linux -- Python 3.9.9 /gnu/store/slsh0qjv5j68xda2bb6h8gsxwyi1j25a-python-wrapper-3.9.9/bin/python

app_client = <datasette.utils.testing.TestClient object at 0x7fffd470bf10>
path = '/fixtures/searchable.json?_search=te*+AND+do*&_searchmode=raw'
expected_rows = [[1, 'barry cat', 'terry dog', 'panther'], [2, 'terry dog', 'sara weasel', 'puma']]

    @pytest.mark.parametrize(
        ""path,expected_rows"",
        [
            (
                ""/fixtures/searchable.json?_search=dog"",
                [
                    [1, ""barry cat"", ""terry dog"", ""panther""],
                    [2, ""terry dog"", ""sara weasel"", ""puma""],
                ],
            ),
            (
                # Special keyword shouldn't break FTS query
                ""/fixtures/searchable.json?_search=AND"",
                [],
            ),
            (
                # Without _searchmode=raw this should return no results
                ""/fixtures/searchable.json?_search=te*+AND+do*"",
                [],
            ),
            (
                # _searchmode=raw
                ""/fixtures/searchable.json?_search=te*+AND+do*&_searchmode=raw"",
                [
                    [1, ""barry cat"", ""terry dog"", ""panther""],
                    [2, ""terry dog"", ""sara weasel"", ""puma""],
                ],
            ),
            (
                # _searchmode=raw combined with _search_COLUMN
                ""/fixtures/searchable.json?_search_text2=te*&_searchmode=raw"",
                [
                    [1, ""barry cat"", ""terry dog"", ""panther""],
                ],
            ),
            (
                ""/fixtures/searchable.json?_search=weasel"",
                [[2, ""terry dog"", ""sara weasel"", ""puma""]],
            ),
            (
                ""/fixtures/searchable.json?_search_text2=dog"",
                [[1, ""barry cat"", ""terry dog"", ""panther""]],
            ),
            (
                ""/fixtures/searchable.json?_search_name%20with%20.%20and%20spaces=panther"",
                [[1, ""barry cat"", ""terry dog"", ""panther""]],
            ),
        ],
    )
    def test_searchable(app_client, path, expected_rows):
        response = app_client.get(path)
>       assert expected_rows == response.json[""rows""]
E       AssertionError: assert [[1, 'barry cat', 'terry dog', 'panther'],\n [2, 'terry dog', 'sara weasel', 'puma']] == []
E         Left contains 2 more items, first extra item: [1, 'barry cat', 'terry dog', 'panther']
E         Full diff:
E           [
E         -  ,
E         +  [1,
E         +   'barry cat',
E         +   'terry dog',
E         +   'panther'],
E         +  [2,
E         +   'terry dog',
E         +   'sara weasel',
E         +   'puma'],
E           ]

/tmp/guix-build-datasette-0.64.2.drv-0/source/tests/test_table_api.py:402: AssertionError
_____ test_searchmode[table_metadata1-_search=te*+AND+do*-expected_rows1] ______
[gw20] linux -- Python 3.9.9 /gnu/store/slsh0qjv5j68xda2bb6h8gsxwyi1j25a-python-wrapper-3.9.9/bin/python

table_metadata = {'searchmode': 'raw'}, querystring = '_search=te*+AND+do*'
expected_rows = [[1, 'barry cat', 'terry dog', 'panther'], [2, 'terry dog', 'sara weasel', 'puma']]

    @pytest.mark.parametrize(
        ""table_metadata,querystring,expected_rows"",
        [
            (
                {},
                ""_search=te*+AND+do*"",
                [],
            ),
            (
                {""searchmode"": ""raw""},
                ""_search=te*+AND+do*"",
                _SEARCHMODE_RAW_RESULTS,
            ),
            (
                {},
                ""_search=te*+AND+do*&_searchmode=raw"",
                _SEARCHMODE_RAW_RESULTS,
            ),
            # Can be over-ridden with _searchmode=escaped
            (
                {""searchmode"": ""raw""},
                ""_search=te*+AND+do*&_searchmode=escaped"",
                [],
            ),
        ],
    )
    def test_searchmode(table_metadata, querystring, expected_rows):
        with make_app_client(
            metadata={""databases"": {""fixtures"": {""tables"": {""searchable"": table_metadata}}}}
        ) as client:
            response = client.get(""/fixtures/searchable.json?"" + querystring)
>           assert expected_rows == response.json[""rows""]
E           AssertionError: assert [[1, 'barry cat', 'terry dog', 'panther'],\n [2, 'terry dog', 'sara weasel', 'puma']] == []
E             Left contains 2 more items, first extra item: [1, 'barry cat', 'terry dog', 'panther']
E             Full diff:
E               [
E             -  ,
E             +  [1,
E             +   'barry cat',
E             +   'terry dog',
E             +   'panther'],
E             +  [2,
E             +   'terry dog',
E             +   'sara weasel',
E             +   'puma'],
E               ]

/tmp/guix-build-datasette-0.64.2.drv-0/source/tests/test_table_api.py:442: AssertionError
_ test_searchmode[table_metadata2-_search=te*+AND+do*&_searchmode=raw-expected_rows2] _
[gw20] linux -- Python 3.9.9 /gnu/store/slsh0qjv5j68xda2bb6h8gsxwyi1j25a-python-wrapper-3.9.9/bin/python

table_metadata = {}, querystring = '_search=te*+AND+do*&_searchmode=raw'
expected_rows = [[1, 'barry cat', 'terry dog', 'panther'], [2, 'terry dog', 'sara weasel', 'puma']]

    @pytest.mark.parametrize(
        ""table_metadata,querystring,expected_rows"",
        [
            (
                {},
                ""_search=te*+AND+do*"",
                [],
            ),
            (
                {""searchmode"": ""raw""},
                ""_search=te*+AND+do*"",
                _SEARCHMODE_RAW_RESULTS,
            ),
            (
                {},
                ""_search=te*+AND+do*&_searchmode=raw"",
                _SEARCHMODE_RAW_RESULTS,
            ),
            # Can be over-ridden with _searchmode=escaped
            (
                {""searchmode"": ""raw""},
                ""_search=te*+AND+do*&_searchmode=escaped"",
                [],
            ),
        ],
    )
    def test_searchmode(table_metadata, querystring, expected_rows):
        with make_app_client(
            metadata={""databases"": {""fixtures"": {""tables"": {""searchable"": table_metadata}}}}
        ) as client:
            response = client.get(""/fixtures/searchable.json?"" + querystring)
>           assert expected_rows == response.json[""rows""]
E           AssertionError: assert [[1, 'barry cat', 'terry dog', 'panther'],\n [2, 'terry dog', 'sara weasel', 'puma']] == []
E             Left contains 2 more items, first extra item: [1, 'barry cat', 'terry dog', 'panther']
E             Full diff:
E               [
E             -  ,
E             +  [1,
E             +   'barry cat',
E             +   'terry dog',
E             +   'panther'],
E             +  [2,
E             +   'terry dog',
E             +   'sara weasel',
E             +   'puma'],
E               ]

/tmp/guix-build-datasette-0.64.2.drv-0/source/tests/test_table_api.py:442: AssertionError
=========================== short test summary info ============================
FAILED tests/test_api.py::test_row_strange_table_name - assert 400 == 200
FAILED tests/test_api.py::test_database_page_for_database_with_dot_in_name - ...
FAILED tests/test_api.py::test_tilde_encoded_database_names[fo%o] - assert 30...
FAILED tests/test_api.py::test_tilde_encoded_database_names[f~/c.d] - assert ...
FAILED tests/test_api.py::test_database_with_space_in_name[/searchable.json]
FAILED tests/test_api.py::test_database_with_space_in_name[.json] - httpx.Too...
FAILED tests/test_api.py::test_database_with_space_in_name[/searchable_view]
FAILED tests/test_api.py::test_database_with_space_in_name[/] - httpx.TooMany...
FAILED tests/test_api.py::test_database_with_space_in_name[/searchable] - htt...
FAILED tests/test_api.py::test_database_with_space_in_name[/searchable_view.json]
FAILED tests/test_cli.py::test_weird_database_names[database (1).sqlite] - As...
FAILED tests/test_cli.py::test_weird_database_names[test-database (1).sqlite]
FAILED tests/test_html.py::test_row_html_compound_primary_key[/fixtures/compound_primary_key/a~2Fb,~2Ec~2Dd-expected1]
FAILED tests/test_html.py::test_css_classes_on_body[/fixtures/table~2Fwith~2Fslashes~2Ecsv-expected_classes5]
FAILED tests/test_html.py::test_templates_considered[/fixtures/table~2Fwith~2Fslashes~2Ecsv-table-fixtures-tablewithslashescsv-fa7563.html, *table.html]
FAILED tests/test_html.py::test_alternate_url_json[/fixtures/table~2Fwith~2Fslashes~2Ecsv-http://localhost/fixtures/table~2Fwith~2Fslashes~2Ecsv.json]
FAILED tests/test_html.py::test_edit_sql_link_on_canned_queries[/fixtures/~F0~9D~90~9C~F0~9D~90~A2~F0~9D~90~AD~F0~9D~90~A2~F0~9D~90~9E~F0~9D~90~AC-/fixtures?sql=select+id%2C+name+from+facet_cities+order+by+id+limit+1%3B]
FAILED tests/test_table_api.py::test_table_with_slashes_in_name - assert 302 ...
FAILED tests/test_table_api.py::test_custom_query_with_unicode_characters - j...
FAILED tests/test_table_api.py::test_searchable[/fixtures/searchable.json?_search=te*+AND+do*&_searchmode=raw-expected_rows3]
FAILED tests/test_table_api.py::test_searchmode[table_metadata1-_search=te*+AND+do*-expected_rows1]
FAILED tests/test_table_api.py::test_searchmode[table_metadata2-_search=te*+AND+do*&_searchmode=raw-expected_rows2]
=========== 22 failed, 1049 passed, 3 skipped in 1522.28s (0:25:22) ============
error: in phase 'check': uncaught exception:
%exception #<&invoke-error program: ""/gnu/store/ziqwkzz6znb5d3c245xn0cq5ra2ly0w3-python-pytest-7.1.3/bin/pytest"" arguments: (""-vv"" ""-n"" ""24"" ""-m"" ""not serial"") exit-status: 1 term-signal: #f stop-signal: #f> 
phase `check' failed after 1523.3 seconds
```
The tests run in a private namespace without internet connectivity, and the Python dependencies are at:
```
python-aiofiles@0.6.0 python-asgi-csrf@0.9 python-asgiref@3.4.1
+ python-beautifulsoup4@4.11.1 python-black@22.3.0 python-click-default-group@1.2.2 python-click@8.1.3
+ python-cogapp@3.3.0 python-httpx@0.23.0 python-hupper@1.10.3 python-itsdangerous@2.0.1
+ python-janus@1.0.0 python-jinja2@3.1.1 python-mergedeep@1.3.4 python-pint@0.20.1 python-pluggy@1.0.0
+ python-pytest-asyncio@0.17.2 python-pytest-runner@5.2 python-pytest-timeout@2.0.2
+ python-pytest-xdist@2.5.0 python-pytest@7.1.3 python-pyyaml@6.0 python-setuptools@64.0.3
+ python-trustme@0.9.0 python-uvicorn@0.17.6
```
With Python 3.9.9.

Thank you!",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2048/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
791381623,MDU6SXNzdWU3OTEzODE2MjM=,1197,DB size limit for publishing with Heroku,1186275,mtdukes,closed,0,,,,,1,2021-01-21T18:08:43Z,2021-01-24T20:53:44Z,2021-01-24T20:53:44Z,NONE,,"Hello,
I tried searching for this, but can't seem to get a great answer: Does anybody know the size limit for databases deploying to Heroku? The files I'm working with are pretty large, but I might be able to pare them down if I have a limit in mind.

 I'm getting the following error when running `datasette heroku publish`:

`RangeError [ERR_INVALID_OPT_VALUE]: The value ""14504095744"" is invalid for option ""size""`",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1197/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
600120439,MDU6SXNzdWU2MDAxMjA0Mzk=,726,Foreign key : case of a link to the associated row not displayed,6371750,JBPressac,closed,0,,,,,1,2020-04-15T08:31:27Z,2020-04-27T22:05:47Z,2020-04-27T22:05:46Z,CONTRIBUTOR,,"Hello,
I use Datasette to publish tsv files linked together by foreign keys declared thanks to sqlite-utils. In one table, [prelib_personne](http://crbc-dataset.huma-num.fr/prelib/prelib_personne), the foreign keys are properly noticed by a link to the associated row (for instance ville_naissance_id is properly linked to prelib_ville). But every link to the foreign key prelib_oeuvre.id fails. For instance, [prelib_ecritoeuvre](http://crbc-dataset.huma-num.fr/prelib/prelib_ecritoeuvre) has links to prelib_personne but none to prelib_oeuvre. In despite of the schema:

CREATE TABLE ""prelib_ecritoeuvre"" (
""id"" INTEGER,
  ""fonction_id"" INTEGER,
  ""oeuvre_id"" INTEGER,
  ""personne_id"" INTEGER
  ,PRIMARY KEY ([id]),
   FOREIGN KEY(fonction_id) REFERENCES prelib_fonctionecritoeuvre(id),
   FOREIGN KEY(personne_id) REFERENCES prelib_personne(id),
   FOREIGN KEY(oeuvre_id) REFERENCES prelib_oeuvre(id)
); 

Would you have any clue to investigate the reason of this problem?
Thanks,",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/726/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
546961357,MDU6SXNzdWU1NDY5NjEzNTc=,656,Display of the column definitions,6371750,JBPressac,closed,0,,,,,1,2020-01-08T16:16:53Z,2020-01-20T14:17:11Z,2020-01-20T14:14:33Z,CONTRIBUTOR,,"Hello,
Is the nice display of headers and definitions at the top of https://fivethirtyeight.datasettes.com/fivethirtyeight-ac35616/antiquities-act%2Factions_under_antiquities_act is configured in the metadata.json file ?
Thank you,",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/656/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1123393829,I_kwDODFE5qs5C9aEl,10,sqlite3.OperationalError: no such table: main.my_activity,69208826,glxblt14,open,0,,,,,1,2022-02-03T17:59:29Z,2022-03-20T02:38:07Z,,NONE,,"Hello,
When i run the command `google-takeout-to-sqlite my-activity db.db takeout-20220203T174446Z-001.zip`, i get this error :
```
Traceback (most recent call last):
  File ""c:\users\julie\appdata\local\programs\python\python39-32\lib\runpy.py"", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File ""c:\users\julie\appdata\local\programs\python\python39-32\lib\runpy.py"", line 87, in _run_code
    exec(code, run_globals)
  File ""C:\Users\julie\AppData\Local\Programs\Python\Python39-32\Scripts\google-takeout-to-sqlite.exe\__main__.py"", line 7, in <module>
  File ""c:\users\julie\appdata\local\programs\python\python39-32\lib\site-packages\click\core.py"", line 1128, in __call__
    return self.main(*args, **kwargs)
  File ""c:\users\julie\appdata\local\programs\python\python39-32\lib\site-packages\click\core.py"", line 1053, in main
    rv = self.invoke(ctx)
  File ""c:\users\julie\appdata\local\programs\python\python39-32\lib\site-packages\click\core.py"", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File ""c:\users\julie\appdata\local\programs\python\python39-32\lib\site-packages\click\core.py"", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File ""c:\users\julie\appdata\local\programs\python\python39-32\lib\site-packages\click\core.py"", line 754, in invoke
    return __callback(*args, **kwargs)
  File ""c:\users\julie\appdata\local\programs\python\python39-32\lib\site-packages\google_takeout_to_sqlite\cli.py"", line 31, in my_activity
    utils.save_my_activity(db, zf)
  File ""c:\users\julie\appdata\local\programs\python\python39-32\lib\site-packages\google_takeout_to_sqlite\utils.py"", line 19, in save_my_activity
    db[""my_activity""].create_index([""time""])
  File ""c:\users\julie\appdata\local\programs\python\python39-32\lib\site-packages\sqlite_utils\db.py"", line 629, in create_index
    self.db.conn.execute(sql)
sqlite3.OperationalError: no such table: main.my_activity
```
Thank you for your help
Sorry for my bad English
EDIT: i used the json format",206649770,google-takeout-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/10/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1157182254,I_kwDOBm6k_c5E-TMu,1646,Configuration directory mode does not pick up other file extensions than .db,15640196,dnsos,closed,0,,,,,3,2022-03-02T13:15:23Z,2022-10-07T23:06:17Z,2022-10-07T23:03:35Z,NONE,,"Hello, I've been trying to run Datasette with the [configuration directory mode](https://docs.datasette.io/en/stable/settings.html#configuration-directory-mode) with a structure such as this one:

```plain
some-directory/
    example.sqlite3
    another-example.db
    one-more.custom
    [...]
```

(In my scenario I can't just change the filename extension without other problems arising)

Now databases with the `.sqlite3` or the custom filename extension are ignored by Datasette in this case. I'm aware that the docs state that a `.db` extension is required, but I was wondering if there is a reason for restricting this or any workaround available? When I run `datasette example.sqlite3` or `datasette one-more.custom` the databases are served by Datasette without a problem. 

",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1646/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1727478903,I_kwDOBm6k_c5m9zx3,2081,Update Endpoints defined in metadata throws 403 Forbidden after a while,15085007,cutmasta-kun,open,0,,,,,0,2023-05-26T11:52:30Z,2023-05-26T11:52:30Z,,NONE,,"Hello. I expose an endpoint to update `tasks`:
```
{
    ""title"": ""My Datasette Instance"",
    ""databases"": {
        ""tasks"": {
            ""queries"": {
                ""update_task"": {
                    ""sql"": ""UPDATE tasks SET status = :status, result = :result, systemMessage = :systemMessage WHERE queueID = :queueID"",
                    ""write"": true,
                    ""on_success_message"": ""Task updated"",
                    ""on_success_redirect"": ""/tasks/tasks.json"",
                    ""on_error_message"": ""Task update failed"",
                    ""on_error_redirect"": ""/tasks.json"",
                    ""params"": [""queueID"", ""taskData"", ""status"", ""result"", ""systemMessage""]
                }
            }
        }
    }
}
```

This works really well! But after a while, the Datasette Instanz answers with **403 Forbidden**.
I have to delete the database and recreate it in order to work again.

Any help here? (´。＿。｀)",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2081/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
457147936,MDU6SXNzdWU0NTcxNDc5MzY=,512,"""about"" parameter in metadata does not appear when alone",7936571,chrismp,open,0,,,,,3,2019-06-17T21:04:20Z,2019-10-11T15:49:13Z,,NONE,,"Here's an example of metadata I have for one database on datasette.

```
""Records-requests"": {
	""tables"": {
		""Some table"": {
			""about"": ""This table has data.""
		}
	}
}
```

The text in `about` does not show up when I publish the data. But it shows up after I add a `""source""` parameter in the metadata.

Is this intended?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/512/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
275166669,MDU6SXNzdWUyNzUxNjY2Njk=,131,UI support for running FTS searches,9599,simonw,closed,0,,,,,3,2017-11-19T15:16:20Z,2017-11-19T17:18:05Z,2017-11-19T17:00:12Z,OWNER,,"Here's an example query that searches all FTS indexed columns in a table: https://sf-trees-search.now.sh/sf-trees-search-a899b92?sql=select+*+from+Street_Tree_List+where+rowid+in+%28select+rowid+from+Street_Tree_List_fts+where+Street_Tree_List_fts+match+%27grove+london+dpw%27%29%0D%0A

And here's a query that searches a specific column: https://sf-trees-search.now.sh/sf-trees-search-a899b92?sql=select+*+from+Street_Tree_List+where+rowid+in+%28select+rowid+from+Street_Tree_List_fts+where+qSpecies+match+%27london%27%29%0D%0A

If we detect that a table has FTS enabled (which we can do by looking for it as a content table reference in another FTS table's create definition) we should add a search box to the table page which constructs this query - maybe using `?_search=XXX` in the query string?

<s>To support search against specified columns, we can do `?_search__ qSpecies=London`.</s> - not necessary, see comment below.

- [x] Detect if a table has a FTS index defined against it as a content= parameter
- [x] Decide what to do if there is more than one FTS index (maybe just pick the first one?)
- [x] Add the `?_search=` query string argument
- [x] Add the UI",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/131/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
273191608,MDU6SXNzdWUyNzMxOTE2MDg=,65,Re-implement ?sql= mode,9599,simonw,closed,0,,,2857392,Ship first public release,1,2017-11-12T01:47:17Z,2017-11-12T02:36:37Z,2017-11-12T02:35:42Z,OWNER,,"Here's the code I removed:

    async def data(self, request, name, hash):
        sql = 'select * from sqlite_master'
        custom_sql = False
        params = {}
        if request.args.get('sql'):
            params = request.raw_args
            sql = params.pop('sql')
            validate_sql_select(sql)
            custom_sql = True
        rows = await self.execute(name, sql, params)
        columns = [r[0] for r in rows.description]
        return {
            'database': name,
            'rows': rows,
            'columns': columns,
            'query': {
                'sql': sql,
                'params': params,
            }
        }, {
            'database_hash': hash,
            'custom_sql': custom_sql,
        }
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/65/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
801780625,MDU6SXNzdWU4MDE3ODA2MjU=,9,SSL Error,12669260,jfeiwell,open,0,,,,,2,2021-02-05T02:12:56Z,2021-02-07T18:45:04Z,,NONE,,"Here's the error I get when running `pip install pocket-to-sqlite`: 

```
  Could not fetch URL https://pypi.python.org/simple/pocket-to-sqlite/: There was a problem confirming the ssl certificate: [SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:661) - skipping
  Could not find a version that satisfies the requirement pocket-to-sqlite (from versions: )
No matching distribution found for pocket-to-sqlite
```

Does this require python 3? ",213286752,pocket-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/9/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1094981339,I_kwDOCGYnMM5BRBbb,363,Better error message if `--convert` code fails to return a dict,9599,simonw,closed,0,,,,,4,2022-01-06T05:26:28Z,2022-02-03T22:52:30Z,2022-02-03T22:51:30Z,OWNER,,"Here's the traceback if your `--convert` function doesn't return a dict right now:
```
% sqlite-utils insert /tmp/all.db blah /tmp/log.log --convert 'all.upper()' --all         

Traceback (most recent call last):
  File ""/Users/simon/.local/share/virtualenvs/sqlite-utils-C4Ilevlm/bin/sqlite-utils"", line 33, in <module>
    sys.exit(load_entry_point('sqlite-utils', 'console_scripts', 'sqlite-utils')())
  File ""/Users/simon/.local/share/virtualenvs/sqlite-utils-C4Ilevlm/lib/python3.8/site-packages/click/core.py"", line 1137, in __call__
    return self.main(*args, **kwargs)
  File ""/Users/simon/.local/share/virtualenvs/sqlite-utils-C4Ilevlm/lib/python3.8/site-packages/click/core.py"", line 1062, in main
    rv = self.invoke(ctx)
  File ""/Users/simon/.local/share/virtualenvs/sqlite-utils-C4Ilevlm/lib/python3.8/site-packages/click/core.py"", line 1668, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File ""/Users/simon/.local/share/virtualenvs/sqlite-utils-C4Ilevlm/lib/python3.8/site-packages/click/core.py"", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File ""/Users/simon/.local/share/virtualenvs/sqlite-utils-C4Ilevlm/lib/python3.8/site-packages/click/core.py"", line 763, in invoke
    return __callback(*args, **kwargs)
  File ""/Users/simon/Dropbox/Development/sqlite-utils/sqlite_utils/cli.py"", line 949, in insert
    insert_upsert_implementation(
  File ""/Users/simon/Dropbox/Development/sqlite-utils/sqlite_utils/cli.py"", line 834, in insert_upsert_implementation
    db[table].insert_all(
  File ""/Users/simon/Dropbox/Development/sqlite-utils/sqlite_utils/db.py"", line 2602, in insert_all
    first_record = next(records)
  File ""/Users/simon/Dropbox/Development/sqlite-utils/sqlite_utils/db.py"", line 3044, in fix_square_braces
    for record in records:
  File ""/Users/simon/Dropbox/Development/sqlite-utils/sqlite_utils/cli.py"", line 831, in <genexpr>
    docs = (decode_base64_values(doc) for doc in docs)
  File ""/Users/simon/Dropbox/Development/sqlite-utils/sqlite_utils/utils.py"", line 86, in decode_base64_values
    to_fix = [
  File ""/Users/simon/Dropbox/Development/sqlite-utils/sqlite_utils/utils.py"", line 89, in <listcomp>
    if isinstance(doc[k], dict)
TypeError: string indices must be integers
```
It would be nicer if that returned a more useful error message.

_Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/361#issuecomment-1006295276_",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/363/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
695553522,MDU6SXNzdWU2OTU1NTM1MjI=,18,Deleted records stay in the search index,9599,simonw,open,0,,,,,2,2020-09-08T05:14:23Z,2020-09-08T05:15:51Z,,MEMBER,,"Here's why: https://github.com/dogsheep/dogsheep-beta/blob/24f7898d41a39218058f174c75ba62f7c0fcfff6/dogsheep_beta/utils.py#L44-L53

That should probably do `DELETE FROM index1.search_index WHERE [table] = ?` first.",197431109,dogsheep-beta,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-beta/issues/18/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
273846123,MDU6SXNzdWUyNzM4NDYxMjM=,90,datasette publish heroku,9599,simonw,closed,0,,,,,8,2017-11-14T16:01:39Z,2017-12-10T03:06:34Z,2017-12-10T03:05:48Z,OWNER,,"Heroku has Docker container support so this should not be too hard:

https://devcenter.heroku.com/articles/container-registry-and-runtime

See also #59 

This should work exactly like the existing “datasette publish now....” command except it would be “datasette publish heroku...”",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/90/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
276091279,MDU6SXNzdWUyNzYwOTEyNzk=,144,apsw as alternative sqlite3 binding (for full text search),649467,mhalle,closed,0,,,,,3,2017-11-22T14:40:39Z,2018-05-28T21:29:42Z,2018-05-28T21:29:42Z,NONE,,"Hey there,

Have you considered providing apsw support as an alternative to stock python sqlite3?  I use apsw because it keeps up with sqlite3 and is straightforward to bring in extensions like FTS5.  FTS really accelerates the kind of searching often done by web clients.

I may be able to help (it shouldn't be much code), but there are a couple of stylistic questions that come up when supporting an optional package.  Also, apsw is tricky in that it doesn't have a pypi package (author says limitations in providing options to setup.py). ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/144/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
640330278,MDU6SXNzdWU2NDAzMzAyNzg=,851,Having trouble getting writable canned queries to work,3243482,abdusco,closed,0,,,,,1,2020-06-17T10:30:28Z,2020-06-17T10:33:25Z,2020-06-17T10:32:33Z,CONTRIBUTOR,,"Hey,

I'm trying to get canned inserts to work. I have an DB with following metadata:

```text
sqlite> .mode line

sqlite> select name, sql from sqlite_master where name like '%search%';
 name = search
  sql = CREATE TABLE ""search"" (""id"" INTEGER NOT NULL PRIMARY KEY, ""name"" VARCHAR(255) NOT NULL, ""url"" VARCHAR(255) NOT NULL)
```

```yaml
# ...
queries:
    add_search:
        sql: insert into search(name, url) VALUES (:name, :url),
        write: true
```
which renders a form as expected, but when I submit the form I get `incomplete input` error.

![image](https://user-images.githubusercontent.com/3243482/84885285-7f46fe80-b09b-11ea-8a05-92c8986bbf7a.png)

but when submit post the form

I've attached a debugger to see where the error comes from, because `incomplete input` string doesn't appear in datasette codebase.

Inside `datasette.database.Database.execute_write_fn` 

https://github.com/simonw/datasette/blob/4fa7cf68536628344356d3ef8c92c25c249067a0/datasette/database.py#L69

```py
result = await reply_queue.async_q.get()
```

this line raises an exception. 

That led me to believe I had something wrong with my SQL. But running the command in `sqlite3` inserts the record just fine.

```text
sqlite> insert into search (name, url) values ('my name', 'my url');
sqlite> SELECT last_insert_rowid();
last_insert_rowid() = 3
```

So I'm a bit lost here.

---
- datasette, version 0.44
- Python 3.8.1",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/851/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
642572841,MDU6SXNzdWU2NDI1NzI4NDE=,859,Database page loads too slowly with many large tables (due to table counts),3243482,abdusco,open,0,,,,,21,2020-06-21T14:23:17Z,2021-08-25T21:59:55Z,,CONTRIBUTOR,,"Hey,
I have a database that I save in HTML from couple of web scrapers. There are around 200k+, 50+ rows in a couple of tables, with sqlite file weighing around 600MB.

The app runs on a VPS with 2 core CPU, 4GB RAM and refreshing database page regularly takes more than 10 seconds. I was suspecting that counting tables was the culprit, but manually running `select count(*) from table_name` for the largest table finishes under a second.

I've looked at the source code. There's a check for index page for mutable databases larger than 100MB
https://github.com/simonw/datasette/blob/799c5d53570d773203527f19530cf772dc2eeb24/datasette/views/index.py#L15

but this check is not performed for database page. 
I've manually crippled `Database::table_counts` method
```py
async def table_counts(self, limit=10):
    if not self.is_mutable and self.cached_table_counts is not None:
        return self.cached_table_counts
    # Try to get counts for each table, $limit timeout for each count
    counts = {}
    for table in await self.table_names():
        try:
            # table_count = (
            #     await self.execute(
            #         ""select count(*) from [{}]"".format(table),
            #         custom_time_limit=limit,
            #     )
            # ).rows[0][0]
            counts[table] = 10 # table_count
        # In some cases I saw ""SQL Logic Error"" here in addition to
        # QueryInterrupted - so we catch that too:
        except (QueryInterrupted, sqlite3.OperationalError, sqlite3.DatabaseError):
            counts[table] = None
    if not self.is_mutable:
        self.cached_table_counts = counts
    return counts
```

now the page loads in <100ms.

Is it possible to apply size check on database page too?

<details>
<summary>
/-/versions output
</summary>
<pre>
{
    ""python"": {
        ""version"": ""3.8.0"",
        ""full"": ""3.8.0 (default, Oct 28 2019, 16:14:01) \n[GCC 8.3.0]""
    },
    ""datasette"": {
        ""version"": ""0.44""
    },
    ""asgi"": ""3.0"",
    ""uvicorn"": ""0.11.5"",
    ""sqlite"": {
        ""version"": ""3.22.0"",
        ""fts_versions"": [
            ""FTS5"",
            ""FTS4"",
            ""FTS3""
        ],
        ""extensions"": {
            ""json1"": null
        },
        ""compile_options"": [
            ""COMPILER=gcc-7.4.0"",
            ""ENABLE_COLUMN_METADATA"",
            ""ENABLE_DBSTAT_VTAB"",
            ""ENABLE_FTS3"",
            ""ENABLE_FTS3_PARENTHESIS"",
            ""ENABLE_FTS3_TOKENIZER"",
            ""ENABLE_FTS4"",
            ""ENABLE_FTS5"",
            ""ENABLE_JSON1"",
            ""ENABLE_LOAD_EXTENSION"",
            ""ENABLE_PREUPDATE_HOOK"",
            ""ENABLE_RTREE"",
            ""ENABLE_SESSION"",
            ""ENABLE_STMTVTAB"",
            ""ENABLE_UNLOCK_NOTIFY"",
            ""ENABLE_UPDATE_DELETE_LIMIT"",
            ""HAVE_ISNAN"",
            ""LIKE_DOESNT_MATCH_BLOBS"",
            ""MAX_SCHEMA_RETRY=25"",
            ""MAX_VARIABLE_NUMBER=250000"",
            ""OMIT_LOOKASIDE"",
            ""SECURE_DELETE"",
            ""SOUNDEX"",
            ""TEMP_STORE=1"",
            ""THREADSAFE=1""
        ]
    }
}
</pre>
</details>",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/859/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
977128935,MDU6SXNzdWU5NzcxMjg5MzU=,21,Duplicate Column,32016596,FabianHertwig,open,0,,,,,1,2021-08-23T15:00:44Z,2021-08-23T17:00:59Z,,NONE,,"Hey, thank you for this repo!

When I try to convert my export, I get a multiple column error. Here is the stack trace:

```sh
(.venv) (base) computer:bodyweight_app user$ healthkit-to-sqlite ./data/Health_export.zip ./data/healthkit.db
Importing from HealthKit  [###############################-----]   87%  00:00:22
Traceback (most recent call last):
  File ""/MyProject/.venv/bin/healthkit-to-sqlite"", line 10, in <module>
    sys.exit(cli())
  File ""/MyProject/.venv/lib/python3.7/site-packages/click/core.py"", line 829, in __call__
    return self.main(*args, **kwargs)
  File ""/MyProject/.venv/lib/python3.7/site-packages/click/core.py"", line 782, in main
    rv = self.invoke(ctx)
  File ""/MyProject/.venv/lib/python3.7/site-packages/click/core.py"", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File ""/MyProject/.venv/lib/python3.7/site-packages/click/core.py"", line 610, in invoke
    return callback(*args, **kwargs)
  File ""/MyProject/.venv/lib/python3.7/site-packages/healthkit_to_sqlite/cli.py"", line 57, in cli
    convert_xml_to_sqlite(fp, db, progress_callback=bar.update, zipfile=zf)
  File ""/MyProject/.venv/lib/python3.7/site-packages/healthkit_to_sqlite/utils.py"", line 41, in convert_xml_to_sqlite
    write_records(records, db)
  File ""/MyProject/.venv/lib/python3.7/site-packages/healthkit_to_sqlite/utils.py"", line 146, in write_records
    batch_size=50,
  File ""/MyProject/.venv/lib/python3.7/site-packages/sqlite_utils/db.py"", line 2579, in insert_all
    extracts=extracts,
  File ""/MyProject/.venv/lib/python3.7/site-packages/sqlite_utils/db.py"", line 1246, in create
    extracts=extracts,
  File ""/MyProject/.venv/lib/python3.7/site-packages/sqlite_utils/db.py"", line 767, in create_table
    self.execute(sql)
  File ""/MyProject/.venv/lib/python3.7/site-packages/sqlite_utils/db.py"", line 421, in execute
    return self.conn.execute(sql)
sqlite3.OperationalError: duplicate column name: metadata_Meal
```",197882382,healthkit-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/21/reactions"", ""total_count"": 5, ""+1"": 5, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1572766460,I_kwDOCGYnMM5dvoL8,524,Transformation type `--type DATETIME`,21095447,4l1fe,closed,0,,,,,15,2023-02-06T15:18:42Z,2023-02-15T12:10:54Z,2023-02-15T12:10:54Z,NONE,,"Hey. Currently i do transformation with the type `--type TEXT`, but i noticed using the sqlalchemy based library [dataset](https://github.com/pudo/dataset) that is reading and writing differ depending on the column types `TEXT`, `DATETIME`.

Is it possible to alter a column type to `DATETIME` somehow using Sqlite-Utils?",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/524/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1063388037,I_kwDOCGYnMM4_YgOF,343,Provide function to generate hash_id from specified columns,82988,psychemedia,closed,0,,,,,4,2021-11-25T10:12:12Z,2022-03-02T04:25:25Z,2022-03-02T04:25:25Z,NONE,,"Hi

I note that you define `_hash()` to create a `hash_id` from non-id column values in a table [here](https://github.com/simonw/sqlite-utils/blob/8f386a0d300d1b1c76132bb75972b755049fb742/sqlite_utils/db.py#L2996).

It would be useful to be able to call a complementary function to generate a corresponding `_id` from a subset of specified columns when adding items to another table, eg to support the creation of foreign keys.

Or is there a better pattern for doing that?",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/343/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
925406964,MDU6SXNzdWU5MjU0MDY5NjQ=,1382,Datasette with Glitch - is it possible to use CSV with ISO-8859-1 encoding?,23701514,reichaves,closed,0,,,,,1,2021-06-19T14:37:20Z,2021-06-20T00:21:02Z,2021-06-20T00:20:06Z,NONE,,"Hi
Please, I used Remix on Glitch to create a project on Glitch and uploaded a CSV
But it's a CSV with ISO-8859-1 encoding (https://en.wikipedia.org/wiki/ISO/IEC_8859-1)
Is it possible for me to change the encoding to correctly visualize the data?
Example: https://emphasized-carpal-pillow.glitch.me/data/Emendas
Best",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1382/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1907281675,I_kwDOCGYnMM5xrs8L,595,Cascading DELETE not working with Table.delete(pk),123451970,cycle-data,closed,0,,,,,1,2023-09-21T15:46:41Z,2023-09-25T09:38:57Z,2023-09-25T09:38:13Z,NONE,,"Hi !
I noticed that when I am trying to use the delete method of the Table object,
the record get properly deleted from the table, but the cascading delete triggers on foreign keys do not activate.

`self.db[""contact""].delete(contact_id)`

I tried querying the database directly via DB Browser and the triggers work without any issue.
Looked up the source code and behind the scene this method is just querying the database normally so I'm not exactly sure where this behavior comes from.

Thank you in advance for your time ! ",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/595/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
500783373,MDU6SXNzdWU1MDA3ODMzNzM=,62,[enhancement] Method to delete a row in python,4454869,Sergeileduc,closed,0,,,,,5,2019-10-01T09:45:47Z,2019-11-04T16:30:34Z,2019-11-04T16:18:18Z,NONE,,"Hi !
Thanks for the lib !

Obviously, every possible sql queries won't have a dedicated method.

But I was thinking : a method to delete a row (I'm terrible with names, maybe `delete_where()` or something, would be useful.

I have a Database, with primary key.

For the moment, I use :

```Python3
db.conn.execute(f""DELETE FROM table WHERE key = {key_id}"")
db.conn.commit()
```
to delete a row I don't need anymore, giving his primary key.

Works like a charm.

Just an idea :

```Python3
table.delete_where_pkey({'key': key_id})
```
or something (I know, I'm terrible at naming methods...).

Pros : well, no need to write SQL query.

Cons : WHERE normally allows to do many more things (operators =, <>, >, <, BETWEEN), not to mention AND, OR, etc...
Method is maybe to specific, and/or a pain to render more flexible.

Again, just a thought. Writing his own sql works too, so...

Thanks again.
See yah.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/62/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1515883470,I_kwDOC8tyDs5aWovO,24,DOC: xml.etree.ElementTree.ParseError due to healthkit version 12 ,6231413,mmngreco,open,0,,,,,2,2023-01-01T23:00:38Z,2023-03-30T10:17:31Z,,NONE,,"Hi @simonw 

I hope you find this issue ok, the idea is provide some documentation to other users like me about how to solve this problem and save some time.

Following the instructions from the `README.md` I've faced this error:

```bash
(venv) mgreco@pop-os apple-health master* (23:44|0s)
$ healthkit-to-sqlite apple_health_export/export.xml healthkit.db --xml
Importing from HealthKit  [------------------------------------]    0%
Traceback (most recent call last):
  File ""/home/mgreco/github/mmngreco/apple-health/venv/bin/healthkit-to-sqlite"", line 33, in <module>
    sys.exit(load_entry_point('healthkit-to-sqlite', 'console_scripts', 'healthkit-to-sqlite')())
  File ""/home/mgreco/github/mmngreco/apple-health/venv/lib/python3.10/site-packages/click/core.py"", line 1130, in __call__
    return self.main(*args, **kwargs)
  File ""/home/mgreco/github/mmngreco/apple-health/venv/lib/python3.10/site-packages/click/core.py"", line 1055, in main
    rv = self.invoke(ctx)
  File ""/home/mgreco/github/mmngreco/apple-health/venv/lib/python3.10/site-packages/click/core.py"", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File ""/home/mgreco/github/mmngreco/apple-health/venv/lib/python3.10/site-packages/click/core.py"", line 760, in invoke
    return __callback(*args, **kwargs)
  File ""/home/mgreco/github/mmngreco/apple-health/.deps/healthkit-to-sqlite/healthkit_to_sqlite/cli.py"", line 57, in cli
    convert_xml_to_sqlite(fp, db, progress_callback=bar.update, zipfile=zf)
  File ""/home/mgreco/github/mmngreco/apple-health/.deps/healthkit-to-sqlite/healthkit_to_sqlite/utils.py"", line 25, in convert_xml_to_sqlite
    for tag, el in find_all_tags(
  File ""/home/mgreco/github/mmngreco/apple-health/.deps/healthkit-to-sqlite/healthkit_to_sqlite/utils.py"", line 12, in find_all_tags
    for event, el in parser.read_events():
  File ""/home/mgreco/github/mmngreco/apple-health/venv/lib/python3.10/xml/etree/ElementTree.py"", line 1324, in read_events
    raise event
  File ""/home/mgreco/github/mmngreco/apple-health/venv/lib/python3.10/xml/etree/ElementTree.py"", line 1296, in feed
    self._parser.feed(data)
xml.etree.ElementTree.ParseError: syntax error: line 156, column 0
```

So, after debugging and searching on internet I found this useful link: https://discussions.apple.com/thread/254202523 (etresoft, the real hero). Which basically says that the xml given by the health app (healthkit version 12) has some bugs but  fortunately, they can be solved with a couple of commads:

1. Uncompress the zip and move the new folder where `export.xml` is.
1. Create a `patch.txt` with the following content

    ```diff
    --- export.xml	2022-09-18 15:17:09.000000000 -0400
    +++ export-fixed.xml	2022-09-18 16:37:08.000000000 -0400
    @@ -15,6 +15,7 @@
       HKCharacteristicTypeIdentifierBiologicalSex       CDATA #REQUIRED
       HKCharacteristicTypeIdentifierBloodType           CDATA #REQUIRED
       HKCharacteristicTypeIdentifierFitzpatrickSkinType CDATA #REQUIRED
    +  HKCharacteristicTypeIdentifierCardioFitnessMedicationsUse CDATA #IMPLIED
     >
     <!ELEMENT Record ((MetadataEntry|HeartRateVariabilityMetadataList)*)>
     <!ATTLIST Record
    @@ -39,7 +40,7 @@
       startDate     CDATA #REQUIRED
       endDate       CDATA #REQUIRED
     >
    -<!ELEMENT Workout ((MetadataEntry|WorkoutEvent|WorkoutRoute)*)>
    +<!ELEMENT Workout ((MetadataEntry|WorkoutEvent|WorkoutRoute|WorkoutStatistics)*)>
     <!ATTLIST Workout
       workoutActivityType   CDATA #REQUIRED
       duration              CDATA #IMPLIED
    @@ -63,7 +64,7 @@
       duration             CDATA #IMPLIED
       durationUnit         CDATA #IMPLIED
     >
    -<!ELEMENT WorkoutEvent EMPTY>
    +<!ELEMENT WorkoutEvent (MetadataEntry?)>
     <!ATTLIST WorkoutEvent
       type                 CDATA #REQUIRED
       date                 CDATA #REQUIRED
    @@ -79,6 +80,7 @@
       minimum              CDATA #IMPLIED
       maximum              CDATA #IMPLIED
       sum                  CDATA #IMPLIED
    +  unit                 CDATA #IMPLIED
     >
     <!ELEMENT WorkoutRoute ((MetadataEntry|FileReference)*)>
     <!ATTLIST WorkoutRoute
    @@ -153,6 +155,7 @@
       dateIssued       CDATA #REQUIRED
       expirationDate   CDATA #REQUIRED
       brand            CDATA #IMPLIED
    +>
     <!ELEMENT RightEye EMPTY>
     <!ATTLIST RightEye
       sphere           CDATA #IMPLIED
    @@ -203,13 +206,6 @@
       diameter         CDATA #IMPLIED
       diameterUnit     CDATA #IMPLIED
     >
    -  device           CDATA #IMPLIED
    -<!ELEMENT MetadataEntry EMPTY>
    -<!ATTLIST MetadataEntry
    -  key              CDATA #IMPLIED
    -  value            CDATA #IMPLIED
    ->
    ->
     ]>
     <HealthData>
      <ExportDate/>
    ```
1. Apply the path with the command: `patch < patch.txt`
1. Fix endDates with the command `sed 's/startDate/endDate/2' export.xml > export-fixed.xml`
1. Try again `healthkit-to-sqlite export-fixed.xml healthkit.db --xml`",197882382,healthkit-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/24/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
723708310,MDU6SXNzdWU3MjM3MDgzMTA=,188,About loading spatialite,30607,aborruso,closed,0,,,,,1,2020-10-17T08:47:02Z,2022-02-05T00:04:26Z,2020-10-17T08:52:58Z,NONE,,"Hi @simonw ,
If I run

```
sqlite3
.load /usr/local/lib/mod_spatialite.so
select spatialite_version();
```

I have `5.0.0`.

![image](https://user-images.githubusercontent.com/30607/96332706-d8cd3300-1065-11eb-906b-daf99963198e.png)


If I run

```
sqlite-utils :memory: ""select spatialite_version()"" --load-extension=spatialite
```

I have

```
Traceback (most recent call last):
  File ""/home/aborruso/.local/bin/sqlite-utils"", line 8, in <module>
    sys.exit(cli())
  File ""/home/aborruso/.local/lib/python3.8/site-packages/click/core.py"", line 829, in __call__
    return self.main(*args, **kwargs)
  File ""/home/aborruso/.local/lib/python3.8/site-packages/click/core.py"", line 782, in main
    rv = self.invoke(ctx)
  File ""/home/aborruso/.local/lib/python3.8/site-packages/click/core.py"", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File ""/home/aborruso/.local/lib/python3.8/site-packages/click/core.py"", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File ""/home/aborruso/.local/lib/python3.8/site-packages/click/core.py"", line 610, in invoke
    return callback(*args, **kwargs)
  File ""/home/aborruso/.local/lib/python3.8/site-packages/sqlite_utils/cli.py"", line 936, in query
    _load_extensions(db, load_extension)
  File ""/home/aborruso/.local/lib/python3.8/site-packages/sqlite_utils/cli.py"", line 1326, in _load_extensions
    db.conn.load_extension(ext)
TypeError: argument 1 must be str, not None
```

How to load properly spatialite extension in sqlite-utils?

Thank you very muc",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/188/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1410548368,I_kwDODFdgUs5UE0KQ,77,Feature: Support GitHub discussions,631242,frosencrantz,open,0,,,,,0,2022-10-16T16:53:38Z,2022-10-16T16:53:38Z,,CONTRIBUTOR,,"Hi @simonw I've been a happy user of this tool.  Thank you for writing it and sharing it.

I wanted to suggest a feature request to support  Discussions.  For example the VisiData project has discussions https://github.com/saulpw/visidata/discussions , and it would be useful if there was a way to pull that data into the database.

However, I'm not offering a pull request.",207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/77/reactions"", ""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1177059481,I_kwDODFdgUs5GKICZ,71,Store commit parents,64686,carltongibson,closed,0,,,,,0,2022-03-22T17:06:48Z,2022-04-22T12:44:04Z,2022-04-22T12:44:04Z,NONE,,"Hi @simonw 👋 

Currently, stored commit data doesn't quite give me the information I'm needing... 

Committer date and author date are not 100% reliable for dividing a commit history up by release or branch. 
A PR created before a release but merged after can have earlier dates… — this can be quite frustrating if you're trying to pin down commits for a release: _It should be there!_, but then isn't. (This gets worse using release branches.) 

Would you be open to adding the `sha` of a `parent` of a commit to the commit table? (As an FK? 🤔 — likely not feasible.)

It's part of the [response body](https://docs.github.com/en/rest/reference/commits#get-a-commit): 

```
  ""parents"": [
    {
      ""url"": ""https://api.github.com/repos/octocat/Hello-World/commits/6dcb09b5b57875f334f61aebed695e2e4193db5e"",
      ""sha"": ""6dcb09b5b57875f334f61aebed695e2e4193db5e""
    }
  ],
```

I think this list should only have a single entry. (🤔 — not sure why it's a list then...)

With this it would be possible to build/reconstruct a chain of commits from the history, that I don't **think** is available as yet (unless you know a better way). 

It is certainly possible to get sequential lists of commits out of git directly, so the same would be possible combining tools, but wondering if a single tool could do it. 

What do you think? 
Thanks! 🏅 

",207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/71/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
539590148,MDU6SXNzdWU1Mzk1OTAxNDg=,651,fts5 syntax error when using punctuation,2181410,clausjuhl,closed,0,,,,,3,2019-12-18T10:25:35Z,2021-07-14T19:26:06Z,2019-12-30T06:42:55Z,NONE,,"Hi Simon

I get a syntax error when using punctuation or special characters in a fulltext search (using fts5). I created the virtual table using sqlite-utils' ""enable-fts""-command.

The same error appears on Niche Museums [https://www.niche-museums.com/browse/search?q=park.](https://www.niche-museums.com/browse/search?q=park.), but works fine in most of your other datasette-examples, e.g. register-of-members-interests [https://register-of-members-interests.datasettes.com/regmem-98dc8b7/items?_search=mins.](https://register-of-members-interests.datasettes.com/regmem-98dc8b7/items?_search=mins.)

What am I doing wrong? Many thanks!

",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/651/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
612382643,MDU6SXNzdWU2MTIzODI2NDM=,758,Question: Access to immutable database-path,2181410,clausjuhl,open,0,,,,,6,2020-05-05T07:01:18Z,2020-05-28T08:23:27Z,,NONE,,"Hi Simon

Is there anywhere in the app-context where one can access the hashed urlpath of the database? Currently it's included in the template-context (`databases[0][""path"")` when rendering urls of the database (eg. `/db-44b06v9/cases`...), but where can I find the hashed url when rendering the index-page? I'm trying to avoid redirects. Thanks!",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/758/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
448189298,MDU6SXNzdWU0NDgxODkyOTg=,486,Ability to add extra routes and related templates,2181410,clausjuhl,closed,0,,,,,2,2019-05-24T14:04:25Z,2019-05-24T14:43:28Z,2019-05-24T14:43:09Z,NONE,,"Hi Simon

Thank for an excellent job! Datasette is such an obviously good idea (once you have that idea!) and so well done. The only thing that I miss, is the ability to add extras routes (with associated jinja2-templates). For most of the datasets, that I would like to publish, I would also like at least a page, that describes the data (semantics, provenance, biases...) and a page explaining our cookie- and privacy-policies (which would allows us to use something like Goggle Analytics).
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/486/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
556814876,MDU6SXNzdWU1NTY4MTQ4NzY=,662,Escape_fts5_query-hookimplementation does not work with queries to standard tables,2181410,clausjuhl,closed,0,,,,,5,2020-01-29T11:56:03Z,2020-01-30T00:30:20Z,2020-01-30T00:30:19Z,NONE,,"Hi Simon

Thank you for adding the escape_function, but it does not work on my datasette-installation (0.33). I've added the following file to my datasette-dir: /plugins/sql_functions.py:

`from datasette import hookimpl

def escape_fts_query(query):
    bits = query.split()
    return ' '.join('""{}""'.format(bit.replace('""', '')) for bit in bits)

@hookimpl
def prepare_connection(conn):
    conn.create_function(""escape_fts_query"", 1, escape_fts_query)`

It has no effect on the standard queries to the tables though, as they still produce errors when including any characters like '-', '/', '+' or '?'

Does the function only work when using costum queries, where I can include the escape_fts-function explicitly in the sql-query?

PS. I'm calling datasette with --plugins=plugins, and my other plugins work just fine.
PPS. The fts5 virtual table is created with 'sqlite3' like so:

`CREATE VIRTUAL TABLE ""cases_fts"" USING FTS5(
  title,
  subtitle,
  resume,
  suggestion,
  presentation,
  detail = full,
  content_rowid = 'id',
  content = 'cases',
  tokenize='unicode61', 'remove_diacritics 2', 'tokenchars ""-_""'
);`

Thanks!

_Originally posted by @clausjuhl in https://github.com/simonw/datasette/issues/651#issuecomment-579675357_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/662/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1594383280,I_kwDOBm6k_c5fCFuw,2030,How to use Datasette with apache webserver on GCP?,19700859,gk7279,closed,0,,,,,2,2023-02-22T03:08:49Z,2023-02-22T21:54:39Z,2023-02-22T21:54:39Z,NONE,,"Hi Simon and Datasette team-

I have installed apache2 webserver inside GCP VM using apt.

I can see my ""Hello World"" index.html if I use the external IP of this GCP in a browser.

However, when I try to run datasette with different combinations of -h and -p, I am still unable to access the webpage.

I cannot invest Docker on this VM.

Any pointers to use datasette with already existing apache2 webserver on GCP is appreciated.

Thanks.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2030/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
760312579,MDU6SXNzdWU3NjAzMTI1Nzk=,1134,"""_searchmode=raw"" throws an index out of range error when combined with ""_search_COLUMN""",2181410,clausjuhl,closed,0,,,,,4,2020-12-09T13:05:37Z,2020-12-10T05:57:17Z,2020-12-09T19:56:55Z,NONE,,"Hi Simon!
Maybe it's just me, but when [using _searchmode=raw (trying to enable wildcard-searching) in combination with the ""_search_COLUMN""-table argument](https://byraadsarkivet.aarhus.dk/db/cases?_searchmode=raw&_search_title=sundhedsfrem*), I get a list index out of range error. [When combining with the simpler ""_search""-argument everything works, including wildcard-seaches.](https://byraadsarkivet.aarhus.dk/db/cases?_search=sundhedsfrem*&_searchmode=raw). Here's the traceback:

```
Traceback (most recent call last):
  File ""/Users/cjk/.local/share/virtualenvs/minutes-jMDZ8Ssk/lib/python3.7/site-packages/datasette/utils/asgi.py"", line 122, in route_path
    return await view(new_scope, receive, send)
  File ""/Users/cjk/.local/share/virtualenvs/minutes-jMDZ8Ssk/lib/python3.7/site-packages/datasette/utils/asgi.py"", line 196, in view
    request, **scope[""url_route""][""kwargs""]
  File ""/Users/cjk/.local/share/virtualenvs/minutes-jMDZ8Ssk/lib/python3.7/site-packages/datasette/views/base.py"", line 204, in get
    request, database, hash, correct_hash_provided, **kwargs
  File ""/Users/cjk/.local/share/virtualenvs/minutes-jMDZ8Ssk/lib/python3.7/site-packages/datasette/views/base.py"", line 342, in view_get
    request, database, hash, **kwargs
  File ""/Users/cjk/.local/share/virtualenvs/minutes-jMDZ8Ssk/lib/python3.7/site-packages/datasette/views/table.py"", line 393, in data
    search_col = key.split(""_search_"", 1)[1]
IndexError: list index out of range

```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1134/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
984939366,MDU6SXNzdWU5ODQ5MzkzNjY=,58,"Error: Use either --since or --since_id, not both - still broken",42904,rubenv,closed,0,,,,,1,2021-09-01T09:45:28Z,2021-09-21T17:37:41Z,2021-09-21T17:37:41Z,CONTRIBUTOR,,"Hi Simon,

It appears the fix for #57 doesn't fix things for me:

```
$ twitter-to-sqlite --version
twitter-to-sqlite, version 0.21.4
$ python --version
Python 3.9.6
```

```
$ twitter-to-sqlite home-timeline -a twitter-auth.json twitter/timeline.db --since
Importing tweets
Error: Use either --since or --since_id, not both
```

Is there any way I can help debug this?",206156866,twitter-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/58/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
767685961,MDU6SXNzdWU3Njc2ODU5NjE=,210,Support of RData files,23739126,PeterBailey,closed,0,,,,,1,2020-12-15T15:04:14Z,2021-01-02T00:02:40Z,2021-01-02T00:02:40Z,NONE,,"Hi Simon,

Would be great if you could ingest RData files!

I could do this in a few lines of code but I am too lazy - sorry!

Peter",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/210/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1592327343,I_kwDOBm6k_c5e6Pyv,2029,"Sorry Simon, didn't know how else to contact you",5804626,llchristopherson,open,0,,,,,0,2023-02-20T19:02:53Z,2023-02-20T19:02:53Z,,NONE,,"Hi Simon,

Would you be willing to chat with me about Datasette? I have some questions. I am working on a project to evaluate data ingestion tools for a research organization and I ran across Datasette. I have looked through a lot of your documentation, but still have some questions, which are very specific. If you would be willing to write me back about this, my email is laura@renci.org.

Thanks,
Laura",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2029/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
459936585,MDU6SXNzdWU0NTk5MzY1ODU=,527,Unable to use rank when fts-table generated with csvs-to-sqlite,2181410,clausjuhl,closed,0,,,,,3,2019-06-24T14:49:48Z,2019-06-24T15:21:18Z,2019-06-24T15:09:10Z,NONE,,"Hi Simon.

If i generate a fts-table with the csvs-to-sqlite f-option, I'm unable to use (in datasette's GUI) the internal ranking of the table for sorting or viewing, but if I generate the fts-table with the enable-fts argument from sqlite-utils, everyrthing works ok. Eg.:

datasette, version 0.28
sqlite-utils, version 1.2.1
csvs-to-sqlite, version 0.9

No column named rank with these commands:
$ csvs-to-sqlite minutes.csv minutes.db -f text_data
$ datasette -i minutes.db
select rank, * from minutes_fts where minutes_fts match 'dog'

Everything ok with these commands:
$ csvs-to-sqlite minutes.csv minutes.db
$ sqlite-utils enable-fts minutes.db text_data
$ datasette -i minutes.db
select rank, * from minutes_fts where minutes_fts match 'dog'

Am I doing something wrong?

Thank you for a great application!",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/527/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
617323873,MDU6SXNzdWU2MTczMjM4NzM=,766,Enable wildcard-searches by default,2181410,clausjuhl,open,0,,,,,2,2020-05-13T10:14:48Z,2021-03-05T16:35:21Z,,NONE,,"Hi Simon.

It seems that datasette currently has wildcard-searches disabled by default (along with the boolean search-options, NEAR-queries and more, and despite the docs). If I try out the search-url provided in the [docs](https://datasette.readthedocs.io/en/stable/full_text_search.html#the-table-page-and-table-view-api) (https://fara.datasettes.com/fara/FARA_All_ShortForms?_search=manafort), it does not handle wildcard-searches, and I'm unable to make it work on my datasette-instance.

I would argue that wildcard-searches is such a standard query, that it should be enabled by default. Requiring ""_searchmode=raw"" when using prefix-searches seems unnecessary. Plus: What happens to non-ascii searches when using ""_searchmode=raw""? Is the ""escape_fts""-function from datasette.utils ignored?


Thanks!

/Claus",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/766/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
611835285,MDU6SXNzdWU2MTE4MzUyODU=,752,Non-utf8 encoding in exceptionhandlers and custom-pages,2181410,clausjuhl,closed,0,,,,,1,2020-05-04T12:24:42Z,2020-05-04T17:42:20Z,2020-05-04T17:42:20Z,NONE,,"Hi Simon.

Whenever a response is not piped through a router-view, the template is encoded in latin-1 (I think). This is especially a problem (for me) with the new custom_pages-functionality, but also problematic with the 404- and 500-handlers. Thanks!",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/752/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
569317377,MDU6SXNzdWU1NjkzMTczNzc=,681,Cashe-header missing in http-response,2181410,clausjuhl,closed,0,,,,,4,2020-02-22T10:50:45Z,2020-02-24T20:53:57Z,2020-02-24T20:53:56Z,NONE,,"Hi Simon. I need some help with both understanding and adding http-headers. If I call datasette on localhost with --config default_cache_ttl:120 and --cors, I only get the following response-headers:

access-control-allow-origin: *
content-type: text/html; charset=utf-8
date: Sat, 22 Feb 2020 10:32:15 GMT
referrer-policy: no-referrer
server: uvicorn
transfer-encoding: chunked

Cors works, but no caching-header is set? Same thing happens if I use the command in a Dockerfile and run datasette with docker.

Second, how can one add headers to uvicorn? I've tried to add uvicorn commands to the Dockerfile, before the final datasette command, but it doesn't work. Is there any way to add headers to the uvicorn.run() command i datasette? I particular, I would like to add some of the missing security-headers:

<img width=""1010"" alt=""Screenshot 2020-02-22 at 11 48 03"" src=""https://user-images.githubusercontent.com/2181410/75091037-5ab59c80-5569-11ea-8dbb-22357f1aa4c8.png"">

Thank you for a great product!",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/681/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
451585764,MDU6SXNzdWU0NTE1ODU3NjQ=,499,Accessibility for non-techie newsies? ,7936571,chrismp,open,0,,,,,3,2019-06-03T16:49:37Z,2019-06-05T21:22:55Z,,NONE,,"Hi again, I'm having fun uploading datasets to Heroku via datasette. I'd like to set up datasette so that it's easy for other newsroom workers, who don't use Linux and aren't programmers, to upload datasets. Does datsette provide this out-of-the-box, or as a plugin? ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/499/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
612673948,MDU6SXNzdWU2MTI2NzM5NDg=,759,fts search on a column doesn't work anymore due to escape_fts,133845,Krazybug,closed,0,,,,,3,2020-05-05T15:03:44Z,2021-07-16T02:11:54Z,2020-05-06T17:50:57Z,NONE,,"Hi and first, thank you for this awesome work you make with this projet.

On a db indexed in full text search, I can't query on indexed column anymore.

This request ""cauvin language:ita"": is running smoothly on a old version of datasette but not on the current version.

Compare the current version query
`select uuid, title, authors, year, series, language, formats, publisher, tags, identifiers from summary where rowid in (select rowid from summary_fts where summary_fts match escape_fts(:search)) order by uuid limit 101`


To an older version:

`select title, authors, series, uuid, language, identifiers, tags, publisher, formats, year, links from summary where rowid in (select rowid from summary_fts where summary_fts match :search) order by uuid limit 101`

_language_ is a searchable column but now the search string is known as ""cauvin language:ita"" literally as a search term. columns are not parsed.


",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/759/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
282971961,MDU6SXNzdWUyODI5NzE5NjE=,175,"Add project topic ""automatic-api""",3179832,dbohdan,closed,0,,,,,1,2017-12-18T18:09:17Z,2017-12-21T18:33:55Z,2017-12-21T18:33:55Z,NONE,,"Hi there! Could you add the ~~tag~~ topic `automatic-api` to your repository? I am [making a list](https://github.com/dbohdan/automatic-api) of all projects that automatically expose APIs to databases. (Your Show&nbsp;HN made me do it.&nbsp;:-) I knew about PostgREST and PostGraphQL, but it took adding Datasette to sell me on the concept.) They will be easier to discover if there is a standard GitHub tag, and `automatic-api` seems as good a candidate as any. Two projects [already use it](https://github.com/topics/automatic-api).",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/175/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
664485022,MDU6SXNzdWU2NjQ0ODUwMjI=,46,Feature: pull request reviews and comments,1326704,bhrutledge,open,0,,,,,6,2020-07-23T13:43:45Z,2022-12-20T14:40:15Z,,NONE,,"Hi there! I saw your [presentation at Boston Python](https://www.meetup.com/bostonpython/events/271887195). I'm already a light user of Datasette (thank you!), but wasn't aware of this project.

I've been working on a ""pull request dashboard"" to get a comprehensive view of the state of open PR's, esp. related to reviews (i.e., pending, approved, changes requested). Currently it's a CLI command, but I thought a Datasette UI might be fun.

I see that PR's are available from the `issues` command, but I don't see reviews anywhere. From the [API docs](https://docs.github.com/en/rest/reference/pulls#reviews), it looks like there are separate endpoints for those (as well as pull requests in general). What do you think about adding that? Would you accept a PR? Any sense of the level of effort?",207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/46/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
919508498,MDU6SXNzdWU5MTk1MDg0OTg=,1375,JSON export dumps JSON fields as TEXT,4068,frafra,closed,0,,,,,2,2021-06-12T09:45:08Z,2021-06-14T09:41:59Z,2021-06-13T15:37:58Z,NONE,,"Hi!
When a user tries to export data as JSON, I would expect to see the value of JSON columns represented as JSON instead of being rendered as a string. What do you think?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1375/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
539204432,MDU6SXNzdWU1MzkyMDQ0MzI=,70,Implement ON DELETE and ON UPDATE actions for foreign keys,26292069,LucasElArruda,open,0,,,,,2,2019-12-17T17:19:10Z,2020-02-27T04:18:53Z,,NONE,,"Hi! I did not find any mention on the library about ON DELETE and ON UPDATE actions for foreign keys. Are those expected to be implemented? If not, it would be a nice thing to include!",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/70/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1383646615,I_kwDOCGYnMM5SeMWX,491,Ability to merge databases and tables,8904453,sgraaf,open,0,,,,,7,2022-09-23T11:10:55Z,2023-06-14T22:14:24Z,,NONE,,"Hi! Let me firstly say that I am a big fan of your work -- I follow your tweets and blog posts with great interest 😄.

Now onto the matter at hand: I think it would be great if `sqlite-utils` included a `merge` or `combine` command, with the purpose of combining different SQLite databases into a single SQLite database. This way, the newly ""merged"" database would contain all differently named tables contained in the databases to be merged as-is, as well a concatenation of all tables of the same name.

This could look something like this:

```bash
sqlite-utils merge cats.db dogs.db > animals.db
```

I imagine this is rather straightforward if all databases involved in the merge contain differently named tables (i.e. no chance of conflicts), but things get slightly more complicated if two or more of the databases to be merged contain tables with the same name. Not only do you have to ""do something"" with the primary key(s), but these tables could also simply have different schemas (and therefore be incompatible for concatenation to begin with).

Anyhow, I would love your thoughts on this, and, if you are open to it, work together on the design and implementation!",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/491/reactions"", ""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
919250621,MDU6SXNzdWU5MTkyNTA2MjE=,269,bool type not supported,4068,frafra,closed,0,,,,,3,2021-06-11T22:00:36Z,2021-06-15T01:34:10Z,2021-06-15T01:34:10Z,NONE,,"Hi! Thank you for sharing this very nice tool :)
It would be nice to have support for more types, like `bool`: it is not possible to convert to boolean at the moment. My suggestion would be to handle it as `bool(int(value))`, like csvkit does.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/269/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1690765434,I_kwDOBm6k_c5kxwh6,2067,Litestream-restored db: errors on 3.11 and 3.10.8; but works on py3.10.7 and 3.10.6,39538958,justmars,open,0,,,,,1,2023-05-01T12:42:28Z,2023-05-03T00:16:03Z,,NONE,,"Hi! Wondering if this issue is limited to my local system or if it affects others as well. 

It seems like 3.11 errors out on a ""litestream-restored"" database. On further investigation, it also appears to conk out on 3.10.8 but works on 3.10.7 and 3.10.6.

To demo issue I created a test database, replicated it to an aws s3 bucket, then restored the same under various .pyenv-versioned shells where I test whether I can read the database via the sqlite3 cli.

```sh
# create new shell with 3.11.3
litestream restore -o data/db.sqlite s3://mytestbucketxx/db
sqlite3 data/db.sqlite  
# SQLite version 3.41.2 2023-03-22 11:56:21
# Enter "".help"" for usage hints.
# sqlite> .tables
# _litestream_lock  _litestream_seq   movie           
# sqlite> 
```

However this get me an `OperationalError` when reading via datasette:

<details>
<summary>Error on 3.11.3 and 3.10.8</summary>

```sh
datasette data/db.sqlite
```

```console
/tester/.venv/lib/python3.11/site-packages/pkg_resources/__init__.py:121: DeprecationWarning: pkg_resources is deprecated as an API
  warnings.warn(""pkg_resources is deprecated as an API"", DeprecationWarning)
Traceback (most recent call last):
  File ""/tester/.venv/bin/datasette"", line 8, in <module>
    sys.exit(cli())
             ^^^^^
  File ""/tester/.venv/lib/python3.11/site-packages/click/core.py"", line 1130, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ""/tester/.venv/lib/python3.11/site-packages/click/core.py"", line 1055, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File ""/tester/.venv/lib/python3.11/site-packages/click/core.py"", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ""/tester/.venv/lib/python3.11/site-packages/click/core.py"", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ""/tester/.venv/lib/python3.11/site-packages/click/core.py"", line 760, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ""/tester/.venv/lib/python3.11/site-packages/datasette/cli.py"", line 143, in wrapped
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File ""/tester/.venv/lib/python3.11/site-packages/datasette/cli.py"", line 615, in serve
    asyncio.get_event_loop().run_until_complete(check_databases(ds))
  File ""/Users/mv/.pyenv/versions/3.11.3/lib/python3.11/asyncio/base_events.py"", line 653, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File ""/tester/.venv/lib/python3.11/site-packages/datasette/cli.py"", line 660, in check_databases
    await database.execute_fn(check_connection)
  File ""/tester/.venv/lib/python3.11/site-packages/datasette/database.py"", line 213, in execute_fn
    return await asyncio.get_event_loop().run_in_executor(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ""/Users/mv/.pyenv/versions/3.11.3/lib/python3.11/concurrent/futures/thread.py"", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ""/tester/.venv/lib/python3.11/site-packages/datasette/database.py"", line 211, in in_thread
    return fn(conn)
           ^^^^^^^^
  File ""/tester/.venv/lib/python3.11/site-packages/datasette/utils/__init__.py"", line 951, in check_connection
    for r in conn.execute(
             ^^^^^^^^^^^^^
sqlite3.OperationalError: unable to open database file
```

</details>


<details>
<summary>Works on 3.10.7, 3.10.6</summary>

```sh
# create new shell with 3.10.7 / 3.10.6
litestream restore -o data/db.sqlite s3://mytestbucketxx/db
datasette data/db.sqlite
# ...
# INFO:     Uvicorn running on http://127.0.0.1:8001 (Press CTRL+C to quit)
```

</details>

In both scenarios, the only dependencies were the pinned python version and the latest Datasette version 0.64.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2067/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1733198948,I_kwDOCGYnMM5nToRk,555,Filter table by a large bunch of ids,10843208,redraw,open,0,,,,,1,2023-05-31T00:29:51Z,2023-06-14T22:01:57Z,,NONE,,"Hi! this might be a question related to both SQLite & sqlite-utils, and you might be more experienced with them.

I have a large bunch of ids, and I'm wondering which is the best way to query them in terms of performance, and simplicity if possible.

The naive approach would be something like `select * from table where rowid in (?, ?, ?...)` but that wouldn't scale if ids are >1k.

Another approach might be creating a temp table, or in-memory db table, insert all ids in that table and then join with the target one.

I failed to attach an in-memory db both using sqlite-utils, and plain sql's execute(), so my closest approach is something like,

```python
def filter_existing_video_ids(video_ids):
    db = get_db()  # contains a ""videos"" table
    db.execute(""CREATE TEMPORARY TABLE IF NOT EXISTS tmp (video_id TEXT NOT NULL PRIMARY KEY)"")
    db[""tmp""].insert_all([{""video_id"": video_id} for video_id in video_ids])
    for row in db[""tmp""].rows_where(""video_id not in (select video_id from videos)""):
        yield row[""video_id""]
    db[""tmp""].drop()
```

That kinda worked, I couldn't find an option in sqlite-utils's `create_table()` to tell it's a temporary table. Also, `tmp` table is not dropped finally, neither using `.drop()` despite being created with the keyword `TEMPORARY`. I believe it should be automatically dropped after connection/session ends though I read.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/555/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
983221851,MDU6SXNzdWU5ODMyMjE4NTE=,34,Data folder as index command parameter,1223625,humrochagf,open,0,,,,,0,2021-08-30T21:29:33Z,2021-08-30T21:29:33Z,,NONE,,"Hi,

First of all, thank you for this wonderful project :smile:

I started to use dogsheep to make my personal data searchable, and by using the project I noticed an issue with the index command.

It always expects you are running it from the root folder from where the data is located, so I got some errors while trying to make it work on my setup.

I separate all databases inside a `data` folder (I published my setup to be easier to follow: https://github.com/humrochagf/my-dogsheep)

Before, I configured `dogsheep.yml` to add the data folder to its path like this:

```yml
data/twitter.db:
    tweets:
        sql: |-
...
```

And running the index command like this:

```
dogsheep-beta index data/dogsheep.db dogsheep.yml
```

It worked to the normal search feature with no problem this way, but when I started adding `display_sql` rules the app started to crash, because at datasette `get_database` it was looking for `data/twitter` and it only had a db called `twitter` there.

So my workaround to that was to cd into the data folder and run the indexer. You can check the way I'm doing it at this line of the makefile: https://github.com/humrochagf/my-dogsheep/blob/main/makefile#L3

It works but it would be nice to have an option to pass the path where the data is located to the index function.",197431109,dogsheep-beta,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-beta/issues/34/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
870946764,MDU6SXNzdWU4NzA5NDY3NjQ=,1312,how to query many-to-many relationship via json API?,5268174,bram2000,open,0,,,,,0,2021-04-29T12:09:49Z,2021-04-29T12:09:49Z,,NONE,,"Hi,

Firstly thanks for Datasette, it's great!

I'm trying to use the JSON API to query data from a Datasette instance. I have a simple 3 table many-to-many relationship, like so:

`category` - list of categories
`document` - list of documents
`document_category` - join table (a category contains many documents, and a document can be a member of multiple categories)

the `document_category` table foreign keys to the other two using their respective row_ids.

Now I want to return ""all documents within category X"" but I cannot see a way to do this without executing two queries; the first to lookup the row_id of category X, and the second to join `document` with `document_category` where category ID is <id>.

I could easily write this in SQL, but this makes programmatic handling of pagination much more difficult (we'd have to dynamically modify the SQL to select the row_id and include the correct where and limit clauses).

Is there a way to achieve this using the JSON API?

",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1312/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1117132741,I_kwDOBm6k_c5ClhfF,1615,Potential simplified publishing mechanism,369053,aidansteele,closed,0,,,,,2,2022-01-28T08:34:50Z,2022-02-02T07:34:21Z,2022-02-02T07:34:17Z,NONE,,"Hi,

Forewarning: this idea is one I've only been thinking about for a while and it's not fully fleshed-out yet.

I love Datasette and what it stands for. I was thinking about how we could make it accessible to more people, especially those without access to credit cards required for a lot of hosting options. Or they might not feel comfortable signing up for said services.

So I was thinking I might create a service that hosts Datasette instances for folks. I'd probably stick it on AWS Lambda and limit requests to something like n/month to avoid bankrupting myself. If I did build such a hypothetical service, I was thinking I would rely on GitHub Actions to do the heavy lifting. 

E.g. user `johndoe` creates a repo `my-animals` with a couple of files: `dogs.csv`, `cats.csv` and the following GitHub Actions workflow:

```yaml
# .github/workflows/push.yml
on:
  push

# this allows the publish action to use OIDC to authenticate johndoe/my-animals
permissions:
  id-token: write
  contents: read

jobs:
  publish:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/setup-python@v2

      - run: pip install sqlite-utils

      - uses: actions/checkout@v2

      - run: |
          set -eux
          
          sqlite-utils create-database animals.db          
          sqlite-utils insert animals.db dogs dogs.csv --csv          
          sqlite-utils insert animals.db cats cats.csv --csv

      - uses: datasette-hub/publish@v1
        with:
          db: animals.db
          metadata: meta.yml

        # this step is helpful for debugging why the
        # generated sqlite db was rejected
      - uses: actions/upload-artifact@v2
        if: failure()
        with:
          path: animals.db
          retention-days: 1
```

This would then cause a Datasette instance to be available at `https://johndoe-my-animals.datasette-hub.test/`. It feels like this could significantly reduce the friction to someone being able to go from data set to Datasette.

What do you think? Does this address a real need? Or am I perhaps misunderstanding the main friction points? As a bonus: it feels like this would pair well with [git scraping](https://simonwillison.net/2020/Oct/9/git-scraping/).",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1615/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
891969037,MDU6SXNzdWU4OTE5NjkwMzc=,1326,How to limit fields returned from the JSON API?,5268174,bram2000,closed,0,,,,,1,2021-05-14T14:27:41Z,2021-05-23T02:55:06Z,2021-05-23T02:55:00Z,NONE,,"Hi,

I have quite wide tables, and in many cases only want a subset of the data (to save on network bandwidth). I need to use the JSON API as handling pagination is so much easier, but I can't see a way to select specific columns.

Is there a way to do this, or is it a feature request?

Thanks!",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1326/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
870125126,MDU6SXNzdWU4NzAxMjUxMjY=,1310,I'm creating a plugin to export a spreadsheet file (.ods or .xlsx),3747136,ColinMaudry,closed,0,,,,,2,2021-04-28T16:20:11Z,2021-04-30T07:26:11Z,2021-04-30T06:58:46Z,NONE,,"Hi,

I have started developing a plugin to export records as a spreadsheet file. It could be ods or xlsx, whatever is easier.

I have spotted the following packages:

- ods files: https://pypi.org/project/odswriter/
- xlsx files: https://openpyxl.readthedocs.io/en/stable/index.html (quite powerful) or https://xlsxwriter.readthedocs.io/ (faster)

This is the code I have so far, I test it with the `--plugins-dir` option:

```python
from datasette import hookimpl
from datasette.utils.asgi import Response
import odswriter as ods

def render_spreadsheet(rows):
    with ods.writer(open(""test.ods"",""wb"")) as odsfile:
        for row in rows:
            odsfile.writerow([""String"", ""ABCDEF123456"", ""123456""])
        return Response(odsfile, content_type=""application/vnd.oasis.opendocument.spreadsheet"", status=200)


@hookimpl
def register_output_renderer():
    return {""extension"": ""ods"", ""render"": render_spreadsheet}

``` 

I get the following error:

```
Traceback (most recent call last):
  File ""/home/colin/.local/lib/python3.8/site-packages/datasette/app.py"", line 1128, in route_path
    await response.asgi_send(send)
  File ""/home/colin/.local/lib/python3.8/site-packages/datasette/utils/asgi.py"", line 339, in asgi_send
    body = body.encode(""utf-8"")
AttributeError: 'ODSWriter' object has no attribute 'encode'
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File ""/home/colin/.local/lib/python3.8/site-packages/datasette/app.py"", line 1128, in route_path
    await response.asgi_send(send)
  File ""/home/colin/.local/lib/python3.8/site-packages/datasette/utils/asgi.py"", line 339, in asgi_send
    body = body.encode(""utf-8"")
AttributeError: 'ODSWriter' object has no attribute 'encode'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File ""/home/colin/.local/lib/python3.8/site-packages/uvicorn/protocols/http/h11_impl.py"", line 396, in run_asgi
    result = await app(self.scope, self.receive, self.send)
  File ""/home/colin/.local/lib/python3.8/site-packages/uvicorn/middleware/proxy_headers.py"", line 45, in __call__
    return await self.app(scope, receive, send)
  File ""/home/colin/.local/lib/python3.8/site-packages/datasette/utils/asgi.py"", line 161, in __call__
    await self.app(scope, receive, send)
  File ""/home/colin/.local/lib/python3.8/site-packages/datasette/tracer.py"", line 75, in __call__
    await self.app(scope, receive, send)
  File ""/home/colin/.local/lib/python3.8/site-packages/asgi_csrf.py"", line 107, in app_wrapped_with_csrf
    await app(scope, receive, wrapped_send)
  File ""/home/colin/.local/lib/python3.8/site-packages/datasette/app.py"", line 1086, in __call__
    return await self.route_path(scope, receive, send, path)
  File ""/home/colin/.local/lib/python3.8/site-packages/datasette/app.py"", line 1133, in route_path
    return await self.handle_500(request, send, exception)
  File ""/home/colin/.local/lib/python3.8/site-packages/datasette/app.py"", line 1267, in handle_500
    await asgi_send_html(
  File ""/home/colin/.local/lib/python3.8/site-packages/datasette/utils/asgi.py"", line 217, in asgi_send_html
    await asgi_send(
  File ""/home/colin/.local/lib/python3.8/site-packages/datasette/utils/asgi.py"", line 237, in asgi_send
    await asgi_start(send, status, headers, content_type)
  File ""/home/colin/.local/lib/python3.8/site-packages/datasette/utils/asgi.py"", line 246, in asgi_start
    await send(
  File ""/home/colin/.local/lib/python3.8/site-packages/asgi_csrf.py"", line 103, in wrapped_send
    await send(event)
  File ""/home/colin/.local/lib/python3.8/site-packages/uvicorn/protocols/http/h11_impl.py"", line 482, in send
    raise RuntimeError(msg % message_type)
RuntimeError: Expected ASGI message 'http.response.body', but got 'http.response.start'.
```

I tried with `AsgiFileDownload` like in [DatabaseDownload](https://github.com/simonw/datasette/blob/main/datasette/views/database.py#L150) to deal with the binary nature of the ods file, but the renderer expects a Response:

> <datasette.utils.asgi.AsgiFileDownload object at 0x7f5b9bd503d0> should be dict or Response

However, the `Response` class only supports the following methods, not binary:

- html
- text
- json
- redirect

How would you suggest me to proceed to have my ods file downloaded?

",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1310/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1236693079,I_kwDOCGYnMM5JtnBX,432,"Support `rows_where()`, `delete_where()` etc for attached alias databases",11597658,luxint,open,0,,,,,5,2022-05-16T06:38:58Z,2022-06-14T22:16:48Z,,NONE,,"Hi,

I noticed `rows_where()` doesn't return any rows from tables which are from attached databases. The `exists()` function returns false. As far as I can see this is because the `table_names()` function only looks for table names in the current database and not in attached (or temp) databases.

Besides, `rows_where()`, also `insert_all()` and `delete_where()` didn't do what I was expecting because of this. For the moment I've patched `table_names()` for myself, see below but I'm not sure what the total impact is on the other functions like lookup truncate etc which all use `exists()`. Also `view_names()` doesn't look for views in attached or temp databases. 
```python
 def table_names(self, fts4: bool = False, fts5: bool = False) -> List[str]:
        ""A list of string table names in this database.""
        where = [""type = 'table'""]
        if fts4:
            where.append(""sql like '%USING FTS4%'"")
        if fts5:
            where.append(""sql like '%USING FTS5%'"")
        dbs = [x[1] for x in self.execute('pragma database_list').fetchall()]    
        lst=[]
        for db in dbs:    
            sql = ""select name from {} where {}"".format(db+"".sqlite_master"","" AND "".join(where))
            lst.extend(r[0] for r in self.execute(sql).fetchall())
        return lst
```",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/432/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1082651698,I_kwDOCGYnMM5Ah_Qy,358,Support for CHECK constraints,11597658,luxint,open,0,,,,,7,2021-12-16T21:19:45Z,2022-09-25T07:15:59Z,,NONE,,"Hi,

I noticed the `transform.table()` method doesn't have an option to add/change or drop a check constraint (see https://sqlite.org/lang_createtable.html -> 3.7 Check Constraints. would be great to have this as an option! 
",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/358/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
333238932,MDU6SXNzdWUzMzMyMzg5MzI=,316,datasette inspect takes a very long time on large dbs,132230,gavinband,closed,0,,,,,5,2018-06-18T11:56:27Z,2019-05-11T18:26:25Z,2019-05-11T18:26:25Z,NONE,,"Hi,

I want to expose data in a very large sqlite database (~600Gb) to the web.  I have used datasette with success on smaller test databases with the same schema - it works very well (thanks!).  However, using the full db, both `datasette inspect` and `datasette serve` seem to hang or pause for a very long time (tens of minutes) on startup.  Is this expected behaviour?

(I noticed that the output of `datasette inspect` includes row counts for each table.   Simply counting the rows in this db will take a long time (tens of millions of rows across each of ~10 tables), so I wondered if this is the source of the problem.)

Any help on a workaround would be appreciated.
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/316/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
661605489,MDU6SXNzdWU2NjE2MDU0ODk=,900,Some links don't honor base_url,50220,noteed,closed,0,,,6026070,0.51,3,2020-07-20T09:40:50Z,2020-10-23T19:44:04Z,2020-10-15T22:57:55Z,NONE,,"Hi,

I've been playing with Datasette behind Nginx (awesome tool, thanks !). It seems some URLs are OK but some aren't. For instance in https://github.com/simonw/datasette/blob/master/datasette/templates/query.html#L61 it seems that `url_csv` includes a `/` prefix, resulting in the `base_url` not beeing honored.

Actually here, it seems that dropping the prefix `/` to make the link relative is enough (so it may not be strictly related to `base_url`).

Additional information:

```
datasette, version 0.45+0.gf1f581b.dirty
```

Relevant Nginx configuration (note that all the trailing slashes have some effect):

```
  location /datasette/ {
    proxy_pass http://127.0.0.1:9001/;
    proxy_set_header Host $host;
  }
```

Relelvant Datasette configuration (slashes matter too):

```
  --config base_url:/datasette/
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/900/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
849512840,MDU6SXNzdWU4NDk1MTI4NDA=,1288,Facets: show counts for null,1111743,jungle-boogie,open,0,,,,,0,2021-04-02T22:33:44Z,2021-04-02T22:33:44Z,,NONE,,"Hi,

Thank you for Datasette and being a fan of SQLite!

Not all rows in a record will always contain data.
So when using a facet on a column where some records have data and others don't, you don't get an accurate count of the results.

Please consider also counting and showing null records with facets.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1288/reactions"", ""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1824457306,I_kwDOBm6k_c5svwJa,2122,Parameters on canned queries: fixed or query-generated list?,1563881,meowcat,open,0,,,,,0,2023-07-27T14:07:07Z,2023-07-27T14:07:07Z,,NONE,,"Hi,

currently parameters in canned queries are just text fields. It would be cool to have one of the options below. Would you accept a PR doing something in this direction? (Possibly this could even work as a plugin.)

* adding facets, which would work like facets on tables or views, giving a list of selectable options (and leaving parameters as is)
* making it possible to provide a query which returns selectable values for a parameter, e.g.
```     
calendar_entries_current_instrument:
        sql: |      
          select * from calendar_entries 
              where 
                    DTEND_UNIX > UNIXEPOCH() and
                    DTSTART_UNIX < UNIXEPOCH() + :days *24*60*60 and
                    current = 1 and
                    MACHINE = :instrument
              order by
                    DTSTART_UNIX
        params:
            days: 
              sql: ""SELECT VALUE FROM generate_series(1, 30, 1)""
              # this obviously requires the corresponding sqlite extension
            instrument:
              sql: ""SELECT DISTINCT MACHINE FROM calendar_entries""
```
* making it possible to provide a fixed list of parameters
```     
calendar_entries_current_instrument:
        sql: |      
          select * from calendar_entries 
              where 
                    DTEND_UNIX > UNIXEPOCH() and
                    DTSTART_UNIX < UNIXEPOCH() + :days *24*60*60 and
                    current = 1 and
                    MACHINE = :instrument
              order by
                    DTSTART_UNIX
        params:
            days: 
              values: [1, 2, 3, 5, 10, 20, 30]
            instrument:
              values: [supermachine, crappymachine, boringmachine]
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2122/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
915421499,MDU6SXNzdWU5MTU0MjE0OTk=,267,row.update() or row.pk,12721157,Gravitar64,open,0,,,,,4,2021-06-08T19:56:00Z,2021-06-22T17:27:27Z,,NONE,,"Hi,

fantastic framework for working with Sqlite3 databases!!!

I tried to update spezific rows in a table and used

for row in db[tablename]:
    newValue = row[""counter""] * row[""prize""]    
    row.update({""Fieldname"": newValue})
    print(row)


This updates the value in the printet row, but not in the database. So I switched to

db[tablename].update(id, {""Filedname"": newValue})

This works fine. But row.update would be nicer, because no need for the id (its that row), no need for the tablename and the db (all defined in the for row ... loop).

Thx

",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/267/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
548591089,MDU6SXNzdWU1NDg1OTEwODk=,657,Allow creation of virtual tables at startup,1055831,dazzag24,open,0,,,,,4,2020-01-12T16:10:55Z,2021-01-15T20:24:35Z,,NONE,,"Hi,
 
I've been experimenting with SQLite reading from huge datasets using this excellent Parquet extension from @cldellow.
https://cldellow.com/2018/06/22/sqlite-parquet-vtable.html
https://github.com/cldellow/sqlite-parquet-vtable

This works really well, but I was keen to see if I could combine datasette with this. Having previously experimented with the spatialite extension I knew that datasette supports loading extensions in the underlying sqlite instance.  However I hit a blocker as the current design only allows SELECT statements to be executed and so I am unable to execute the crucial 

CREATE VIRTUAL TABLE .........

command that is required to load the data from the parquet file into the table.

It seems like this would be a simple-ish change, but I don't know enough about the architecture of datasette to start implementing this myself?  Could this be done as a datasette plugin?  or would this require more fundamental changes at initialisation time?

My thoughts are that something at init time could detect that the user was loading a *.parquet file and then switch to a mode were it loads that via the ""CREATE VIRTUAL TABLE..."" rather than loading the *.db file in the default case??

I'm happy to contribute code and testing, I just need some pointers on the best approach.

Thanks
Darren",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/657/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
400229984,MDU6SXNzdWU0MDAyMjk5ODQ=,401,How to pass configuration to plugins?,1055831,dazzag24,closed,0,,,,,3,2019-01-17T11:20:41Z,2019-01-18T11:48:13Z,2019-01-18T06:49:07Z,NONE,,"Hi,
Firstly, thanks for your work on datasette, it is a hugely useful tool!

I've been working on a fork [https://github.com/dazzag24/datasette-cluster-map] of datasette-cluster-map to allow the tileserver to be easily switched. Primarily because the tiles being served in the current version use localised text for labels and I'd like to have English used for these names instead.

It uses http://leaflet-extras.github.io/leaflet-providers/preview/ to allow you to simply set the tile provider using a call like so:
```    
let tiles = L.tileLayer.provider('Esri.WorldTopoMap');
```
instead of the current:
```
let tiles = L.tileLayer('https://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png', {
        maxZoom: 19,
        detectRetina: true,
        attribution: '&copy; <a href=""https://www.openstreetmap.org/copyright"">OpenStreetMap</a> contributors'
    }),
  ```
However I've got stuck in trying to work out how to pass the provider string to the plugin.
In the documentation: https://datasette.readthedocs.io/en/stable/plugins.html you discuss configuration of plugins and use an example of passing in which latitude and longitude columns should be used.  However I cannot seem to see anywhere in the current datasette-cluster-map code where these config params are passed in or used.

Can you please point me to an example or how to pass configuration from the metadata.json down into a plugin.  Once I've over come this issue I was wondering if you would be interested in taking this change into your version?

Many thanks
Darren",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/401/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
806743116,MDU6SXNzdWU4MDY3NDMxMTY=,1220,Installing datasette via docker: Path 'fixtures.db' does not exist,30607,aborruso,closed,0,,,,,4,2021-02-11T21:09:14Z,2021-02-12T21:35:17Z,2021-02-12T21:35:17Z,NONE,,"Hi,
If I run

```
docker run -p 8001:8001 -v `pwd`:/mnt \                                                                         1 ↵
    datasetteproject/datasette \
    datasette -p 8001 -h 0.0.0.0 fixtures.db
```

I have 

```
Error: Invalid value for '[FILES]...': Path 'fixtures.db' does not exist.
```

If I run `test -f fixtures.db && echo ""it exists.""` I have `it exists.`.

What's my error?

Thank you",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1220/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
944326512,MDU6SXNzdWU5NDQzMjY1MTI=,296,"`table.search(..., quote=True)` parameter and `sqlite-utils search --quote` option",32427188,deafmute1,closed,0,,,,,6,2021-07-14T11:26:47Z,2021-08-18T20:13:12Z,2021-08-18T20:10:48Z,NONE,,"Hi,
Recently got this error:
```
Traceback (most recent call last):
  File ""<stdin>"", line 1, in <module>
  File ""/home/ethan/git/music-metadata-indexer/src/mmindexer/__init__.py"", line 38, in <module>
    start(""/home/ethan/git/music-metadata-indexer/sample"", ""/home/ethan/git/music-metadata-indexer/test.db"")
  File ""/home/ethan/git/music-metadata-indexer/src/mmindexer/__init__.py"", line 23, in start
    scanner.build_database()
  File ""/home/ethan/git/music-metadata-indexer/src/mmindexer/scan.py"", line 79, in build_database
    _import_song(self.db, Path(dirpath).joinpath(f), self.logger) 
  File ""/home/ethan/git/music-metadata-indexer/src/mmindexer/scan.py"", line 23, in _import_song
    db.add_song(filepath)
  File ""/home/ethan/git/music-metadata-indexer/src/mmindexer/index.py"", line 166, in add_song
    for match in self.search(""albums"", album): 
  File ""/home/ethan/git/music-metadata-indexer/env/lib/python3.9/site-packages/sqlite_utils/db.py"", line 1625, in search
    cursor = self.db.execute(
  File ""/home/ethan/git/music-metadata-indexer/env/lib/python3.9/site-packages/sqlite_utils/db.py"", line 243, in execute
    return self.conn.execute(sql, parameters)
sqlite3.OperationalError: fts5: syntax error near ""."" 
```
So, the error seems to suggest there was a ""."" character somewhere in the SQL command that was causing the error. I did a little digging and found this in the docs: https://www.sqlite.org/fts5.html#fts5_strings. ""."" is one of the many prohibited characters.

My solution was to just strip these out of the query using this line
`query = query.translate({e: None for e in itertools.chain(range(0,26), range(27, 48), range(58,65), range(91,95), [96], range(123,128))})`

Perhaps this could be included into the `table.search()` function?

",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/296/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
410384988,MDU6SXNzdWU0MTAzODQ5ODg=,411,How to pass named parameter into spatialite MakePoint() function,1055831,dazzag24,closed,0,,,,,3,2019-02-14T16:30:22Z,2023-10-25T13:23:04Z,2019-05-05T12:25:04Z,NONE,,"Hi,
datasette version: ""0.26.2""
extensions: 
       spatialite: ""4.4.0-RC0""
sqlite version: ""3.22.0""

I have a table of airports with latitude and longitude columns.  I've added spatialite (with KNN support).  After creating the db using csvs-to-sqlit, I run these commands to setup the spatialite tables:

```
conn.execute('SELECT InitSpatialMetadata(1)')

conn.execute(""SELECT AddGeometryColumn('airports', 'point_geom', 4326, 'POINT', 2);"")

conn.execute('''UPDATE airports SET point_geom = GeomFromText('POINT('||""longitude""||' '||""latitude""||')',4326);''')

conn.execute(""SELECT CreateSpatialIndex('airports', 'point_geom');"")
```

I'm attempting to create a canned query and have this in my metadata.json file:
```
""find_airports_nearest_to_point"":{
    ""sql"":""SELECT a.pos AS rank, b.id, b.name, b.country, b.latitude AS latitude, b.longitude AS longitude, a.distance / 1000.0 AS dist_km FROM KNN AS a JOIN airports AS b ON (b.rowid = a.fid) WHERE f_table_name = \""airports\"" AND ref_geometry = MakePoint( :Long , :Lat ) AND max_items = 10;""}
```
which doesn't seem to perform the templating of the name parameters correctly and I get no results.  

Have also tired:
```
MakePoint( || :Long || , || :Lat || )
```
which returns this error:
```
near ""||"": syntax error
```

However I cannot seem to find the correct combination of named parameter syntax (:Lat) or sqlite concatenation operator to make it work.  Any ideas if using named parameters inside functions is supported?

Thanks
Darren",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/411/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
408518024,MDU6SXNzdWU0MDg1MTgwMjQ=,410,How to setup a multi database environment?,30607,aborruso,closed,0,,,,,1,2019-02-10T09:39:24Z,2019-04-12T04:42:28Z,2019-04-12T04:42:27Z,NONE,,"Hi,
first of all I need to write that Simon Willison and datasette are really great.

I have probably a stupid question, but it seems to me that I do not have the reply in the documentation.

I have installed datasette and run it with `datasette mydb.db`, and I can reach it  on `http://127.0.0.1:8001`.

But how to work with more than one db? Imagine I have ten sqlite databases, and that I need to explore/query these via datasette, how to run datasette? Is it possibile to create a sort of db index and than run `datasette serve myindex`?

Thank you",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/410/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
605806386,MDU6SXNzdWU2MDU4MDYzODY=,735,"Error when I click on ""View and edit SQL""",30607,aborruso,closed,0,,,,,2,2020-04-23T19:31:32Z,2020-04-28T06:10:20Z,2020-04-27T19:00:30Z,NONE,,"Hi,
when I do it [here](https://my-database.now.sh/commissioniComunePalermo/youtube), I have ""unrecognized token: ""["""" error.

Is it normal?

Thank you",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/735/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
608058890,MDU6SXNzdWU2MDgwNTg4OTA=,744,link_or_copy_directory() error - Invalid cross-device link,30607,aborruso,closed,0,,,,,28,2020-04-28T06:26:45Z,2020-05-28T14:32:53Z,2020-05-27T06:01:28Z,NONE,,"Hi,
when I run

```
datasette  publish heroku -n myapp --template-dir ./template mydb.db
```

I have this error

```
Traceback (most recent call last):
  File ""/home/aborruso/.local/lib/python3.7/site-packages/datasette/utils/__init__.py"", line 607, in link_or_copy_directory
    shutil.copytree(src, dst, copy_function=os.link)
  File ""/usr/lib/python3.7/shutil.py"", line 365, in copytree
    raise Error(errors)
shutil.Error: [('/myfolder/youtubeComunePalermo/processing/./template/base.html', '/tmp/tmps9_4mzc4/templates/base.html', ""[Errno 18] Invalid cross-device link: '/myfolder/youtubeComunePalermo/processing/./template/base.html' -> '/tmp/tmps9_4mzc4/templates/base.html'""), ('/myfolder/youtubeComunePalermo/processing/./template/index.html', '/tmp/tmps9_4mzc4/templates/index.html', ""[Errno 18] Invalid cross-device link: '/myfolder/youtubeComunePalermo/processing/./template/index.html' -> '/tmp/tmps9_4mzc4/templates/index.html'"")]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File ""/home/aborruso/.local/bin/datasette"", line 8, in <module>
    sys.exit(cli())
  File ""/home/aborruso/.local/lib/python3.7/site-packages/click/core.py"", line 829, in __call__
    return self.main(*args, **kwargs)
  File ""/home/aborruso/.local/lib/python3.7/site-packages/click/core.py"", line 782, in main
    rv = self.invoke(ctx)
  File ""/home/aborruso/.local/lib/python3.7/site-packages/click/core.py"", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File ""/home/aborruso/.local/lib/python3.7/site-packages/click/core.py"", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File ""/home/aborruso/.local/lib/python3.7/site-packages/click/core.py"", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File ""/home/aborruso/.local/lib/python3.7/site-packages/click/core.py"", line 610, in invoke
    return callback(*args, **kwargs)
  File ""/home/aborruso/.local/lib/python3.7/site-packages/datasette/publish/heroku.py"", line 103, in heroku
    extra_metadata,
  File ""/usr/lib/python3.7/contextlib.py"", line 112, in __enter__
    return next(self.gen)
  File ""/home/aborruso/.local/lib/python3.7/site-packages/datasette/publish/heroku.py"", line 191, in temporary_heroku_directory
    os.path.join(tmp.name, ""templates""),
  File ""/home/aborruso/.local/lib/python3.7/site-packages/datasette/utils/__init__.py"", line 609, in link_or_copy_directory
    shutil.copytree(src, dst)
  File ""/usr/lib/python3.7/shutil.py"", line 321, in copytree
    os.makedirs(dst)
  File ""/usr/lib/python3.7/os.py"", line 221, in makedirs
    mkdir(name, mode)
FileExistsError: [Errno 17] File exists: '/tmp/tmps9_4mzc4/templates'
```

I'm attaching my very basic template folder.

Thank you 

[template.zip](https://github.com/simonw/datasette/files/4543751/template.zip)
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/744/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
606720674,MDU6SXNzdWU2MDY3MjA2NzQ=,736,strange behavior using accented characters,30607,aborruso,closed,0,,,,,3,2020-04-25T08:34:51Z,2020-04-28T06:09:28Z,2020-04-27T18:59:16Z,NONE,,"Hi,
when I search `incompatibilità` [here](https://my-database.now.sh/commissioniComunePalermo/youtube), using full text search, it becomes `incompatibilità` and I have no result.

If I encode the `à` char in the URL (`incompatibilit%C3%A0`) I have the right result.

![image](https://user-images.githubusercontent.com/30607/80275201-00a79380-86e0-11ea-865e-f7e1474e8098.png)
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/736/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1577548579,I_kwDOBm6k_c5eB3sj,2021,Docker images for 1.0 alphas?,1563881,meowcat,open,0,,,,,0,2023-02-09T09:35:52Z,2023-02-09T09:35:52Z,,NONE,,"Hi,
would you consider putting 1.0alpha images on Dockerhub? 

(Also, how usable are the alphas?)",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2021/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
467218270,MDU6SXNzdWU0NjcyMTgyNzA=,558,Support unicode in url,380586,0x1997,closed,0,,,,,4,2019-07-12T04:43:24Z,2019-07-15T01:29:30Z,2019-07-14T02:49:33Z,NONE,,"Hi, I defined some custom queries in my `metadata.json`. There are Chinese characters in the names of the queries. So the urls are like `http://127.0.0.1:8001/mydb/测试查询`.
When opening such urls, datasette will throw an exception.
```
Traceback (most recent call last):
  File ""/home/zhe/miniconda3/lib/python3.7/site-packages/datasette/utils/asgi.py"", line 100, in __call__
    return await view(new_scope, receive, send)
  File ""/home/zhe/miniconda3/lib/python3.7/site-packages/datasette/utils/asgi.py"", line 172, in view
    request, **scope[""url_route""][""kwargs""]
  File ""/home/zhe/miniconda3/lib/python3.7/site-packages/datasette/views/base.py"", line 267, in get
    request, database, hash, correct_hash_provided, **kwargs
  File ""/home/zhe/miniconda3/lib/python3.7/site-packages/datasette/views/base.py"", line 471, in view_get
    for key in self.ds.renderers.keys()
  File ""/home/zhe/miniconda3/lib/python3.7/site-packages/datasette/views/base.py"", line 471, in <dictcomp>
    for key in self.ds.renderers.keys()
  File ""/home/zhe/miniconda3/lib/python3.7/site-packages/datasette/utils/__init__.py"", line 655, in path_with_format
    path = request.path
  File ""/home/zhe/miniconda3/lib/python3.7/site-packages/datasette/utils/asgi.py"", line 49, in path
    self.scope.get(""raw_path"", self.scope[""path""].encode(""latin-1""))
UnicodeEncodeError: 'latin-1' codec can't encode characters in position 9-11: ordinal not in range(256)
```
This used to work when datasette was based on sanic.

Btw, thanks for the great work!",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/558/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1446657889,I_kwDOBm6k_c5WOj9h,1885,Integrate inside GUI app (tkinter),5115787,dmalves,open,0,,,,,0,2022-11-13T00:10:43Z,2022-11-13T00:11:09Z,,NONE,,"Hi, I'd like to integrate datasette inside a tkinter app. The app should be able to start/stop datasette server. How could I integrate datasette inside my app, so it can start and stop datasette server?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1885/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1977726056,I_kwDOBm6k_c514bRo,2203,custom plugin not seen as sql function,7113541,LyzardKing,open,0,,,,,0,2023-11-05T10:30:19Z,2023-11-05T10:30:19Z,,NONE,,"Hi, I'm not sure if this is the right repo for this issue.

I'm using datasette with the parquet (to read a duckdb), and jellyfish plugins. Both work perfectly.

Now I need to create a simple plugin that uses the python rouge package and returns a similarity score (similarly to how the jellyfish plugin works).
If I create a custom plugin, even the example hello_world one, copied directly from the tutorial, I get the following error:
```duckdb.duckdb.CatalogException: Catalog Error: Scalar Function with name hello_world does not exist!```

Since the jellyfish plugin doesn't do anything more complex, I'm wondering if there is some other kind of issue with my setup.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2203/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1817281557,I_kwDOC8SPRc5sUYQV,37,cannot use jinja filters in display?,10352819,rprimet,closed,0,,,,,1,2023-07-23T20:09:54Z,2023-07-23T20:18:27Z,2023-07-23T20:18:26Z,NONE,,"Hi, I'm trying to have a display function in Dogsheep's `config.yml` that includes something like this:

```         
<h3>
  <a href=""{{ urls.row('my_database', 'my_table', key) }}"">{{ display.title }}</a>
  <a href=""{{ display.url }}🔗"" target=""_blank"">(source)</a>
</h3>
<p>{{ display.snippet|safe }}</p>
```

Unfortunately, rendering fails with a message 'urls is undefined'.

The same happens if I'm trying to build a row URL manually, using filters like `quote_plus` (as my keys are URLs).

Any hints?

Thanks!",197431109,dogsheep-beta,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-beta/issues/37/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1899310542,I_kwDOBm6k_c5xNS3O,2187,Datasette for serving JSON only,19705106,geofinder,open,0,,,,,0,2023-09-16T05:48:29Z,2023-09-16T05:48:29Z,,NONE,,"Hi, is there any way to use datasette for serving json only without displaying webpage? I've tried to search about this in documentation but didn't get any information",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2187/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
534507142,MDU6SXNzdWU1MzQ1MDcxNDI=,69,Feature request: enable extensions loading,30607,aborruso,closed,0,,,,,3,2019-12-08T08:06:25Z,2022-02-05T00:04:25Z,2020-10-16T18:42:49Z,NONE,,"Hi, it would be great to add a parameter that enables the load of a sqlite extension you need.

Something like ""-ext modspatialite"".

In this way your great tool would be even more comfortable and powerful.


Thank you very much",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/69/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1145882578,I_kwDOCGYnMM5ETMfS,408,`deterministic=True` fails on versions of SQLite prior to 3.8.3,24938923,learning4life,closed,0,,,,,6,2022-02-21T14:36:43Z,2022-03-13T16:54:09Z,2022-03-02T00:38:11Z,NONE,,"Hi, love your work.

I am unable to lookup indexes in a database using sqlite-utils:

`
sqlite-utils indexes city_spec.db --table`

or

`sqlite-utils indexes city_spec.db MyTable
`

**Software**
sqlite-utils, version 3.24
sqlite3 --version: 3.36.0 

**Output:**

Traceback (most recent call last):
  File ""/opt/app-root/bin/sqlite-utils"", line 8, in <module>
    sys.exit(cli())
  File ""/opt/app-root/lib64/python3.8/site-packages/click/core.py"", line 1128, in __call__
    return self.main(*args, **kwargs)
  File ""/opt/app-root/lib64/python3.8/site-packages/click/core.py"", line 1053, in main
    rv = self.invoke(ctx)
  File ""/opt/app-root/lib64/python3.8/site-packages/click/core.py"", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File ""/opt/app-root/lib64/python3.8/site-packages/click/core.py"", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File ""/opt/app-root/lib64/python3.8/site-packages/click/core.py"", line 754, in invoke
    return __callback(*args, **kwargs)
  File ""/opt/app-root/lib64/python3.8/site-packages/click/decorators.py"", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File ""/opt/app-root/lib64/python3.8/site-packages/sqlite_utils/cli.py"", line 2123, in indexes
    ctx.invoke(
  File ""/opt/app-root/lib64/python3.8/site-packages/click/core.py"", line 754, in invoke
    return __callback(*args, **kwargs)
  File ""/opt/app-root/lib64/python3.8/site-packages/sqlite_utils/cli.py"", line 1624, in query
    db.register_fts4_bm25()
  File ""/opt/app-root/lib64/python3.8/site-packages/sqlite_utils/db.py"", line 403, in register_fts4_bm25
    self.register_function(rank_bm25, deterministic=True)
  File ""/opt/app-root/lib64/python3.8/site-packages/sqlite_utils/db.py"", line 399, in register_function
    register(fn)
  File ""/opt/app-root/lib64/python3.8/site-packages/sqlite_utils/db.py"", line 392, in register
    self.conn.create_function(name, arity, fn, **kwargs)
sqlite3.NotSupportedError: deterministic=True requires SQLite 3.8.3 or higher
",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/408/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1841501975,I_kwDOBm6k_c5twxcX,2133,[feature request]`datasette install plugins.json` options,54462,HaveF,closed,0,,,,,9,2023-08-08T15:06:50Z,2023-08-10T00:31:24Z,2023-08-09T22:04:46Z,NONE,,"Hi, simon ❤️ 

`datasette plugins  --all > plugins.json` could generate all plugins info. On another machine, it would be great to install all plugins just by `datasette install plugins.json`",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2133/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1280799259,I_kwDOBm6k_c5MV3Ib,1761,ensure_ascii=False,1473102,mustafa0x,open,0,,,,,0,2022-06-22T19:58:13Z,2022-06-22T19:58:30Z,,NONE,,"Hi, thanks for the project!

For the JSON output, I would consider defaulting to `ensure_ascii=False` (UTF-8 seems pretty universal) or making it an option. When dealing with non-Latin text, `ensure_ascii=True` (the default) can triple the size of the output.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1761/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1387712501,I_kwDOBm6k_c5Sts_1,1824,Convert &_hide_sql=1 to #_hide_sql,562352,CharlesNepote,open,0,,,,,1,2022-09-27T12:53:31Z,2022-10-05T12:56:27Z,,NONE,,"Hiding the SQL textarea with `&_hide_sql=1` enforces a page reload, which can take several seconds and use server resource (which is annoying for big database or complex queries).

It could probably be done with a few lines of Javascript (I'm going to see if I can do that).",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1824/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
271831408,MDU6SXNzdWUyNzE4MzE0MDg=,47,Create neat example database,9599,simonw,closed,0,,,,,5,2017-11-07T13:29:38Z,2017-11-14T03:08:13Z,2017-11-14T03:08:13Z,OWNER,,How about data from open elections  eg https://github.com/openelections/openelections-data-ca?files=1,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/47/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
493670426,MDU6SXNzdWU0OTM2NzA0MjY=,3,Command to fetch all repos belonging to a user or organization,9599,simonw,closed,0,,,,,2,2019-09-14T21:54:21Z,2019-09-17T00:17:53Z,2019-09-17T00:17:53Z,MEMBER,,"How about this:

    $ github-to-sqlite repos simonw",207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/3/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
750089847,MDU6SXNzdWU3NTAwODk4NDc=,1109,Deprecate --config in Datasette 1.0 (in favour of --setting),9599,simonw,open,0,,,3268330,Datasette 1.0,0,2020-11-24T21:43:57Z,2020-12-17T22:07:49Z,,OWNER,,I added a deprecation warning to this in #992.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1109/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1217759117,I_kwDOBm6k_c5IlYeN,1727,Research: demonstrate if parallel SQL queries are worthwhile,9599,simonw,open,0,,,,,32,2022-04-27T18:54:21Z,2022-09-26T14:48:31Z,,OWNER,,"I added parallel SQL query execution here:
- https://github.com/simonw/datasette/issues/1723

My hunch is that this will take advantage of multiple cores, since Python's `sqlite3` module releases the GIL once a query is passed to SQLite.

I'd really like to prove this is the case though. Just not sure how to do it!

Larger question: is this performance optimization actually improving performance at all? Under what circumstances is it worthwhile?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1727/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1083718998,I_kwDOBm6k_c5AmD1W,1567,Remove undocumented sqlite_functions mechanism,9599,simonw,closed,0,,,7571612,Datasette 0.60,0,2021-12-18T01:51:10Z,2022-01-13T22:27:04Z,2021-12-18T01:54:46Z,OWNER,,"I added this in 0b8c1b0a6da9cb8ac0d28cc90dd783de87554036 but it's never been documented and the same thing can now be achieved using the `prepare_connection` plugin hook.

https://github.com/simonw/datasette/blob/0c91e59d2bbfc08884cfcf5d1b902a2f4968b7ff/datasette/app.py#L262

https://github.com/simonw/datasette/blob/0c91e59d2bbfc08884cfcf5d1b902a2f4968b7ff/datasette/app.py#L551-L552

It's used here in the tests:

https://github.com/simonw/datasette/blob/69244a617b1118dcbd04a8f102173f04680cf08c/tests/fixtures.py#L156",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1567/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
723803777,MDU6SXNzdWU3MjM4MDM3Nzc=,1028,--load-extension=spatialite shortcut,9599,simonw,closed,0,,,6026070,0.51,1,2020-10-17T17:02:08Z,2022-01-20T21:29:41Z,2020-10-19T22:37:55Z,OWNER,,I added this to `sqlite-utils` in https://github.com/simonw/sqlite-utils/issues/136 and I really like it: pass a special value of `spatialite` and Datasette should attempt to load it from known likely installation locations.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1028/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1122450452,I_kwDOBm6k_c5C5zwU,1625,Try running tests against macOS and Windows in addition to Ubuntu,9599,simonw,open,0,,,,,0,2022-02-02T22:25:57Z,2022-02-02T22:25:57Z,,OWNER,,"I already do this for `sqlite-utils`: https://github.com/simonw/sqlite-utils/blob/3.22.1/.github/workflows/test.yml

Related:
- #1617
- #1545",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1625/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1227571375,I_kwDOCGYnMM5JK0Cv,431,Allow making m2m relation of a table to itself,738408,rafguns,open,0,,,,,3,2022-05-06T08:30:43Z,2022-06-23T14:12:51Z,,NONE,,"I am building a database, in which one of the tables has a many-to-many relationship to itself. As far as I can see, this is not (yet) possible using `.m2m()` in sqlite-utils. This may be a bit of a niche use case, so feel free to close this issue if you feel it would introduce too much complexity compared to the benefits.

Example: suppose I have a table of people, and I want to store the information that John and Mary have two children, Michael and Suzy. It would be neat if I could do something like this:

```python
from sqlite_utils import Database

db = Database(memory=True)
db[""people""].insert({""name"": ""John""}, pk=""name"").m2m(
    ""people"", [{""name"": ""Michael""}, {""name"": ""Suzy""}], m2m_table=""parent_child"", pk=""name""
)
db[""people""].insert({""name"": ""Mary""}, pk=""name"").m2m(
    ""people"", [{""name"": ""Michael""}, {""name"": ""Suzy""}], m2m_table=""parent_child"", pk=""name""
)
```

But if I do that, the many-to-many table `parent_child` has only one column:
```
CREATE TABLE [parent_child] (
   [people_id] TEXT REFERENCES [people]([name]),
   PRIMARY KEY ([people_id], [people_id])
)
```

This could be solved by adding one or two keyword_arguments to `.m2m()`, e.g. `.m2m(..., left_name=None, right_name=None)` or `.m2m(..., names=(None, None))`.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/431/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1620254998,I_kwDOCGYnMM5gkyEW,532,Show more information when JSON can't be imported with sqlite-utils insert,83080728,voltagex,closed,0,,,,,2,2023-03-12T06:41:44Z,2023-05-08T20:32:16Z,2023-05-08T20:32:02Z,NONE,,"I am currently trying to import the [JSON export of my data from Discord](https://support.discord.com/hc/en-us/articles/360004027692-Requesting-a-Copy-of-your-Data), specifically `activity/reporting/events-*.json`

```
sqlite-utils.exe insert test.db reporting events-2023-00000-of-00001.json
  [###################################-]   99%  00:00:00
Error: Invalid JSON - use --csv for CSV or --tsv for TSV files
```

Please show more information as to *why* this is invalid, if possible.

I am using version 3.30 with Python 3.10 on Windows 11.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/532/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
834602299,MDU6SXNzdWU4MzQ2MDIyOTk=,1262,Plugin hook that could support 'order by random()' for table view,19328961,henry501,open,0,,,,,3,2021-03-18T10:02:01Z,2021-03-18T17:55:01Z,,NONE,,"I am frequently using Datasette to quickly get a visual impression for a table without reviewing it in its entirety. Because I have some groups of similar records, the default sorting options mean that each page is very similar and not representative of the full dataset. The current interface allows sorting by columns, but random sorting is only available via custom SQL.

Maybe this could be a button or link.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1262/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
792851444,MDU6SXNzdWU3OTI4NTE0NDQ=,11,XML parse error,3613583,dskrad,closed,0,,,,,2,2021-01-24T17:38:54Z,2021-02-11T21:18:58Z,2021-02-11T21:18:48Z,NONE,,"I am on Windows 10 using Windows Subsystem for Linux, Python 3.8. I installed evernote-to-sqlite via pipx (in a venv). I tried using enex files from the latest version of Evernote for Windows (10.6.9 which only lets you export 50 notes at a time) and from Legacy Evernote (6.25.2.9198 which lets you export all your notes at once). The enex file from latest evernote gives this error:

    File ""/usr/lib/python3.8/xml/etree/ElementTree.py"", line 1320, in XML parser.feed(text)
    xml.etree.ElementTree.ParseError: XML or text declaration not at start of entity: line 2, column 6

The enex file from Legacy Evernote gives this error:

    File ""/home/david/.local/pipx/venvs/evernote-to-sqlite/lib/python3.8/site-packages/evernote_to_sqlite/utils.py"", line 28, in save_note
    updated = note.find(""updated"").text
    AttributeError: 'NoneType' object has no attribute 'text'",303218369,evernote-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/11/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
450862577,MDU6SXNzdWU0NTA4NjI1Nzc=,496,Additional options to gcloud build command in cloudrun - timeout,1740337,costrouc,closed,0,,,,,1,2019-05-31T15:43:55Z,2019-05-31T23:05:05Z,2019-05-31T23:05:05Z,NONE,,"I am trying to deploy a 3.1 GB dataset to cloudrun with datasette. Currrently the docker build times out. Would be nice to have a timeout flag or additional gcloud commands that could be specified. 

Here is the line https://github.com/simonw/datasette/blob/f825e2012109247fa246e2b938f8174069e574f1/datasette/publish/cloudrun.py#L78

I would be happy to submit a PR to allow for a timeout option. What are your ideas of allowing the user additional build publishing flag options?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/496/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1250161887,I_kwDOCGYnMM5Kg_Tf,438,illegal UTF-16 surrogate,4068,frafra,closed,0,,,,,2,2022-05-26T22:49:52Z,2022-05-27T08:21:53Z,2022-05-27T08:21:53Z,NONE,,"I am trying to insert `https://artsdatabanken.no/Fab2018/api/export/csv` into a SQLite database, but I have an error when using `sqlite-utils`:

```
sqlite-utils insert --csv --delimiter "";"" --encoding=""utf-16-le"" --pk ""Id"" csv fremmedart test.db
  [------------------------------------]    0%
Error: 'utf-16-le' codec can't decode bytes in position 98-99: illegal UTF-16 surrogate

The input you provided uses a character encoding other than utf-8.

You can fix this by passing the --encoding= option with the encoding of the file.

If you do not know the encoding, running 'file filename.csv' may tell you.

It's often worth trying: --encoding=latin-1
```

I tried to convert the file using `iconv -f ""utf-16le"" -t ""utf-8""`, but I still get a similar error (slightly different position):

```
sqlite-utils insert --csv --delimiter "";"" --encoding=utf-8 --pk ""Id"" csv_utf8 fremmedart test.db
  [------------------------------------]    0%
Error: 'utf-8' codec can't decode byte 0xd9 in position 99: invalid continuation byte

The input you provided uses a character encoding other than utf-8.

You can fix this by passing the --encoding= option with the encoding of the file.

If you do not know the encoding, running 'file filename.csv' may tell you.

It's often worth trying: --encoding=latin-1
```

I have no issues reading such file using this Python code:
```python
content = open('csv', encoding='utf-16-le').read())
```

`in2csv` works too.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/438/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
791237799,MDU6SXNzdWU3OTEyMzc3OTk=,1196,Access Denied Error in Windows,2826376,QAInsights,open,0,,,,,2,2021-01-21T15:40:40Z,2021-04-14T19:28:38Z,,NONE,,"I am trying to publish a db to vercel. But while issuing the below command throwing `Access Denied` error which is leading to `RecursionError: maximum recursion depth exceeded while calling a Python object`.

I am using PyCharm and Python 3.9. I have reinstalled both and launched PyCharm as Admin in Windows 10. But still the issue persists.

Issued command `datasette publish vercel jmeter.db --project jmeter --install datasette-vega`

PS: localhost is working fine.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1196/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1468603401,I_kwDOBm6k_c5XiRwJ,1913,Release Datasette 1.0a0,9599,simonw,closed,0,,,8658075,Datasette 1.0a0,9,2022-11-29T19:41:42Z,2022-11-29T20:10:35Z,2022-11-29T20:10:35Z,OWNER,,"I attempted the release just now - https://github.com/simonw/datasette/releases/tag/1.0a0 -  and got an unexpected test failure:

https://github.com/simonw/datasette/actions/runs/3577355358/attempts/1

```
>       assert delete_response.status_code == 200
E       assert 404 == 200
E        +  where 404 = <Response [404 Not Found]>.status_code

/home/runner/work/datasette/datasette/tests/test_api_write.py:396: AssertionError
=========================== short test summary info ============================
FAILED tests/test_api_write.py::test_delete_row[compound_pk_table-row_for_create2-pks2-article,k] - assert 404 == 200
 +  where 404 = <Response [404 Not Found]>.status_code
```
I hit ""retry"" on that test but I expect it to fail again.

",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1913/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1720096994,I_kwDOCGYnMM5mhpji,554,"`IndexError` when doing `.insert(..., pk='id')` after `insert_all`",1231935,xavdid,open,0,,,,,1,2023-05-22T17:13:02Z,2023-05-22T17:18:33Z,,NONE,,"I believe this is related to https://github.com/simonw/sqlite-utils/issues/98.

When `pk` is specified by table A's `insert` call, it throws an index error if a different table has written a row with a higher rowid than exists in the first table. Here's a basic example:

```py
from sqlite_utils import Database


def test_pk_for_insert(fresh_db):
    user = {""id"": ""abc"", ""name"": ""david""}

    fresh_db[""users""].insert(user, pk=""id"")

    fresh_db[""comments""].insert_all(
        [
            {""id"": ""def"", ""text"": ""ok""},
            {""id"": ""ghi"", ""text"": ""great""},
        ],
    )

    fresh_db[""users""].insert(
        user,
        ignore=True,
        # BUG: when specifying pk on the second insert call 
        # db.py goes into a block it doesn't expect and we get the error
        pk=""id"",
    )


if __name__ == ""__main__"":
    db = Database(""bug.db"")
    if db[""users""].exists():
        raise ValueError(
            ""bug only shows on a new database - remove bug.db before running the script""
        )
    test_pk_for_insert(db)
```

The error is:

```py
  File ""/Users/david/projects/reddit-to-sqlite/.venv/lib/python3.11/site-packages/sqlite_utils/db.py"", line 2960, in insert_chunk
    row = list(self.rows_where(""rowid = ?"", [self.last_rowid]))[0]
          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^
IndexError: list index out of range
```

The issue is in this block: 

https://github.com/simonw/sqlite-utils/blob/2747257a3334d55e890b40ec58fada57ae8cfbfd/sqlite_utils/db.py#L2954-L2958

relevant locals are:

- `pk`: `'id'`
- `result.lastrowid`: `2`

What's most interesting is the comment `# self.last_rowid will be 0 if a ""INSERT OR IGNORE"" happened`, which doesn't seem to be the case here. ",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/554/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
632056825,MDU6SXNzdWU2MzIwNTY4MjU=,802,"""datasette plugins"" command is broken",9599,simonw,closed,0,,,,,1,2020-06-05T23:33:01Z,2020-06-05T23:46:43Z,2020-06-05T23:46:43Z,OWNER,,I broke it in https://github.com/simonw/datasette/commit/a7137dfe069e5fceca56f78631baebd4a6a19967 - and it turns out there was no test coverage so I didn't realize it was broken.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/802/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
273163905,MDU6SXNzdWUyNzMxNjM5MDU=,60,Rethink how metadata is generated and stored,9599,simonw,closed,0,,,2857392,Ship first public release,1,2017-11-11T18:01:28Z,2017-11-11T20:12:17Z,2017-11-11T20:12:16Z,OWNER,,"I broke the existing mechanism in 407795b61217205625f2d4e084afbf69f1db781b In order to get unit tests for the sanic app working.

I think i should ditch the build-metadata.json cache file entirely and calculate the SHA hashes on startup.

Not sure what to do about the table row counts.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/60/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
683805434,MDU6SXNzdWU2ODM4MDU0MzQ=,135,Code for finding SpatiaLite in the usual locations,9599,simonw,closed,0,,,,,3,2020-08-21T20:15:34Z,2022-02-05T00:04:26Z,2020-08-21T20:30:13Z,OWNER,,"I built this for `shapefile-to-sqlite` but it would be useful in `sqlite-utils` too:

https://github.com/simonw/shapefile-to-sqlite/blob/e754d0747ca2facf9a7433e2d5d15a6a37a9cf6e/shapefile_to_sqlite/utils.py#L16-L19

```python
SPATIALITE_PATHS = (
    ""/usr/lib/x86_64-linux-gnu/mod_spatialite.so"",
    ""/usr/local/lib/mod_spatialite.dylib"",
)
```

https://github.com/simonw/shapefile-to-sqlite/blob/e754d0747ca2facf9a7433e2d5d15a6a37a9cf6e/shapefile_to_sqlite/utils.py#L105-L109

```python
def find_spatialite():
    for path in SPATIALITE_PATHS:
        if os.path.exists(path):
            return path
    return None
```",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/135/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
688670158,MDU6SXNzdWU2ODg2NzAxNTg=,147,SQLITE_MAX_VARS maybe hard-coded too low,96218,simonwiles,open,0,,,,,7,2020-08-30T07:26:45Z,2021-02-15T21:27:55Z,,CONTRIBUTOR,,"I came across this while about to open an issue and PR against the documentation for `batch_size`, which is a bit incomplete.

As mentioned in #145, while:

> [`SQLITE_MAX_VARIABLE_NUMBER`](https://www.sqlite.org/limits.html#max_variable_number) ... defaults to 999 for SQLite versions prior to 3.32.0 (2020-05-22) or 32766 for SQLite versions after 3.32.0.

it is common that it is increased at compile time. Debian-based systems, for example, seem to ship with a version of sqlite compiled with SQLITE_MAX_VARIABLE_NUMBER set to 250,000, and I believe this is the case for homebrew installations too.

In working to understand what `batch_size` was actually doing and why, I realized that by setting `SQLITE_MAX_VARS` in `db.py` to match the value my sqlite was compiled with (I'm on Debian), I was able to decrease the time to `insert_all()` my test data set (~128k records across 7 tables) from ~26.5s to ~3.5s.  Given that this about .05% of my total dataset, this is time I am keen to save...

Unfortunately, it seems that `sqlite3` in the python standard library doesn't expose the `get_limit()` C API (even though `pysqlite` used to), so it's hard to know what value sqlite has been compiled with (note that this could mean, I suppose, that it's less than 999, and even hardcoding `SQLITE_MAX_VARS` to the conservative default might not be adequate.  It can also be lowered -- but not raised -- at runtime).  The best I could come up with is `echo """" | sqlite3 -cmd "".limits variable_number""` (only available in `sqlite >= 2015-05-07 (3.8.10)`).

Obviously this couldn't be relied upon in `sqlite_utils`, but I wonder what your opinion would be about exposing `SQLITE_MAX_VARS` as a user-configurable parameter (with suitable ""here be dragons"" warnings)?  I'm going to go ahead and monkey-patch it for my purposes in any event, but it seems like it might be worth considering.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/147/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
694136490,MDU6SXNzdWU2OTQxMzY0OTA=,15,Add a bunch of config examples,9599,simonw,open,0,,,,,1,2020-09-05T17:58:43Z,2020-09-18T23:17:39Z,,MEMBER,,I can bring these over from my personal Dogsheep.,197431109,dogsheep-beta,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-beta/issues/15/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
321624016,MDU6SXNzdWUzMjE2MjQwMTY=,252,/-/versions should report the FTS version supported by SQLite,9599,simonw,closed,0,,,,,0,2018-05-09T15:43:47Z,2018-05-11T13:19:52Z,2018-05-11T13:19:52Z,OWNER,,I can copy this function from `csvs-to-sqlite`: https://github.com/simonw/csvs-to-sqlite/blob/dccbf65b37bc9eed50e9edb80a42f257e93edb1f/csvs_to_sqlite/utils.py#L283-L293,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/252/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
663317875,MDU6SXNzdWU2NjMzMTc4NzU=,905,/database.db download should include content-length header,9599,simonw,closed,0,,,,,2,2020-07-21T21:23:48Z,2020-07-22T04:59:46Z,2020-07-22T04:52:45Z,OWNER,,I can do this by modifying this function: https://github.com/simonw/datasette/blob/02dc6298bdbfb1d63e0d2a39ff597b5fcc60e06b/datasette/utils/asgi.py#L248-L270,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/905/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
677250834,MDU6SXNzdWU2NzcyNTA4MzQ=,926,"datasette fixtures.db --get ""/fixtures.json""",9599,simonw,closed,0,,,,,2,2020-08-11T22:55:36Z,2020-08-12T00:26:17Z,2020-08-12T00:24:42Z,OWNER,,"I can expose ALL of Datasette's functionality on the command-line (without even running a web server) by adding `--get` and `--post` options to `datasette serve`.

    datasette fixtures.db --get ""/fixtures.json""

This would instantiate the Datasette ASGI app, run a fake request for `/fixtures.json` through it, dump the results out to standard output and quit.

A `--post` option could do the same for a POST request. Treating that as a stretch goal for the moment.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/926/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
444749373,MDU6SXNzdWU0NDQ3NDkzNzM=,469,publish commands should use new -i option,9599,simonw,closed,0,,,,,1,2019-05-16T04:31:40Z,2019-05-19T22:53:41Z,2019-05-19T22:53:41Z,OWNER,,"I can make this change only after releasing 0.28 - if I make the change earlier than that `publish heroku` etc will break because they will install the latest release of Datasette which will not understand the `-i` option.

This is a one-line fix: replace this:

https://github.com/simonw/datasette/blob/2ad9d15cd6901654e6801e2faa29e6fc08bae5fa/datasette/utils.py#L489

With this: (need to do it for other publishers too though)

```
quoted_files = "" "".join(
    [""-i {}"".format(shlex.quote(file_name)) for file_name in file_names]
)
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/469/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
488874815,MDU6SXNzdWU0ODg4NzQ4MTU=,5,Write tests that simulate the Twitter API,9599,simonw,open,0,,,,,1,2019-09-03T23:55:35Z,2019-09-03T23:56:28Z,,MEMBER,,I can use betamax for this: https://pypi.org/project/betamax/,206156866,twitter-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/5/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
432217625,MDU6SXNzdWU0MzIyMTc2MjU=,19,Incorrect help text for enable-fts command,9599,simonw,closed,0,,,4348046,1.0,0,2019-04-11T19:46:44Z,2019-05-25T00:44:31Z,2019-05-25T00:44:31Z,OWNER,,"I clearly copied-and-pasted this from the `tables` command without updating it:

https://github.com/simonw/sqlite-utils/blob/0b1af42ead3b3902347951180b3364ce1942da6e/sqlite_utils/cli.py#L216-L222",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/19/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
315142414,MDU6SXNzdWUzMTUxNDI0MTQ=,221,Allow plugins to add new cli sub commands ,9599,simonw,closed,0,,,,,3,2018-04-17T16:40:13Z,2021-01-04T20:12:14Z,2021-01-04T20:12:14Z,OWNER,,I could then test this out by having https://github.com/simonw/csvs-to-sqlite register itself as a plugin,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/221/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
476437213,MDU6SXNzdWU0NzY0MzcyMTM=,566,Unexpected keyword argument 'hidden',8330931,dvot197007,closed,0,,,,,1,2019-08-03T10:07:57Z,2019-08-03T16:13:36Z,2019-08-03T16:13:36Z,NONE,,"I couldn't get a test example running. I am running python 3.6.8 and tried both windows and windows subsystem for linux, getting the same error. My test.db was created by converting a five line csv file with csvs-to-sqlite. The csv file is:
col1, col2, col3
1,2,3
4,5,6
7,8,9
10,11,12
Here is the error message:
(myvenv) davido@DESKTOP-L29G79U:~/dot/datasette-eg$ datasette test.db                                                   Traceback (most recent call last):                                                                                        File ""/home/davido/dot/datasette-eg/myvenv/bin/datasette"", line 7, in <module>                                            from datasette.cli import cli                                                                                         File ""/home/davido/dot/datasette-eg/myvenv/lib/python3.6/site-packages/datasette/cli.py"", line 2, in <module>             import uvicorn                                                                                                        File ""/home/davido/dot/datasette-eg/myvenv/lib/python3.6/site-packages/uvicorn/__init__.py"", line 2, in <module>          from uvicorn.main import Server, main, run                                                                            File ""/home/davido/dot/datasette-eg/myvenv/lib/python3.6/site-packages/uvicorn/main.py"", line 224, in <module>            headers: typing.List[str],                                                                                            File ""/home/davido/dot/datasette-eg/myvenv/lib/python3.6/site-packages/click/decorators.py"", line 170, in decorator       _param_memo(f, OptionClass(param_decls, **attrs))                                                                     File ""/home/davido/dot/datasette-eg/myvenv/lib/python3.6/site-packages/click/core.py"", line 1430, in __init__             Parameter.__init__(self, param_decls, type=type, **attrs)                                                           TypeError: __init__() got an unexpected keyword argument 'hidden'  

Thanks.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/566/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
733560417,MDU6SXNzdWU3MzM1NjA0MTc=,1073,Remove load_template plugin hook,9599,simonw,closed,0,,,6026070,0.51,1,2020-10-30T22:51:52Z,2020-10-31T16:22:00Z,2020-10-31T16:22:00Z,OWNER,,"I couldn't get it working correctly with async (necessary for include/extend to function), and on deeper investigation it appears that I can build something equivalent to what I wanted using the existing `prepare_jinja2_environment` hook.

> I'm going to remove the `load_template` plugin hook and see if it's possible to build the edit templates extension against `prepare_jinja2_environment` instead.
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1072#issuecomment-719833744_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1073/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
440222719,MDU6SXNzdWU0NDAyMjI3MTk=,448,_facet_array should work against views,9599,simonw,closed,0,,,3268330,Datasette 1.0,12,2019-05-03T21:08:04Z,2021-11-16T01:32:05Z,2021-11-16T01:19:40Z,OWNER,,"I created this view: https://json-view-facet-bug-demo-j7hipcg4aq-uc.a.run.app/russian-ads-8dbda00/ads_with_targets

```
CREATE VIEW ads_with_targets as select ads.*, json_group_array(targets.name) as target_names from ads
  join ad_targets on ad_targets.ad_id = ads.id
  join targets on ad_targets.target_id = targets.id
  group by ad_targets.ad_id
```

When I try to apply faceting by array it appears to work at first: https://json-view-facet-bug-demo-j7hipcg4aq-uc.a.run.app/russian-ads/ads_with_targets?_facet_array=target_names

But actually it's doing the wrong thing - the SQL for the facets uses rowid, but rowid is not present on views at all! These results are incorrect, and clicking to select a facet will fail to produce any rows: https://json-view-facet-bug-demo-j7hipcg4aq-uc.a.run.app/russian-ads/ads_with_targets?_facet_array=target_names&target_names__arraycontains=people_who_match%3Ainterests%3AAfrican-American+Civil+Rights+Movement+%281954%E2%80%9468%29

Here's the SQL it should be using when you select a facet (note that it does not use a rowid):

https://json-view-facet-bug-demo-j7hipcg4aq-uc.a.run.app/russian-ads?sql=select+*+from+ads_with_targets+where+id+in+%28%0D%0A++++++++++++select+ads_with_targets.id+from+ads_with_targets%2C+json_each%28ads_with_targets.target_names%29+j%0D%0A++++++++++++where+j.value+%3D+%3Ap0%0D%0A++++++++%29+limit+101&p0=people_who_match%3Ainterests%3ABlack+%28Color%29

So we need to do something a lot smarter here. I'm not sure what the fix will look like, or even if it's feasible given that views don't have a rowid to hook into so the JSON faceting SQL may have to be completely rewritten.

```
datasette publish cloudrun \
    russian-ads.db \
    --name json-view-facet-bug-demo \
    --branch master \
    --extra-options ""--config sql_time_limit_ms:5000 --config facet_time_limit_ms:5000""
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/448/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1665053646,I_kwDOBm6k_c5jPrPO,2059,"""Deceptive site ahead"" alert on Heroku deployment",1186275,mtdukes,open,0,,,,,1,2023-04-12T18:34:51Z,2023-04-13T01:13:01Z,,NONE,,"I deployed a fairly basic instance of Datasette (`datasette-auth-passwords` is the only plugin) using Heroku. The deployed URL now gives a ""Deceptive site ahead"" warning to users.

Is there way around this? Maybe a way to add ownership verification [through Google's search console](https://search.google.com/search-console/welcome)? ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2059/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
521275281,MDU6SXNzdWU1MjEyNzUyODE=,13,Set up a live demo Datasette instance,9599,simonw,closed,0,,,5225818,1.0,9,2019-11-12T01:27:02Z,2020-03-24T00:03:26Z,2020-03-24T00:03:25Z,MEMBER,,"I deployed https://github-to-sqlite-releases-j7hipcg4aq-uc.a.run.app/ by running this:
```
#!/bin/bash
# Fetch repos for simonw and dogsheep
github-to-sqlite repos github.db simonw dogsheep -a auth.json

# Fetch releases for the repos tagged 'datasette-io'
sqlite-utils github.db ""
select full_name from repos where rowid in (
    select repos.rowid from repos, json_each(repos.topics) j
    where j.value = 'datasette-io'
)"" --csv --no-headers | while read repo;
    do github-to-sqlite releases \
            github.db $(echo $repo | tr -d '\r') \
            -a auth.json;
        sleep 2;
    done;
```
And then deploying using this:
```
$ datasette publish cloudrun github.db \
  --title ""github-to-sqlite releases demo"" \
  --about_url=""https://github.com/simonw/github-to-sqlite"" \
  --about='github-to-sqlite' \
  --install=datasette-render-markdown \
  --install=datasette-json-html \
  --service=github-to-sqlite-releases
```
This should happen automatically for every release. I can run it once a day in Circle CI to keep the demo database up-to-date.",207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/13/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
714449879,MDU6SXNzdWU3MTQ0NDk4Nzk=,992,"Change ""--config foo:bar"" to ""--setting foo bar""",9599,simonw,closed,0,,,6055094,Datasette 0.52,6,2020-10-05T01:27:45Z,2020-11-24T20:01:54Z,2020-11-24T20:01:54Z,OWNER,,I designed the config format before I had a good feel for CLI design using Click. `--config max_page_size 2000` is better than `--config max_page_size:2000`.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/992/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
772438273,MDU6SXNzdWU3NzI0MzgyNzM=,1157,Use time.perf_counter() instead of time.time() to measure performance,9599,simonw,closed,0,,,6346396,Datasette 0.54,1,2020-12-21T20:21:41Z,2021-01-24T21:20:42Z,2020-12-21T21:49:20Z,OWNER,,"I do that in a bunch of places: https://ripgrep.datasette.io/-/ripgrep?pattern=time%28%29&literal=on&glob=datasette%2F%2A%2A

https://docs.python.org/3/library/time.html#time.perf_counter

> `time.``perf_counter`() → float
> 
> Return the value (in fractional seconds) of a performance counter, i.e. a clock with the highest available resolution to measure a short duration. It does include time elapsed during sleep and is system-wide. The reference point of the returned value is undefined, so that only the difference between the results of consecutive calls is valid.
> 
> _New in version 3.3._",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1157/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
839008371,MDU6SXNzdWU4MzkwMDgzNzE=,1274,Might there be some way to comment metadata.json?,192568,mroswell,closed,0,,,,,2,2021-03-23T18:33:00Z,2021-03-23T20:14:54Z,2021-03-23T20:14:54Z,CONTRIBUTOR,,"I don't know what license to use... Would be nice to be able to add a comment regarding that uncertainty in my metadata.json file

I like laktak's little video comment in favor of Human json (Hjson)
https://stackoverflow.com/questions/244777/can-comments-be-used-in-json

Hmmm... one of the commenters there said comments are allowed in yaml... so that's a good argument for yaml.

Anyhow, just came to mind, and thought I'd mention it here. Looks like https://hjson.github.io/ has the details.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1274/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
470856782,MDU6SXNzdWU0NzA4NTY3ODI=,6,Break up records into different tables for each type,9599,simonw,closed,0,,,,,1,2019-07-22T01:54:59Z,2019-07-22T03:28:55Z,2019-07-22T03:28:50Z,MEMBER,,"I don't think there's much benefit to having all of the different record types stored in the same enormous table. Here's what I get when I use `_facet=type`:

<img width=""358"" alt=""hello2__records__2_672_233_rows"" src=""https://user-images.githubusercontent.com/9599/61601118-e2f54d00-abe8-11e9-8bf6-3df2ef969112.png"">

I'm going to try splitting these up into separate tables - so `HKQuantityTypeIdentifierBodyMassIndex` becomes a table called `rBodyMassIndex` - and see if that's nicer to work with.",197882382,healthkit-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/6/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
748372469,MDU6SXNzdWU3NDgzNzI0Njk=,9,ParseError: undefined entity &scaron;,4028322,mkorosec,closed,0,,,,,1,2020-11-22T23:04:35Z,2021-02-11T22:10:55Z,2021-02-11T22:10:55Z,CONTRIBUTOR,,"I encountered a parse error if the enex file contained &scaron; or &nbsp; 

Run command: 
evernote-to-sqlite enex evernote.db evernote.enex

```
Traceback (most recent call last):
...
  File ""evernote_to_sqlite/cli.py"", line 31, in enex
    save_note(db, note)
  File ""evernote_to_sqlite/utils.py"", line 35, in save_note
    content = ET.tostring(ET.fromstring(content_xml)).decode(""utf-8"")
  File ""/usr/lib/python3.8/xml/etree/ElementTree.py"", line 1320, in XML
    parser.feed(text)
xml.etree.ElementTree.ParseError: undefined entity &scaron;: line 3, column 35
```

Workaround:
```
sed -i 's/&scaron;//g' evernote.enex
sed -i 's/&nbsp;//g' evernote.enex
```",303218369,evernote-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/9/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1178456794,I_kwDOCGYnMM5GPdLa,418,Add generated files to .gitignore,25778,eyeseast,closed,0,,,,,0,2022-03-23T17:48:12Z,2022-03-24T21:01:44Z,2022-03-24T21:01:44Z,CONTRIBUTOR,,"I end up with these in my local directory:

	.hypothesis/
	Pipfile
	Pipfile.lock
	pyproject.toml

Might as well gitignore them.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/418/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1840417903,I_kwDOBm6k_c5tsoxv,2131,Refactor code that supports templates_considered comment,9599,simonw,open,0,,,3268330,Datasette 1.0,1,2023-08-08T01:28:36Z,2023-08-09T15:27:41Z,,OWNER,,"I ended up duplicating it here: https://github.com/simonw/datasette/blob/7532feb424b1dce614351e21b2265c04f9669fe2/datasette/views/database.py#L164-L167

I think it should move to `datasette.render_template()` - and maybe have a renamed template variable too.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2131/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1423069384,I_kwDOCGYnMM5U0lDI,504,"db.close() method, calling db.conn.close()",9599,simonw,closed,0,,,,,1,2022-10-25T20:50:50Z,2022-10-25T21:00:29Z,2022-10-25T20:57:47Z,OWNER,,"I ended up needing to use `db.conn.close()` to fix this issue:
- #503

I think `.close()` should be a method on `Database` itself.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/504/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
787104850,MDU6SXNzdWU3ODcxMDQ4NTA=,1192,Form Plugin for in-depth Datasette Querying,1024355,tomershvueli,open,0,,,,,0,2021-01-15T18:24:50Z,2021-01-15T18:24:50Z,,NONE,,I envision a sort of easy-to-build form plugin that would be able to map a user's inputs to different fields/columns in a Datasette database. ,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1192/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1865232341,I_kwDOBm6k_c5vLS_V,2153,Datasette --get --actor option,9599,simonw,closed,0,,,,,5,2023-08-24T14:00:03Z,2023-08-28T20:19:15Z,2023-08-28T20:15:53Z,OWNER,,"I experimented with a prototype of this here:
- https://github.com/simonw/datasette/issues/2102#issuecomment-1691037971_

Which lets me run requests as if they belonged to a specific actor like this:

```bash
datasette fixtures.db --get '/fixtures/facetable.json' --actor '{
    ""_r"": {
        ""r"": {
            ""fixtures"": {
                ""facetable"": [
                    ""vt""
                ]
            }
        }
    },
    ""a"": ""user""
}'
```

Really useful for testing actors an `_r` options. Is this worth adding as a feature?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2153/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1352946135,I_kwDOCGYnMM5QpFHX,472,Reuse the locals/globals fix from --functions for other code accepting options,9599,simonw,closed,0,,,8355157,3.29,2,2022-08-27T05:12:05Z,2022-08-27T05:20:12Z,2022-08-27T05:20:12Z,OWNER,,"I figured out a workaround for the ugly `global x` hack here:
- https://github.com/simonw/sqlite-utils/issues/471#issuecomment-1229120653",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/472/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
811054000,MDU6SXNzdWU4MTEwNTQwMDA=,1230,"Vega charts are plotted only for rows on the visible page, cluster maps only for rows in the remaining pages",7107523,Kabouik,open,0,,,,,1,2021-02-18T12:27:02Z,2021-02-18T15:22:15Z,,NONE,,"I filtered a data set on some criteria and obtain 265 results, split over three pages (100, 100, 65), and reazlized that Vega plots are only applied to the results displayed on the current page, instead of the whole filtered data, _e.g._, 100 on page 1, 100 on page 2, 65 on page 3. Is there a way to force the graphs to consider all results instead of just the page, considering that pages rarely represent sensible information?

Likewise, while the cluster map does show all results on the first page, if you go to next pages, it will show all remaining results except the previous page(s), _e.g._, 265 on page 1, 165 on page 2, 65 on page 3.

In both cases, I don't see many situations where one would like to represent the data this way, and it might even lead to interpretation errors when viewing the data. Am I missing some cases where this would be best? Perhaps a clickable option to subset visual representations according visible pages _vs._ display all search results would do?

[Edit] Oh, I just saw the ""Load all"" button under the cluster map as well as the [setting to alter the max number or results](https://docs.datasette.io/en/stable/settings.html#max-returned-rows). So I guess this issue only is about the Vega charts.",107914493,datasette,issue,,,,,
675839512,MDU6SXNzdWU2NzU4Mzk1MTI=,132,Features for enabling and disabling WAL mode,9599,simonw,closed,0,,,,,5,2020-08-10T03:25:44Z,2020-08-10T18:59:35Z,2020-08-10T18:59:35Z,OWNER,,I finally figured out how to enable WAL - turns out it's a property of the database file itself: https://github.com/simonw/til/blob/master/sqlite/enabling-wal-mode.md,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/132/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
613422636,MDU6SXNzdWU2MTM0MjI2MzY=,760,Way of seeing full schema for a database,9599,simonw,open,0,,,,,3,2020-05-06T15:46:08Z,2020-05-06T23:49:06Z,,OWNER,,"I find myself wanting to quickly figure out all of the BLOB columns in a database.

A `/-/schema` page showing the full schema (actually since it's per-database probably `/dbname/-/schema` or `/-/schema/dbname`) would be really handy.

It would need to be carefully constructed from various queries against `sqlite_master` - just doing `select * from sqlite_master where type='table'` isn't quite enough because I also want to show indexes, triggers etc.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/760/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
991191951,MDU6SXNzdWU5OTExOTE5NTE=,1464,clean checkout & clean environment has test failures,51016,ctb,open,0,,,,,6,2021-09-08T14:16:23Z,2021-09-13T22:17:17Z,,CONTRIBUTOR,,"I followed the instructions [here](https://docs.datasette.io/en/stable/contributing.html#setting-up-a-development-environment), and even after running `python update-docs-help.py` I get the following failed tests -- any thoughts?

```
FAILED tests/test_api.py::test_searchable[/fixtures/searchable.json?_search=te*+AND+do*&_searchmode=raw-expected_rows3]
FAILED tests/test_api.py::test_searchmode[table_metadata1-_search=te*+AND+do*-expected_rows1]
FAILED tests/test_api.py::test_searchmode[table_metadata2-_search=te*+AND+do*&_searchmode=raw-expected_rows2]
```

This is with python 3.9.7 and lots of other packages, as in attached environment listing from `conda list`.
[conda-installed.txt](https://github.com/simonw/datasette/files/7129487/conda-installed.txt)
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1464/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
830283447,MDU6SXNzdWU4MzAyODM0NDc=,34,bucket name,6213,dsisnero,open,0,,,,,0,2021-03-12T16:40:57Z,2021-03-12T16:40:57Z,,NONE,,I followed the instructions to setup credentials but I am getting a invalid bucket name. Can you put a sample auth.json file in the base that shows the correct format for this?  Thanks,256834907,dogsheep-photos,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-photos/issues/34/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1497909798,I_kwDOBm6k_c5ZSEom,1958,datasette --root running in Docker doesn't reliably show the magic URL,11729897,davidhaley,closed,0,,,,,11,2022-12-13T16:29:13Z,2022-12-16T00:59:12Z,2022-12-16T00:55:19Z,NONE,,"I followed these steps:

`docker run datasetteproject/datasette  pip install datasette-upload-csvs`

`docker commit $(docker ps -lq) datasette-with-plugins`

`docker run -p 8001:8001 -v $(pwd):/mnt   datasette-with-plugins   datasette --root -p 8001 -h 0.0.0.0`

Visited: http://127.0.0.1:8001/-/plugins

![image](https://user-images.githubusercontent.com/11729897/207392071-d939cd5e-1d96-4e11-b0be-dc06dd207866.png)


Visited: http://localhost:8001/-/upload-csvs

![image](https://user-images.githubusercontent.com/11729897/207389241-3e96ca66-ca74-4a16-8b7d-4427ee862c5e.png)

I may have missed a step?

Thank you.

---

Ubuntu 22.04.1 LTS",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1958/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1174655187,I_kwDOBm6k_c5GA9DT,1671,Filters fail to work correctly against calculated numeric columns returned by SQL views because type affinity rules do not apply,9308268,rayvoelker,open,0,,,,,8,2022-03-20T19:17:24Z,2022-03-22T17:43:12Z,,NONE,,"I found a strange behavior, and I'm not sure if it's related to views and boolean values perhaps, or if there's something else weird going on here, but I'll provide an example that may help show what I'm seeing happen.

```bash
#!/bin/bash

echo ""\""id\"",\""expiration_date\""
0,2018-01-04
1,2019-01-05
2,2020-01-06
3,2021-01-07
4,2022-01-08
5,2023-01-09
6,2024-01-10
7,2025-01-11
8,2026-01-12
9,2027-01-13
"" > test.csv
csvs-to-sqlite test.csv test.db
sqlite-utils create-view --replace test.db test_view ""select id, expiration_date, case when julianday('NOW') >= julianday(expiration_date) then 1 else 0 end as has_expired FROM test""
```

```bash
datasette test.db
```
![image](https://user-images.githubusercontent.com/9308268/159178745-9c6152f7-eac6-4bf9-bef5-a2d63d3ee13f.png)

![image](https://user-images.githubusercontent.com/9308268/159178824-c8952137-270c-42a4-ad1c-f6ad2c51e499.png)

![image](https://user-images.githubusercontent.com/9308268/159178877-23e00b36-443a-43ef-83e5-e0bdddd3fdcd.png)

![image](https://user-images.githubusercontent.com/9308268/159178918-65922cc7-2514-4735-a72d-4904b99976d4.png)

Thanks again and let me know if you want me to provide anything else!",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1671/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1695428235,I_kwDOCGYnMM5lDi6L,538,`table.upsert_all` fails to write rows when `not_null` is present,1231935,xavdid,closed,0,,,,,9,2023-05-04T07:30:38Z,2023-05-08T20:06:35Z,2023-05-08T19:27:02Z,NONE,,"I found an odd bug today, where calls to `table.upsert_all` don't write rows if you include the `not_null` kwarg.

## Repro Example

```py
from sqlite_utils import Database

db = Database(""upsert-test.db"")

db[""comments""].upsert_all(
    [{""id"": 1, ""name"": ""david""}],
    pk=""id"",
    not_null=[""name""],
)

assert list(db[""comments""].rows) # err!
```

The schema is correctly created:

```sql
CREATE TABLE [comments] (
   [id] INTEGER PRIMARY KEY,
   [name] TEXT NOT NULL
)
```

But no rows are created. Removing either the `not_null` kwargs works as expected, as does an `insert_all` call.

## Version Info

- Python: `3.11.0`
- sqlite-utils: `3.30`
- sqlite: `3.39.5 2022-10-14`",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/538/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
611252244,MDU6SXNzdWU2MTEyNTIyNDQ=,750,Add notlike table filter,9599,simonw,closed,0,,,,,3,2020-05-02T18:54:36Z,2020-05-02T19:10:44Z,2020-05-02T19:10:44Z,OWNER,,"I found myself wanting that for applying the opposite of this: https://github-to-sqlite.dogsheep.net/github/dependent_repos?dependent__like=%25simonw%2F%25&_sort_desc=dependent_stars

",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/750/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1777548699,I_kwDOCGYnMM5p8z2b,561,`--stop-after` option for `insert` and `upsert` commands,9599,simonw,closed,0,,,,,1,2023-06-27T18:44:15Z,2023-06-27T18:50:09Z,2023-06-27T18:50:08Z,OWNER,,I found myself wanting to insert rows from a 849MB CSV file without processing the whole thing: https://huggingface.co/datasets/jerpint-org/HackAPrompt-Playground-Submissions/tree/main,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/561/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
836273891,MDU6SXNzdWU4MzYyNzM4OTE=,1266,Documentation for Response.asgi_send(send) method,9599,simonw,closed,0,,,,,1,2021-03-19T18:52:49Z,2021-03-20T21:35:00Z,2021-03-20T21:32:28Z,OWNER,,"I found myself wanting to use this method for https://github.com/simonw/datasette-auth-passwords/issues/15 - but it's not documented. It should be documented.

https://github.com/simonw/datasette/blob/8e18c7943181f228ce5ebcea48deb59ce50bee1f/datasette/utils/asgi.py#L320-L340",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1266/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1910269679,I_kwDOBm6k_c5x3Gbv,2196,Discord invite link returns 401,1892194,Olshansk,closed,0,,,,,2,2023-09-24T15:16:54Z,2023-10-13T00:07:08Z,2023-10-12T21:54:54Z,NONE,,"I found the link to the datasette discord channel via [this query](https://github.com/search?q=repo%3Asimonw%2Fdatasette%20discord&type=code).

The following video should be self explanatory:

https://github.com/simonw/datasette/assets/1892194/8cd33e88-bcaa-41f3-9818-ab4d589c3f02


Link for reference: https://discord.com/invite/ktd74dm5mw",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2196/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
951581763,MDU6SXNzdWU5NTE1ODE3NjM=,298,Read lines with JSON object,2172260,qqilihq,closed,0,,,,,2,2021-07-23T13:28:52Z,2021-08-03T06:50:47Z,2021-08-02T21:55:16Z,NONE,,"I found this posted on HN a while ago and love it -- thank you!

As a minor improvement, it would be great to have the ability to parse a file with line-separated JSON objects. Currently the parser obviously requires an array wrapping all these objects.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/298/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
274161964,MDU6SXNzdWUyNzQxNjE5NjQ=,101,TemplateAssertionError: no filter named 'tojson',450244,eaubin,closed,0,,,,,1,2017-11-15T13:47:32Z,2017-11-15T13:48:55Z,2017-11-15T13:48:55Z,NONE,,"I get an exception clicking on the table link: 

```
2017-11-15 08:40:10 - (sanic)[ERROR]: Traceback (most recent call last):
  File ""/Users/e/anaconda3-4.2.0/lib/python3.5/site-packages/sanic/app.py"", line 503, in handle_request
    response = await response
  File ""/Users/e/anaconda3-4.2.0/lib/python3.5/site-packages/datasette/app.py"", line 155, in get
    return await self.view_get(request, name, hash, **kwargs)
  File ""/Users/e/anaconda3-4.2.0/lib/python3.5/site-packages/datasette/app.py"", line 219, in view_get
    **context,
  File ""/Users/e/anaconda3-4.2.0/lib/python3.5/site-packages/sanic_jinja2/__init__.py"", line 84, in render
    return html(self.render_string(template, request, **context))
  File ""/Users/e/anaconda3-4.2.0/lib/python3.5/site-packages/sanic_jinja2/__init__.py"", line 81, in render_string
    return self.env.get_template(template).render(**context)
  File ""/Users/e/anaconda3-4.2.0/lib/python3.5/site-packages/jinja2/environment.py"", line 812, in get_template
    return self._load_template(name, self.make_globals(globals))
  File ""/Users/e/anaconda3-4.2.0/lib/python3.5/site-packages/jinja2/environment.py"", line 786, in _load_template
    template = self.loader.load(self, name, globals)
  File ""/Users/e/anaconda3-4.2.0/lib/python3.5/site-packages/jinja2/loaders.py"", line 125, in load
    code = environment.compile(source, name, filename)
  File ""/Users/e/anaconda3-4.2.0/lib/python3.5/site-packages/jinja2/environment.py"", line 565, in compile
    self.handle_exception(exc_info, source_hint=source_hint)
  File ""/Users/e/anaconda3-4.2.0/lib/python3.5/site-packages/jinja2/environment.py"", line 754, in handle_exception
    reraise(exc_type, exc_value, tb)
  File ""/Users/e/anaconda3-4.2.0/lib/python3.5/site-packages/jinja2/_compat.py"", line 37, in reraise
    raise value.with_traceback(tb)
  File ""/Users/e/anaconda3-4.2.0/lib/python3.5/site-packages/datasette/templates/table.html"", line 29, in template
    <pre>params = {{ query.params|tojson(4) }}</pre>
  File ""/Users/e/anaconda3-4.2.0/lib/python3.5/site-packages/jinja2/environment.py"", line 515, in _generate
    return generate(source, self, name, filename, defer_init=defer_init)
  File ""/Users/e/anaconda3-4.2.0/lib/python3.5/site-packages/jinja2/compiler.py"", line 62, in generate
    generator.visit(node)
  File ""/Users/e/anaconda3-4.2.0/lib/python3.5/site-packages/jinja2/visitor.py"", line 38, in visit
    return f(node, *args, **kwargs)
  File ""/Users/e/anaconda3-4.2.0/lib/python3.5/site-packages/jinja2/compiler.py"", line 849, in visit_Template
    self.blockvisit(block.body, block_frame)
  File ""/Users/e/anaconda3-4.2.0/lib/python3.5/site-packages/jinja2/compiler.py"", line 492, in blockvisit
    self.visit(node, frame)
  File ""/Users/e/anaconda3-4.2.0/lib/python3.5/site-packages/jinja2/visitor.py"", line 38, in visit
    return f(node, *args, **kwargs)
  File ""/Users/e/anaconda3-4.2.0/lib/python3.5/site-packages/jinja2/compiler.py"", line 1172, in visit_If
    self.blockvisit(node.body, if_frame)
  File ""/Users/e/anaconda3-4.2.0/lib/python3.5/site-packages/jinja2/compiler.py"", line 492, in blockvisit
    self.visit(node, frame)
  File ""/Users/e/anaconda3-4.2.0/lib/python3.5/site-packages/jinja2/visitor.py"", line 38, in visit
    return f(node, *args, **kwargs)
  File ""/Users/e/anaconda3-4.2.0/lib/python3.5/site-packages/jinja2/compiler.py"", line 1353, in visit_Output
    self.visit(argument, frame)
  File ""/Users/e/anaconda3-4.2.0/lib/python3.5/site-packages/jinja2/visitor.py"", line 38, in visit
    return f(node, *args, **kwargs)
  File ""/Users/e/anaconda3-4.2.0/lib/python3.5/site-packages/jinja2/compiler.py"", line 1565, in visit_Filter
    self.fail('no filter named %r' % node.name, node.lineno)
  File ""/Users/e/anaconda3-4.2.0/lib/python3.5/site-packages/jinja2/compiler.py"", line 427, in fail
    raise TemplateAssertionError(msg, lineno, self.name, self.filename)
jinja2.exceptions.TemplateAssertionError: no filter named 'tojson'
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/101/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1871935751,I_kwDOD079W85vk3kH,40, ImportError: cannot import name 'formatargspec' from 'inspect',36752421,hosslikw,closed,0,,,,,0,2023-08-29T15:36:31Z,2023-08-31T03:18:07Z,2023-08-31T03:18:06Z,NONE,,"I get the following error when running ""pip3 install dogsheep-photos""
""  from inspect import ismethod, isclass, formatargspec
      ImportError: cannot import name 'formatargspec' from 'inspect' (/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/inspect.py). Did you mean: 'formatargvalues'?""
    
Python 3.12.0rc1
sqlite 3.43.0
datasette, version 0.64.3",256834907,dogsheep-photos,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-photos/issues/40/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1524076587,I_kwDOBm6k_c5a15Ar,1979,More useful error message if enable_load_extension is not available,9599,simonw,closed,0,,,,,5,2023-01-07T19:13:19Z,2023-01-08T00:21:23Z,2023-01-08T00:21:23Z,OWNER,,"I get this from:

    datasette --load-extension spatialite --get /-/versions.json

```
  File ""/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/datasette/app.py"", line 614, in _prepare_connection
    conn.enable_load_extension(True)
AttributeError: 'sqlite3.Connection' object has no attribute 'enable_load_extension'
```
It would be useful if Datasette caught this error and output something more friendly.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1979/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1528448642,I_kwDOBm6k_c5bGkaC,1985,Don't let Datasette(path) without a list cause weird errors,9599,simonw,closed,0,,,,,1,2023-01-11T05:17:44Z,2023-01-11T18:25:04Z,2023-01-11T18:25:04Z,OWNER,,"I got a confusing `sqlite3.OperationalError: disk I/O error` error in my tests, it turned out it was because this:
```python
ds = Datasette(path)
```
Should have been this:
```python
ds = Datasette([path])
```

_Originally posted by @simonw in https://github.com/simonw/datasette-faiss/issues/1#issuecomment-1378252673_
      ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1985/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1091257796,I_kwDOBm6k_c5BC0XE,1584,give error with recursive sql,58088336,tunguyenatwork,open,0,,,,,0,2021-12-30T18:53:16Z,2021-12-30T18:53:16Z,,NONE,,"I got an error ""near ""WITH"": syntax error"" after I upgraded to version 0.59 from 0.52.4.    This error is related to recursive sql.  It works great on the previous version but it failed after upgraded.  Below is an example of sql:

WITH RECURSIVE manager_of(position, super_position) AS (SELECT position, case ifnull(INDIRECT_SUPER_POSITION,'') when '' then super_position else INDIRECT_SUPER_POSITION end as SUPER_POSITION FROM position where super_position<>'SGV000000001' and super_position!=''  and position <> super_position),chain_manager_of_position(position, level) AS (SELECT super_position, 1 as level FROM manager_of WHERE super_position!='' and (position=:pos or position in (Select position from employee where employee=:ein)) UNION ALL SELECT super_position, level+1 as level FROM manager_of JOIN chain_manager_of_position USING(position)) SELECT * FROM chain_manager_of_position left join employee using(position)  where  employee is not NULL order by level limit 1",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1584/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1499081664,I_kwDOBm6k_c5ZWivA,1959,Refactor test suite to use mostly `async def` tests,9599,simonw,closed,0,,,,,9,2022-12-15T21:02:54Z,2022-12-17T21:49:37Z,2022-12-17T21:49:36Z,OWNER,,"I got blocked working on this issue due to weird and hard-to-debug test suite problems:

- #1955 

The test suite has needed a major upgrade for several years now. It has a LOT of `def test_...` synchronous functions that could be upgraded to `async def` for better performance and less test complexity - I've used the new `async def` pattern in plugins and new tests for a couple of years now.

Hopefully I can get more of the tests to use in-memory named databases too, ideally so I can fix this consistent problem:

- #1843",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1959/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1856075668,I_kwDOCGYnMM5uoXeU,586,.transform() fails to drop column if table is part of a view,9599,simonw,open,0,,,,,3,2023-08-18T05:25:22Z,2023-08-18T06:13:47Z,,OWNER,,"I got this error trying to drop a column from a table that was part of a SQL view:

> error in view plugins: no such table: main.pypi_releases

Upon further investigation I found that this pattern seemed to fix it:
```python
def transform_the_table(conn):
    # Run this in a transaction:
    with conn:
        # We have to read all the views first, because we need to drop and recreate them
        db = sqlite_utils.Database(conn)
        views = {v.name: v.schema for v in db.views if table.lower() in v.schema.lower()}
        for view in views.keys():
            db[view].drop()
        db[table].transform(
            types=types,
            rename=rename,
            drop=drop,
            column_order=[p[0] for p in order_pairs],
        )
        # Now recreate the views
        for name, schema in views.items():
            db.create_view(name, schema)
```
So grab a copy of any view that might reference this table, start a transaction, drop those views, run the transform, recreate the views again.

> I wonder if this should become an option in `sqlite-utils`? Maybe a `recreate_views=True` argument for `table.tranform(...)`? Should it be opt-in or opt-out?

_Originally posted by @simonw in https://github.com/simonw/datasette-edit-schema/issues/35#issuecomment-1683370548_
            ",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/586/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1516644980,I_kwDOCGYnMM5aZip0,520,rows_from_file() raises confusing error if file-like object is not in binary mode,9599,simonw,closed,0,,,,,3,2023-01-02T19:00:14Z,2023-05-08T22:08:07Z,2023-05-08T22:08:07Z,OWNER,,"I got this error:

```
  File ""/Users/simon/Dropbox/Development/openai-to-sqlite/openai_to_sqlite/cli.py"", line 27, in embeddings
    rows, _ = rows_from_file(input)
              ^^^^^^^^^^^^^^^^^^^^^
  File ""/Users/simon/.local/share/virtualenvs/openai-to-sqlite-jt4obeb2/lib/python3.11/site-packages/sqlite_utils/utils.py"", line 305, in rows_from_file
    first_bytes = buffered.peek(2048).strip()
                  ^^^^^^^^^^^^^^^^^^^
```
From this code:
```python

@cli.command()
@click.argument(
    ""db_path"",
    type=click.Path(file_okay=True, dir_okay=False, allow_dash=False),
)
@click.option(
    ""-i"",
    ""--input"",
    type=click.File(""r""),
    default=""-"",
)
def embeddings(db_path, input):
    ""Store embeddings for one or more text documents""
    click.echo(""Here is some output"")
    db = sqlite_utils.Database(db_path)
    rows, _ = rows_from_file(input)
    print(list(rows))
```
The error went away when I changed it to `type=click.File(""rb"")`.

This should either be called out in the documentation or `rows_from_file()` should be fixed to handle text-mode files in addition to binary files.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/520/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
683804172,MDU6SXNzdWU2ODM4MDQxNzI=,134,--load-extension option for sqlite-utils query,9599,simonw,closed,0,,,,,4,2020-08-21T20:12:42Z,2020-08-21T21:06:26Z,2020-08-21T20:54:19Z,OWNER,,"I got this error:
```
% sqlite-utils calands.db 'create table superunits_with_maps_view_concrete as select * from superunits_with_maps_view'
Traceback (most recent call last):
...
    cursor = db.conn.execute(sql, dict(param))
sqlite3.OperationalError: no such function: AsGeoJSON
```
A `--load-extension=/usr/local/lib/mod_spatialite.dylib` option (imitating the same option for Datasette) would help.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/134/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
450032134,MDU6SXNzdWU0NTAwMzIxMzQ=,495,facet_m2m gets confused by multiple relationships,9599,simonw,open,0,,,,,2,2019-05-29T21:37:28Z,2020-12-17T05:08:22Z,,OWNER,,"I got this for a database I was playing with:

<img width=""794"" alt=""hackathon__hacks_project__24_rows"" src=""https://user-images.githubusercontent.com/9599/58593179-12a65b00-821f-11e9-8621-ec0a62d41b46.png"">

I think this is because of these three tables:

<img width=""733"" alt=""hackathon__select___from_sqlite_master_"" src=""https://user-images.githubusercontent.com/9599/58593271-471a1700-821f-11e9-8d23-b27bfc3dec73.png"">

",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/495/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
741268956,MDU6SXNzdWU3NDEyNjg5NTY=,1088,OperationalError('interrupted') can 500 on row page,9599,simonw,closed,0,,,6055094,Datasette 0.52,3,2020-11-12T04:29:55Z,2020-11-28T23:28:35Z,2020-11-12T04:36:52Z,OWNER,,"I got this on my (private) https://dogsheep.simonwillison.net/twitter/tweets/1188612004572880896 page:

<img width=""1059"" alt=""Error_500"" src=""https://user-images.githubusercontent.com/9599/98895899-a6440980-245c-11eb-98e8-fb97d10ec2fd.png"">
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1088/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
638241779,MDU6SXNzdWU2MzgyNDE3Nzk=,846,"""Too many open files"" error running tests",9599,simonw,closed,0,,,,,6,2020-06-13T22:11:40Z,2020-06-14T00:26:31Z,2020-06-14T00:26:31Z,OWNER,,"I got this on my laptop:
```pytest
...
/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.7/site-packages/jinja2/loaders.py:171: in get_source
    f = open_if_exists(filename)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

filename = '/Users/simon/Dropbox/Development/datasette/datasette/templates/400.html', mode = 'rb'

    def open_if_exists(filename, mode='rb'):
        """"""Returns a file descriptor for the filename if that file exists,
        otherwise `None`.
        """"""
        try:
>           return open(filename, mode)
E           OSError: [Errno 24] Too many open files: '/Users/simon/Dropbox/Development/datasette/datasette/templates/400.html'

/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.7/site-packages/jinja2/utils.py:154: OSError
```
Based on the conversation in https://github.com/pytest-dev/pytest/issues/2970 I'm worried that my tests are opening too many files without closing them.

In particular... I call `sqlite3.connect(filepath)` a LOT - and I don't ever call `conn.close()` on those opened connections:

https://github.com/simonw/datasette/blob/cf7a2bdb404734910ec07abc7571351a2d934828/datasette/database.py#L58-L60

Could this be resulting in my tests eventually opening too many unclosed file handles? How could I confirm this?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/846/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1359557737,I_kwDOBm6k_c5RCTRp,1798,"Parts of YAML file do not work when db name is ""off""",562352,CharlesNepote,closed,0,,,,,4,2022-09-01T22:10:57Z,2022-09-02T00:02:53Z,2022-09-01T23:56:33Z,NONE,,"I guess this issue is not very important and probably rare.

To reproduce:
* create and populate a db named `off.db`
* in the yaml file, add any kind of information below `databases:\n   off:`
* the data are not taken into account (because ""off"" is interpreted as ""false"")

YAML file:
```yaml
title: Some title
description_html: |-
  <p>This is an experiment.</p>
databases:
  off:
    tables:
      products_from_owners:
        title: products_from_owners*
        description_html: |-
          <p>Description</p>
```

The result for http://xxxx.xxx/-/metadata gives:
```json
{
    ""title"": ""Some title"",
    ""description_html"": ""<p>This is an experiment.</p>"",
    ""databases"": {
        ""false"": {
            ""tables"": {
                ""products_from_owners"": {
                    ""title"": ""products_from_owners*"",
                    ""description_html"": ""<p>Description</p>""
                }
            }
        }
    }
}
```
=> see the `""false""` instead of `""off""`.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1798/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1524867951,I_kwDOBm6k_c5a46Nv,1980,"""Cannot sort table by id"" when sortable_columns is used",9599,simonw,open,0,,,,,2,2023-01-09T03:21:33Z,2023-01-09T03:23:53Z,,OWNER,,"I had an instance with this in `metadata.yml`:

```yaml
databases:
  timezones:
    tables:
      timezones:
        sortable_columns:
        - tzid
```
When I clicked on the ""Apply"" button here:

<img width=""702"" alt=""image"" src=""https://user-images.githubusercontent.com/9599/211236149-6c85eca9-5200-440a-9bc8-a913cd8164f5.png"">

It sent me to `/timezones/timezones?_sort=id&id__exact=133` with the error message:

> 500: Cannot sort table by id",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1980/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
649429772,MDU6SXNzdWU2NDk0Mjk3NzI=,886,Reconsider how _actor_X magic parameter deals with missing values,9599,simonw,open,0,,,,,2,2020-07-02T00:00:38Z,2020-09-11T21:35:26Z,,OWNER,,"I had to build a custom `_actorornull` prefix for [datasette-saved-queries](https://github.com/simonw/datasette-saved-queries/blob/37c00e56ac398e1f9aa342d30357de013a9b37b4/datasette_saved_queries/__init__.py):
```python
def actorornull(key, request):
    if request.actor is None:
        return None
    return request.actor.get(key)


@hookimpl
def register_magic_parameters():
    return [
        (""actorornull"", actorornull),
    ]
```
Maybe the `actor` magic in Datasette core should do that out of the box?

https://github.com/simonw/datasette/blob/f1f581b7ffcd5d8f3ae6c1c654d813a6641410eb/datasette/default_magic_parameters.py#L14-L17

",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/886/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1052851176,I_kwDOBm6k_c4-wTvo,1507,ReadTheDocs build failed for 0.59.2 release,9599,simonw,closed,0,,,,,6,2021-11-14T05:24:34Z,2021-11-14T05:41:55Z,2021-11-14T05:41:55Z,OWNER,,"I had to cancel the 0.59.2 release because ReadTheDocs was failing to build the documentation.

https://readthedocs.org/projects/datasette/builds/15268454/

```
 /home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/bin/python -m sphinx -T -b html -d _build/doctrees -D language=en . _build/html
Running Sphinx v1.8.5
loading translations [en]... done
making output directory...
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 27 source files that are out of date
updating environment: 27 added, 0 changed, 0 removed
reading sources... [  3%] authentication

Traceback (most recent call last):
  File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/cmd/build.py"", line 304, in build_main
    app.build(args.force_all, filenames)
  File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/application.py"", line 341, in build
    self.builder.build_update()
  File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/builders/__init__.py"", line 347, in build_update
    len(to_build))
  File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/builders/__init__.py"", line 360, in build
    updated_docnames = set(self.read())
  File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/builders/__init__.py"", line 468, in read
    self._read_serial(docnames)
  File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/builders/__init__.py"", line 490, in _read_serial
    self.read_doc(docname)
  File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/builders/__init__.py"", line 534, in read_doc
    doctree = read_doc(self.app, self.env, self.env.doc2path(docname))
  File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/io.py"", line 318, in read_doc
    pub.publish()
  File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/docutils/core.py"", line 219, in publish
    self.apply_transforms()
  File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/docutils/core.py"", line 200, in apply_transforms
    self.document.transformer.apply_transforms()
  File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/transforms/__init__.py"", line 90, in apply_transforms
    Transformer.apply_transforms(self)
  File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/docutils/transforms/__init__.py"", line 171, in apply_transforms
    transform.apply(**kwargs)
  File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/transforms/__init__.py"", line 245, in apply
    apply_source_workaround(n)
  File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/util/nodes.py"", line 94, in apply_source_workaround
    for classifier in reversed(node.parent.traverse(nodes.classifier)):
TypeError: argument to reversed() must be a sequence

Exception occurred:
  File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/util/nodes.py"", line 94, in apply_source_workaround
    for classifier in reversed(node.parent.traverse(nodes.classifier)):
TypeError: argument to reversed() must be a sequence
The full traceback has been saved in /tmp/sphinx-err-vkl0oE.log, if you want to report the issue to the developers.
Please also report this if it was a user error, so that a better error message can be provided next time.
A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks! 
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1507/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
964400482,MDU6SXNzdWU5NjQ0MDA0ODI=,310,`sqlite-utils insert --flatten` option to flatten nested JSON,9599,simonw,closed,0,,,,,3,2021-08-09T21:23:08Z,2021-10-16T13:54:56Z,2021-08-09T21:44:06Z,OWNER,,"I had to do this with a `jq` recipe today: https://til.simonwillison.net/cloudrun/tailing-cloud-run-request-logs

```
cat log.json | jq -c '[leaf_paths as $path | {
  ""key"": $path | join(""_""), ""value"": getpath($path)
}] | from_entries' \
| sqlite-utils insert /tmp/logs.db logs - --nl --alter --batch-size 1
```
That was to turn something like this:
```json
  {
    ""httpRequest"": {
      ""latency"": ""0.112114537s"",
      ""requestMethod"": ""GET"",
      ""requestSize"": ""534"",
      ""status"": 200,
    },
    ""insertId"": ""6111722f000b5b4c4d4071e2"",
    ""labels"": {
      ""service"": ""datasette-io""
    }
  }
```
Into this instead:
```json
{
    ""httpRequest_latency"": ""0.112114537s"",
    ""httpRequest_requestMethod"": ""GET"",
    ""httpRequest_requestSize"": ""534"",
    ""httpRequest_status"": 200,
    ""insertId"": ""6111722f000b5b4c4d4071e2"",
    ""labels_service"": ""datasette-io""
}
```
I have to do this often enough that I think it should be an option, `--flatten` - so I can do this instead:
```
cat log.json | sqlite-utils insert /tmp/logs.db logs - --flatten
```",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/310/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1532000914,I_kwDOBm6k_c5bUHqS,1990,Suggestion: Highlight error messages ('These facets timed out'),116795,pax,open,0,,,,,0,2023-01-13T09:40:58Z,2023-01-13T09:40:58Z,,NONE,,"I had trouble figuring out why faceting didn't work in some instances, it took a while before I noticed the _These facets timed out_ notice. 

It might help if that would be highlighted, or fading out highlight - if one might think it would be too visually disturbing.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1990/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
751195017,MDU6SXNzdWU3NTExOTUwMTc=,1111,Accessing a database's `.json` is slow for very large SQLite files,15178711,asg017,open,0,,,,,3,2020-11-26T00:27:27Z,2021-01-04T19:57:53Z,,CONTRIBUTOR,,"I have a SQLite DB that's pretty large, 23GB and something like 300 million rows. I expect that most queries I run on it will be slow, which is fine, but there are some things that Datasette does that makes working with the DB very slow. Specifically, when I access the `.json` metadata for a table (which I believe it comes from `datasette/views/database.py`, it takes 43 seconds for the request to come in:

```bash
$ time curl localhost:9999/out.json
{""database"": ""out"", ""size"": 24291454976, ""tables"": [{""name"": ""PageviewsHour"", ""columns"": [""file"", ""code"", ""page"", ""pageviews""], ""primary_keys"": [], ""count"": null, ""hidden"": false, ""fts_table"": null, ""foreign_keys"": {""incoming"": [], ""outgoing"": [{""other_table"": ""PageviewsHourFiles"", ""column"": ""file"", ""other_column"": ""file_id""}]}, ""private"": false}, {""name"": ""PageviewsHourFiles"", ""columns"": [""file_id"", ""filename"", ""sha256"", ""size"", ""day"", ""hour""], ""primary_keys"": [""file_id""], ""count"": null, ""hidden"": false, ""fts_table"": null, ""foreign_keys"": {""incoming"": [{""other_table"": ""PageviewsHour"", ""column"": ""file_id"", ""other_column"": ""file""}], ""outgoing"": []}, ""private"": false}, {""name"": ""sqlite_sequence"", ""columns"": [""name"", ""seq""], ""primary_keys"": [], ""count"": 1, ""hidden"": false, ""fts_table"": null, ""foreign_keys"": {""incoming"": [], ""outgoing"": []}, ""private"": false}], ""hidden_count"": 0, ""views"": [], ""queries"": [], ""private"": false, ""allow_execute_sql"": true, ""query_ms"": 43340.23213386536}
real	0m43.417s
user	0m0.006s
sys	0m0.016s
```

I suspect this is because a `COUNT(*)` is happening under the hood, which, when I run it through sqlite directly, does take around the same time:

```bash
$ time sqlite3 out.db < <(echo ""select count(*) from PageviewsHour;"")
362794272

real	0m44.523s
user	0m2.497s
sys	0m6.703s
```

I'm using the `.json` request in the [Observable Datasette Client](https://observablehq.com/@asg017/datasette-client) to 1) verify that a link passed in is a reachable Datasette instance, and 2) a quick way to look at metadata for a db. A few different solutions I can think of:

1. Have some other endpoint, like `/-/datasette.json` that the Observable Datasette client can fetch from to verify that the passed in URL is a valid Datasette (doesnt solve the slow problem, feel free to split this issue into 2)
2. Have a way to turn off table counts when accessing a database's `.json` view, like `?no_count=1` or something
3. Maybe have a timeout on the `table_counts()` function if it takes too long. which is odd, because it seems like it already does that (I think?), I can debug a little more if that's the case

More than happy to debug further, or send a PR if you like one of the proposals above!",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1111/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
593751293,MDU6SXNzdWU1OTM3NTEyOTM=,97,"Adding a ""recreate"" flag to the `Database` constructor",1448859,betatim,closed,0,,,,,4,2020-04-04T05:41:10Z,2020-04-15T14:29:31Z,2020-04-13T03:52:29Z,NONE,,"I have a [script](https://github.com/betatim/binder-datasette/blob/master/create-db.ipynb) that imports data into a sqlite DB. When I re-run that script I'd like to remove the existing sqlite DB, instead of adding to it. The pragmatic answer is to add the check and file deletion to my script.

However I thought it would be easy and useful for others to add a `recreate=True` flag to `db = sqlite_utils.Database(""binder-launches.db"")`. After taking a look at the code for it I am not so sure any more. This is because the connection string could be a URL (or ""connection string"") like `""file:///tmp/foo.db""`. I don't know what the equivalent of `os.path.exists()` is for a connection string or how to detect that something is a connection string and raise an error ""can't use recreate=True and conn_string at the same time"".

Does anyone have an idea/suggestion where to start investigating?",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/97/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
861622839,MDU6SXNzdWU4NjE2MjI4Mzk=,256,inserting with --nl errors with: sqlite3.OperationalError: table <table> has no column named <column>,279769,rathboma,closed,0,,,,,2,2021-04-19T18:01:03Z,2021-05-19T03:26:54Z,2021-05-19T03:26:54Z,NONE,,"I have a `jsonl` file, it is 10,000 lines long.

Inserting from the cli with `sqlite-utils insert db table file --nl --batch-size 10000` fails with this missing column error, even though I'm telling it to use the whole file in the first batch.

This seems similar to #18 and #139, but maybe it's unique to `--nl`?",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/256/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
824750134,MDU6SXNzdWU4MjQ3NTAxMzQ=,1251,facet option not appearing when table is big,15836677,verajosemanuel,open,0,,,,,0,2021-03-08T16:54:04Z,2021-03-08T16:54:16Z,,NONE,,"I have a big table with more than 500.000 rows.
Trying to facet by one of my columns, the options are not available as for the other smaller tables.

I have tried to set it in URL as:

`&_facet=city_id`

to no avail.

is there any limit?  how can I force the option ""facet"" to appear for big tables?
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1251/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
807817197,MDU6SXNzdWU4MDc4MTcxOTc=,229,Hitting `_csv.Error: field larger than field limit (131072)`,631242,frosencrantz,closed,0,,,,,3,2021-02-13T19:52:44Z,2021-02-14T21:33:33Z,2021-02-14T21:33:33Z,NONE,,"I have a csv file where one of the fields is so large it is throwing an exception with this error and stops loading:
	```
	_csv.Error: field larger than field limit (131072)
	```

The stack trace occurs here: https://github.com/simonw/sqlite-utils/blob/3.1/sqlite_utils/cli.py#L633


There is a way to handle this that helps:
https://stackoverflow.com/questions/15063936/csv-error-field-larger-than-field-limit-131072

One issue I had with this problem was sqlite-utils only provides limited context as to where the problem line is.
There is the progress bar, but that is by percent rather than by line number.  It would have been helpful if it could have provided a line number.

Also, it would have been useful if it had allowed the loading to continue with later lines.
",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/229/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
796234313,MDU6SXNzdWU3OTYyMzQzMTM=,1210,Immutable Database w/ Canned Queries,525780,heyarne,closed,0,,,,,2,2021-01-28T18:08:29Z,2021-02-05T11:30:34Z,2021-02-05T11:30:34Z,NONE,,"I have a database that I only want to read from; when instructing datasette to treat the database as immutable my defined canned queries disappear. Are these two features incompatible or have I hit an unintended bug? Thanks for datasette in any way, it's a joy to use!",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1210/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
352768017,MDU6SXNzdWUzNTI3NjgwMTc=,362,Add option to include/exclude columns in search filters,78156,annapowellsmith,open,0,,,,,1,2018-08-22T01:32:08Z,2020-11-03T19:01:59Z,,NONE,,"I have a dataset with many columns, of which only some are likely to be of interest for searching.

It would be great for usability if the search filters in the UI could be configured to include/exclude columns.

See also: https://github.com/simonw/datasette/issues/292",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/362/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1058790545,I_kwDOBm6k_c4_G9yR,1519,base_url is omitted in JSON and CSV views,157158,phubbard,closed,0,,,,,22,2021-11-19T18:10:45Z,2021-12-01T17:50:09Z,2021-11-20T19:11:21Z,NONE,,"I have a datasette deployment, using Apache2 to reverse proxy:

       ProxyPass /ged http://thor.phfactor.net:8001
       ProxyPreserveHost On

In settings.json I have
```json
{
    ""base_url"": ""/ged/"",
    ""trace_debug"": 1,
    ""template_debug"": 1
}
```
and datasette works correctly. However, if you view a query and then click on the 'This data as json, CSV' both links omit the base_url prefix and are therefore 404.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1519/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
781262510,MDU6SXNzdWU3ODEyNjI1MTA=,1181,"Certain database names results in 404: ""Database not found: None""",1470389,jieter,closed,0,,,6346396,Datasette 0.54,4,2021-01-07T12:01:16Z,2021-12-21T18:25:15Z,2021-01-25T05:13:19Z,NONE,,"I have a file named `test-database (1).sqlite`. When requesting the home route `/`, I see datasette is able to read it correctly:

<img width=""409"" alt=""Screenshot 2021-01-07 at 12 54 46"" src=""https://user-images.githubusercontent.com/1470389/103889964-9442b800-50e7-11eb-9548-fbf10ed3b1ea.png"">

However, if I click any of the links, datasette replies with: `Error 404 Database not found: None`

It seems the hash is crucial, as renaming the file to `database (1).sqlite` makes the error go away.

This lines checks for a single dash:
https://github.com/simonw/datasette/blob/97fb10c17dd007a275ab743742e93e932335ad67/datasette/views/base.py#L184

```
$ datasette test-database\ \(1\).sqlite 
INFO:     Started server process [68314]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://127.0.0.1:8001 (Press CTRL+C to quit)
INFO:     127.0.0.1:54043 - ""GET /favicon.ico HTTP/1.1"" 200 OK
INFO:     127.0.0.1:54043 - ""GET / HTTP/1.1"" 200 OK
...
INFO:     127.0.0.1:54044 - ""GET /favicon.ico HTTP/1.1"" 200 OK
INFO:     127.0.0.1:54044 - ""GET /test-database (1) HTTP/1.1"" 404 Not Found

```
Version:
```
$ datasette --version
datasette, version 0.53
```
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1181/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1855836914,I_kwDOCGYnMM5undLy,583,Get rid of test.utils.collapse_whitespace,9599,simonw,closed,0,,,,,1,2023-08-17T23:31:09Z,2023-08-18T00:59:19Z,2023-08-18T00:59:19Z,OWNER,,"I have a neater pattern for this now - instead of: https://github.com/simonw/sqlite-utils/blob/1dc6b5aa644a92d3654f7068110ed7930989ce71/tests/test_create.py#L472-L475

I now prefer:

https://github.com/simonw/sqlite-utils/blob/1dc6b5aa644a92d3654f7068110ed7930989ce71/tests/test_create.py#L1163-L1171",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/583/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1221849746,I_kwDOBm6k_c5I0_KS,1732,Custom page variables aren't decoded,52649,tannewt,open,0,,,,,2,2022-04-30T14:55:46Z,2022-05-03T01:50:45Z,,NONE,,"I have a page `templates/filer/{filer_id}.html`. It uses `filer_id` in a `sql()` call to fetch data. With 0.61.1 this no longer works because the spaces in IDs isn't preserved. Instead, the escaped version is passed into the template and the id isn't present in my db.

Datasette should unescape the url component before passing them into the template.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1732/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
907642546,MDU6SXNzdWU5MDc2NDI1NDY=,264,"Supporting additional output formats, like GeoJSON",25778,eyeseast,closed,0,,,,,3,2021-05-31T18:03:32Z,2021-06-03T05:12:21Z,2021-06-03T05:12:21Z,CONTRIBUTOR,,"I have a project going where it would be useful to do some spatial processing in SQLite (instead of large files) and then output GeoJSON. So my workflow would be something like this:

1. Read Shapefiles, GeoJSON, CSVs into a SQLite database
2. Join, filter, prune as needed
3. Export GeoJSON for just the stuff I need at that moment, while still having a database of things that will be useful later

I'm wondering if this is worth adding to SQLite-utils itself (GeoJSON, at least), or if it's better to make a counterpart to the ecosystem of `*-to-sqlite` tools, say a suite of `sqlite-to-*` things. Or would it be crazy to have a plugin system?",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/264/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
2023057255,I_kwDOBm6k_c54lWdn,2212,Can't filter with numbers,605070,fzakaria,open,0,,,,,0,2023-12-04T05:26:29Z,2023-12-04T05:26:29Z,,NONE,,"I have a schema that uses numbers for a column (actually it's a boolean 1 or 0 but SQLite doesn't have Boolean).
I can't seem to get the facet to work or even filtering on this column.

My guess is that Datasette is ""stringifying"" the number and it's not matching?
Example: https://debian-sqlelf.fly.dev/debian/elf_symbols?_sort_desc=name&_facet=exported&exported=0",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2212/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
989109888,MDU6SXNzdWU5ODkxMDk4ODg=,1460,Override column metadata with metadata from another column,72577720,MichaelTiemannOSC,open,0,,,,,0,2021-09-06T12:13:33Z,2021-09-06T12:13:33Z,,CONTRIBUTOR,,"I have a table from the PUDL project (https://github.com/catalyst-cooperative/pudl) that looks like this:

```
CREATE TABLE fuel_ferc1 (
	id INTEGER NOT NULL, 
	record_id TEXT, 
	utility_id_ferc1 INTEGER, 
	report_year INTEGER, 
	plant_name_ferc1 TEXT, 
	fuel_type_code_pudl VARCHAR(7), 
	fuel_unit VARCHAR(7),
	fuel_qty_burned FLOAT,
	fuel_mmbtu_per_unit FLOAT, 
	fuel_cost_per_unit_burned FLOAT, 
	fuel_cost_per_unit_delivered FLOAT, 
	fuel_cost_per_mmbtu FLOAT, 
	PRIMARY KEY (id), 
	FOREIGN KEY(plant_name_ferc1, utility_id_ferc1) REFERENCES plants_ferc1 (plant_name_ferc1, utility_id_ferc1), 
	CONSTRAINT fuel_ferc1_fuel_type_code_pudl_enum CHECK (fuel_type_code_pudl IN ('coal', 'oil', 'gas', 'solar', 'wind', 'hydro', 'nuclear', 'waste', 'unknown')), 
	CONSTRAINT fuel_ferc1_fuel_unit_enum CHECK (fuel_unit IN ('ton', 'mcf', 'bbl', 'gal', 'kgal', 'gramsU', 'kgU', 'klbs', 'btu', 'mmbtu', 'mwdth', 'mwhth', 'unknown'))
);
```

Note that `fuel_unit` is a unit that **pint** can understand, and that `fuel_qty_burned` is a column of data that could be expressed in terms of actual units, not merely as a dimensionless number.  Ditto the `fuel_cost_per_unit_...` columns.  Is there a way to give a column a default metadata unit (such as *tons* or *USD/ton*) and then let that be overridden when the metadata in another column says *barrels* or *USD/gramsU*?

@catalyst-cooperative",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1460/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
512996469,MDU6SXNzdWU1MTI5OTY0Njk=,607,Ways to improve fuzzy search speed on larger data sets?,8431341,zeluspudding,closed,0,,,,,6,2019-10-27T17:31:37Z,2019-11-07T03:38:10Z,2019-11-07T03:38:10Z,NONE,,"I have an sqlite table with 16 million rows in it. Having read @simonw article ""[Fast Autocomplete Search for Your Website](https://24ways.org/2018/fast-autocomplete-search-for-your-website/)"" I was curious to try datasette to see what kind of query performance I could get out of it. In truth I don't need to do full text search since all I would like to do is give my users a way to search for the names of investors such as ""Warren Buffet"", or ""Tim Cook"" (who's names are in a single column).

On the first search, Datasette takes over 20 seconds to return all records associated with `elon musk`:

> ![image](https://user-images.githubusercontent.com/8431341/67638889-a86e1100-f8b7-11e9-9f7e-a9d13a42e988.png)

> ![image](https://user-images.githubusercontent.com/8431341/67638825-ed457800-f8b6-11e9-94d1-b44f1a40ee8c.png)

If I rerun the same search, it then takes almost 9 seconds:
> ![image](https://user-images.githubusercontent.com/8431341/67638908-e4a17180-f8b7-11e9-9d00-748c80ef1f21.png)

That's far to slow to implement an autocomplete feature. I could reduce the latency by making a special table of only unique investor names, thereby reducing the search space to less than a million rows (then I'd need to implement a way to add only new investor names to the table as I received new data.. about 4,000 rows a day). If I did that, I'm still concerned the new table wouldn't be lean enough to lookup investor names quickly. Plus, even if I can implement the autocomplete feature, I would still finally have to lookup records for that investors which would take between 8 - 20 seconds. 

Are there any tricks for speeding this up?

Here's my hardware:
> ![image](https://user-images.githubusercontent.com/8431341/67638861-55945980-f8b7-11e9-96a8-ca76c7c68c5d.png)
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/607/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1072106103,I_kwDOBm6k_c4_5wp3,1542,feature request: order and dependency of plugins (that use js),33631,fs111,open,0,,,,,1,2021-12-06T12:40:45Z,2021-12-15T17:47:08Z,,NONE,,"I have been playing with datasette for the last couple of weeks and it is great! I am a big fan of `datasette-cluster-map` and wanted to enhance it a bit with a what I would call a sub-plugin. I basically want to add more controls to the map that cluster map provides. I have been looking into its code and how the plugin management works, but it seems what I am trying to do is not doable without hacks in js.

Basically what would like to have is a way to say load my plugin after the plugins I depend on have been loaded and rendered. There seems to be no prior art where plugins have these dependencies on the js level so I was wondering if that could be added or if it exists how to do it.

Basically what I want to do is:

my-awesome-plugin has a dependency on datastte-cluster-map. Whenever datasette cluster map has finished rendering on page load, call my plugin, but no earlier. To make that work datasette probably needs some total order in which way plugins are loaded intialized.

Since I am new to datastte, I may be missing something obvious, so please let me know if the above makes no sense.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1542/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1553615704,I_kwDOBm6k_c5cmktY,2001,Datasette is not compatible with SQLite's strict quoting compilation option,406380,gwk,open,0,,,,,4,2023-01-23T19:10:07Z,2023-01-25T04:59:58Z,,NONE,,"I have linked Python3.11 on macOS against recent SQLite that was compiled using `-DSQLITE_DQS=0`. This option disables interpretation of double-quoted identifiers as string literals, described in the SQLite docs as a ""MySQL 3.x misfeature"". See https://www.sqlite.org/quirks.html#dblquote for background.

Datasette uses the double-quote syntax in a number of key places, and is thus completely broken in this environment.

My experience was to `pip install datasette`, then run `datasette serve -I my-data.db`. When I visit `http://127.0.0.1:8001` I get a 500 response.

The error: `sqlite3.OperationalError: no such column: geometry_columns`

The responsible SQL: `'select 1 from sqlite_master where tbl_name = ""geometry_columns""'`

I then installed datasette from GitHub master in development mode and changed the offending SQL to use correct quotes: `""select 1 from sqlite_master where tbl_name = 'geometry_columns'""`.

With this change, I get a little further, but have the same problem with the first table name in my database (in my case, ""Meta""):
```
OperationalError: no such column: Meta
Traceback (most recent call last):
  File ""/Users/gwk/external/datasette/datasette/app.py"", line 1522, in route_path
    response = await view(request, send)
               ^^^^^^^^^^^^^^^^^^^^^^^^^
  File ""/Users/gwk/external/datasette/datasette/views/base.py"", line 151, in view
    return await self.dispatch_request(request)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ""/Users/gwk/external/datasette/datasette/views/base.py"", line 105, in dispatch_request
    response = await handler(request)
               ^^^^^^^^^^^^^^^^^^^^^^
  File ""/Users/gwk/external/datasette/datasette/views/index.py"", line 70, in get
    ""fts_table"": await db.fts_table(table),
                 ^^^^^^^^^^^^^^^^^^^^^^^^^
  File ""/Users/gwk/external/datasette/datasette/database.py"", line 363, in fts_table
    return await self.execute_fn(lambda conn: detect_fts(conn, table))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ""/Users/gwk/external/datasette/datasette/database.py"", line 213, in execute_fn
    return await asyncio.get_event_loop().run_in_executor(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ""/usr/local/py/Python.framework/Versions/3.11/lib/python3.11/concurrent/futures/thread.py"", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ""/Users/gwk/external/datasette/datasette/database.py"", line 211, in in_thread
    return fn(conn)
           ^^^^^^^^
  File ""/Users/gwk/external/datasette/datasette/database.py"", line 363, in <lambda>
    return await self.execute_fn(lambda conn: detect_fts(conn, table))
                                              ^^^^^^^^^^^^^^^^^^^^^^^
  File ""/Users/gwk/external/datasette/datasette/utils/__init__.py"", line 588, in detect_fts
    rows = conn.execute(detect_fts_sql(table)).fetchall()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
sqlite3.OperationalError: no such column: Meta
INFO:     127.0.0.1:50258 - ""GET / HTTP/1.1"" 500 Internal Server Error
```

I will try to continue playing with this, but I also hope that the datasette developers will enable this mode in a test environment as I am unlikely to be able to exercise all of the SQL in the codebase, or make a pull request very soon.

Note that the DQS setting compile-time option can be overridden at runtime with calls to the C API:
```
sqlite3_db_config(db, SQLITE_DBCONFIG_DQS_DDL, 0, (void*)0);
sqlite3_db_config(db, SQLITE_DBCONFIG_DQS_DML, 0, (void*)0);
```

As far as I can tell, `sqlite3_db_config` is not exposed in Python, but perhaps we could figure out how to invoke it using `ctypes`.
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2001/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
646992096,MDU6SXNzdWU2NDY5OTIwOTY=,872,Release non-alpha plugins when 0.45 is out,9599,simonw,closed,0,,,5533512,Datasette 0.45,0,2020-06-28T19:42:01Z,2020-07-01T23:48:51Z,2020-07-01T23:48:51Z,OWNER,,"I have several plugins currently marked as alphas because they depend on `0.45a3`. When 0.45 is released I can ship new versions of these plugins that are full releases, not alphas - and switch them to depending on 0.45 (as opposed to the alpha):

- [x] https://github.com/simonw/datasette-init
- [x] https://github.com/simonw/datasette-glitch
- [x] https://github.com/simonw/datasette-saved-queries
- [x] https://github.com/simonw/datasette-write",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/872/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
611540797,MDU6SXNzdWU2MTE1NDA3OTc=,751,Ability to set custom default _size on a per-table basis,9599,simonw,closed,0,,,5471110,Datasette 0.43,4,2020-05-04T00:13:03Z,2020-05-28T05:00:22Z,2020-05-28T05:00:20Z,OWNER,,"I have some tables where I'd like the default page size to be 10, without affecting the rest of my Datasette instance.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/751/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
999902754,I_kwDOBm6k_c47mU4i,1473,base logo link visits `undefined` rather than href url,192568,mroswell,open,0,,,,,2,2021-09-18T04:17:04Z,2021-09-19T00:45:32Z,,CONTRIBUTOR,,"I have two connected sites:
http://www.SaferOrToxic.org 
(a Hugo website)
and:
http://disinfectants.SaferOrToxic.org/disinfectants/listN
(a datasette table page)

The latter is linked as ""The List"" in the former's menu.
(I'd love a prettier URL, but that's what I've got.)

On:
http://disinfectants.SaferOrToxic.org/disinfectants/listN
... all the other menu links should point back to:
https://www.SaferOrToxic.org 
And they do!

But the logo, for some reason--though it has an href pointing to:
https://www.SaferOrToxic.org
Keeps going to this instead:
https://disinfectants.saferortoxic.org/disinfectants/undefined

What is causing that? How can I fix it?

In #1284 back in March, I was doing battle with the index.html template, in a still unresolved issue. (I wanted only a single table page at the root.)

But I thought, well, if I can't resolve that, at least I could just point the main website to the datasette page (""The List,"") and then have the List point back to the home website.

The menu hrefs to https://www.SaferOrToxic.org work just fine, exactly as they should, from the datasette page. Even the Home link works properly.

But the logo link keeps rewriting to: https://disinfectants.saferortoxic.org/disinfectants/undefined

This is the HTML:
```
<a class=""text-3xl font-bold leading-none"" href=""https://www.saferortoxic.org""><img src=""https://www.saferortoxic.org/images/logo_hu26e4dce8d5931af1ea33526b28fc8383_9734_c52a4f1635ef88bda858373270551ed2.webp"" class=""custom-logo"" alt=""Logo: Safer or Toxic?"" width=""300px""></a>
```


Is this somehow related to cloudflare?
Or something in the datasette code?

I'm starting to think it's a cloudflare issue.
<img width=""1046"" alt=""Screen Shot 2021-09-18 at 12 09 57 AM"" src=""https://user-images.githubusercontent.com/192568/133872043-5a008e14-88cb-43fd-b921-e63ff50636fb.png"">

Can I at least rule out it being a datasette issue?

My repository is here:
https://github.com/mroswell/list-N

(BTW, I couldn't figure out how to reference a local image, either, on the datasette side, which is why I'm using the image from the www home page.)
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1473/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
334190959,MDU6SXNzdWUzMzQxOTA5NTk=,321,Wildcard support in query parameters,12617395,bsilverm,closed,0,,,3439337,0.23.1,8,2018-06-20T18:03:56Z,2018-06-21T17:00:10Z,2018-06-21T04:55:26Z,NONE,,"I haven't found a way to get the wildcard (%) inserted automatically in to a query parameter.  This would be useful for cases the query parameter is followed by a LIKE clause.  Wrapping the parameter name using the wildcard character within the metadata file (ie - ...where xyz like %:querystring%) does not seem to work.  Can this be made possible?  Or if not, can the template be extended to provide a tip to the user that they need to insert the wildcard characters themselves?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/321/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1473659191,I_kwDOBm6k_c5X1kE3,1929,Incorrect link from the API explorer to the JSON API documentation,3556,davidbgk,closed,0,,,,,4,2022-12-03T02:08:58Z,2022-12-06T19:36:23Z,2022-12-06T19:34:20Z,CONTRIBUTOR,,"I installed `datasette==1.0a1`.

When I go to http://127.0.0.1:8001/-/api I have a link: `Use this tool to try out the [Datasette API](https://docs.datasette.io/en/1.0a1/json_api.html).` but that documentation page does not exist.

I'm not sure where it has to be fixed, should it link to the stable page https://docs.datasette.io/en/stable/json_api.html , the latest one https://docs.datasette.io/en/latest/json_api.html#the-json-write-api or would it be more appropriated to deploy documentation for the `1.0a1` version?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1929/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1102637351,I_kwDOBm6k_c5BuOkn,1598,Replace update-docs-help.py script with cog,9599,simonw,closed,0,,,,,1,2022-01-14T00:33:27Z,2022-01-14T00:47:57Z,2022-01-14T00:47:57Z,OWNER,,"I introduced `cog` in #1594 - I can use this to replace the older `update-docs-help.py` mechanism:

https://github.com/simonw/datasette/blob/76d66d5b2bf10249c0beaac0999b93ac8d757f48/tests/test_docs.py#L36-L53",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1598/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1189113609,I_kwDOBm6k_c5G4G8J,1697,"`Request.fake(..., url_vars={})`",9599,simonw,closed,0,,,3268330,Datasette 1.0,1,2022-04-01T01:48:40Z,2022-04-01T02:02:18Z,2022-04-01T02:02:10Z,OWNER,,"I just created an alternative `.fake()` method because I wanted to fake the `url_vars` captured in the route as well:
```python
from datasette.utils.asgi import Request
class Request(Request):

    @classmethod
    def fake(cls, path_with_query_string, method=""GET"", scheme=""http"", url_vars=None):
        """"""Useful for constructing Request objects for tests""""""
        path, _, query_string = path_with_query_string.partition(""?"")
        scope = {
            ""http_version"": ""1.1"",
            ""method"": method,
            ""path"": path,
            ""raw_path"": path_with_query_string.encode(""latin-1""),
            ""query_string"": query_string.encode(""latin-1""),
            ""scheme"": scheme,
            ""type"": ""http"",
        }
        if url_vars:
            scope[""url_route""] = {
                ""kwargs"": url_vars
            }
        return cls(scope, None)
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1697/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1181236173,I_kwDOCGYnMM5GaDvN,422,Reconsider not running convert functions against null values,9599,simonw,open,0,,,,,1,2022-03-25T20:22:40Z,2022-03-25T20:23:21Z,,OWNER,,"I just got caught out by the fact that `None` values are not processed by the `.convert()` mechanism https://github.com/simonw/sqlite-utils/blob/0b7b80bd40fe86e4d66a04c9f607d94991c45c0b/sqlite_utils/db.py#L2504-L2510

I had run this code while working on #420 and I wasn't sure why it didn't work:

```
$ sqlite-utils add-column content.db articles score float
$ sqlite-utils convert content.db articles score '
import random
random.seed(10)

def convert(value):
    global random
    return random.random()
'
```
The reason it didn't work is that the newly added `score` column was full of `null` values.

I fixed it by doing this instead:

    $ sqlite-utils add-column content.db articles score float --not-null-default 1.0

But this indicates to me that the design of `convert()` here may be incorrect.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/422/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
594237015,MDU6SXNzdWU1OTQyMzcwMTU=,718,Plugin idea: datasette-redirects,9599,simonw,open,0,,,,,0,2020-04-05T03:41:38Z,2023-08-30T22:17:31Z,,OWNER,,"I just had to write a one-off custom plugin to redirect niche-musems.com to www.niche-museums.com (https://github.com/simonw/museums/issues/21) - it would be great if this kind of thing could be handled by a configurable plugin.

https://github.com/simonw/museums/blob/6b1faf00c463b2228860d4d62d104b11935e01b1/plugins/redirect_www.py",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/718/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,reopened
944870799,MDU6SXNzdWU5NDQ4NzA3OTk=,1394,Big performance boost on faceting: skip the inner order by,9599,simonw,closed,0,,,,,4,2021-07-14T23:32:29Z,2021-07-16T02:23:32Z,2021-07-15T00:05:50Z,OWNER,,"I just noticed something that could make for a huge performance improvement in faceting.

The default query used by Datasette when faceting looks like this:
```sql
select
  country_long,
  count(*)
from (
  select * from [global-power-plants] order by rowid
)
where
  country_long is not null
group by
  country_long
order by
  count(*) desc
```
Here it takes 53ms: https://global-power-plants.datasettes.com/global-power-plants?sql=select%0D%0A++country_long%2C%0D%0A++count%28*%29%0D%0Afrom+%28%0D%0A++select+*+from+%5Bglobal-power-plants%5D+order+by+rowid%0D%0A%29%0D%0Awhere%0D%0A++country_long+is+not+null%0D%0Agroup+by%0D%0A++country_long%0D%0Aorder+by%0D%0A++count%28*%29+desc

Note that there's a `order by rowid` in there which isn't necessary - the order on that inner query doesn't matter since we're grouping and counting.

I had assumed SQLite would optimize this away - but it turns out it doesn't! Consider this version of the query, with that pointless order by removed:
```
select
  country_long,
  count(*)
from (
  select * from [global-power-plants]
)
where
  country_long is not null
group by
  country_long
order by
  count(*) desc
```
https://global-power-plants.datasettes.com/global-power-plants?sql=select%0D%0A++country_long%2C%0D%0A++count%28*%29%0D%0Afrom+%28%0D%0A++select+*+from+%5Bglobal-power-plants%5D%0D%0A%29%0D%0Awhere%0D%0A++country_long+is+not+null%0D%0Agroup+by%0D%0A++country_long%0D%0Aorder+by%0D%0A++count%28*%29+desc runs in 7.2ms!

I tried this optimization on a table with 2.5m rows in it - without the optimization it took 5 seconds, with the optimization it took 450ms. So this is a very significant improvement!",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1394/reactions"", ""total_count"": 2, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 1, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1073712378,I_kwDOBm6k_c4__4z6,1544,Code that detects the label column for a table is case-sensitive,9599,simonw,closed,0,,,,,2,2021-12-07T20:01:25Z,2021-12-07T20:03:43Z,2021-12-07T20:03:43Z,OWNER,,I just noticed that a column called `Name` is not being picked up as the label column for a table.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1544/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
763207948,MDU6SXNzdWU3NjMyMDc5NDg=,1141,Default styling for bullet point lists,9599,simonw,closed,0,,,,,0,2020-12-12T02:49:33Z,2021-03-29T00:14:05Z,2021-03-29T00:14:05Z,OWNER,,"I just noticed that https://datasette.io/content/recent_releases (which uses `datasette-render-markdown`) is missing its bullet points:

<img width=""864"" alt=""content__recent_releases__399_rows"" src=""https://user-images.githubusercontent.com/9599/101970823-95b0bb80-3be1-11eb-9512-9274cef64cf3.png"">
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1141/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1843710170,I_kwDOBm6k_c5t5Mja,2136,Query view shouldn't return `columns`,9599,simonw,closed,0,,,9700784,Datasette 1.0a3,4,2023-08-09T17:23:57Z,2023-08-09T19:03:04Z,2023-08-09T19:03:04Z,OWNER,,"I just noticed that https://latest.datasette.io/fixtures/roadside_attraction_characteristics.json?_labels=on&_size=1 returns:
```json
{
  ""ok"": true,
  ""next"": ""1"",
  ""rows"": [
    {
      ""rowid"": 1,
      ""attraction_id"": {
        ""value"": 1,
        ""label"": ""The Mystery Spot""
      },
      ""characteristic_id"": {
        ""value"": 2,
        ""label"": ""Paranormal""
      }
    }
  ],
  ""truncated"": false
}
```
But https://latest.datasette.io/fixtures.json?sql=select+rowid%2C+attraction_id%2C+characteristic_id+from+roadside_attraction_characteristics+order+by+rowid+limit+1 returns:
```json
{
  ""rows"": [
    {
      ""rowid"": 1,
      ""attraction_id"": 1,
      ""characteristic_id"": 2
    }
  ],
  ""columns"": [
    ""rowid"",
    ""attraction_id"",
    ""characteristic_id""
  ],
  ""ok"": true,
  ""truncated"": false
}
```
The `columns` key in the query response is inconsistent with the table response.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2136/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
725184645,MDU6SXNzdWU3MjUxODQ2NDU=,1034,Better way of representing binary data in .csv output,9599,simonw,closed,0,,,6026070,0.51,19,2020-10-20T04:28:58Z,2021-06-17T18:13:21Z,2020-10-29T22:47:46Z,OWNER,,"I just noticed this: https://latest.datasette.io/fixtures/binary_data.csv

```csv
rowid,data
1,b'\x15\x1c\x02\xc7\xad\x05\xfe'
2,b'\x15\x1c\x03\xc7\xad\x05\xfe'
```
There's no good way to represent binary data in a CSV file, but this seems like one of the more-bad options.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1034/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
519038979,MDU6SXNzdWU1MTkwMzg5Nzk=,10,Failed to import workout points,9599,simonw,closed,0,,,,,4,2019-11-07T04:50:22Z,2019-11-08T01:18:37Z,2019-11-08T01:18:37Z,MEMBER,,"I just ran the script and it failed to import any `workout_points`, though it did import `workouts`.",197882382,healthkit-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/10/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
707407567,MDU6SXNzdWU3MDc0MDc1Njc=,171,Idea: transitive closure tables for tree structures,649467,mhalle,closed,0,,,,,2,2020-09-23T14:17:33Z,2020-10-22T04:38:35Z,2020-10-22T04:07:14Z,NONE,,"I just read that sqlite has a transitive closure table extension using a virtual table in order to represent trees:

https://charlesleifer.com/blog/querying-tree-structures-in-sqlite-using-python-and-the-transitive-closure-extension/

Even without this extension, though, a util to build a transitive closure table would allow trees to be queried easily. Since it relies on self-referential foreign keys, the relationships might even be able to be automatically detected. ",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/171/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1558644003,I_kwDOBm6k_c5c5wUj,2006,Teach `datasette publish` to pin to `datasette<1.0` in a 0.x release,9599,simonw,open,0,,,3268330,Datasette 1.0,2,2023-01-26T19:17:40Z,2023-01-26T19:20:53Z,,OWNER,,"I just realized that when I ship Datasette 1.0 there may be automated deployments out there which could deploy the 1.0 version by accident, potentially breaking any customizations that aren't compatible with the 1.0 changes.

I can hopefully help avoid that by shipping one last entry in the `0.x` series that ensures `datasette publish` pins to `<1.0` when it installs Datasette itself.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2006/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1886771493,I_kwDOCGYnMM5wddkl,592,`table.transform()` should preserve `rowid` values,9599,simonw,closed,0,,,,,6,2023-09-08T00:42:38Z,2023-09-10T17:46:41Z,2023-09-09T00:45:32Z,OWNER,,"I just spotted a bug when using https://datasette.io/plugins/datasette-configure-fts and https://datasette.io/plugins/datasette-edit-schema at the same time.

Steps to reproduce:

- Configure FTS for a table, then run a test search
- Edit the schema for that table and change the order of columns
- Run the test search again

I got the wrong search results, which I think is because the `_fts` table pointed to the first table by `rowid` but those `rowid` values were entirely rewritten as a consequence of running `table.transform()` on the table.

Reconfiguring FTS on the table fixed the problem.

I think `table.transform()` should be able to preserve `rowid` values.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/592/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
836829560,MDU6SXNzdWU4MzY4Mjk1NjA=,248,support for Apache Arrow / parquet files I/O,649467,mhalle,open,0,,,,,1,2021-03-20T14:59:30Z,2021-10-28T23:46:48Z,,NONE,,"I just started looking at Apache Arrow using pyarrow for import and export of tabular datasets, and it looks quite compelling. It might be worth looking at for sqlite-utils and/or datasette.

As a test, I took a random jsonl data dump of a dataset I have with floats, strings, and ints and converted it to arrow's parquet format using the naive `pyarrow.parquet.write_file()` command, which has automatic type inferrence. It compressed down to 7% of the original size. Conversion of a 26MB JSON file and serializing it to parquet was eyeblink instantaneous. Parquet files are portable and can be directly imported into pandas and other analytics software. 

The only hangup is the automatic type inference of the naive reader. It's great for general laziness and for parsing JSON columns (it correctly interpreted a table of mine with a JSON array). However, I did get an exception for a string column where most entries looked integer-like but had a couple values that weren't -- the reader tried to coerce all of them for some reason, even though the JSON type is string. Since the writer optionally takes a schema, it shouldn't be too hard to grab the sqlite header types. With some additional hinting, you might get datetime columns and JSON, which are native Arrow types. 

Somewhat tangentially, someone even wrote an sqlite vfs extension for Parquet: https://cldellow.com/2018/06/22/sqlite-parquet-vtable.html

",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/248/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
718521469,MDU6SXNzdWU3MTg1MjE0Njk=,1011,column name links broken in 0.50.1,649467,mhalle,closed,0,,,,,4,2020-10-10T03:37:51Z,2020-10-10T04:09:32Z,2020-10-10T03:52:07Z,NONE,,"I just upgraded from 0.49 to 0.50.1 and found that the links on column headers are broken. 

If I inspect the source, they have a leading ""//"" (without host or port) rather than including base_url like other links on the page do. The links in the ""gears"" menu for each column do work.

 I don't have custom templates for my project. 
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1011/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
377155320,MDU6SXNzdWUzNzcxNTUzMjA=,370,Integration with JupyterLab,82988,psychemedia,open,0,,,,,4,2018-11-04T13:57:13Z,2022-09-29T08:17:47Z,,CONTRIBUTOR,,"I just watched a demo video for the [JupyterLab Chart Editor](https://www.crowdcast.io/e/introducing-JupyterLab-Chart-Editor/) which wraps the plotly chart editor app in a JupyterLab panel and lets you open a plotly chart JSON file in that editor.  Essentially, it pops an HTML app into a panel in JupyterLab, and I think registers the app as a file viewer for a particular file type. (I'm not completely taken by it, tbh, because it means you can do irreproducible things to the chart definition file, but that's another issue).

JupyterLab extensions can also open files from a dialogue as the iframe/html previewer shows: https://github.com/timkpaine/jupyterlab_iframe.

This made me wonder about what `datasette` integration with JupyterLab might do.

For example, by right-clicking on a CSV file (for which there is already a CSV table view) in the file browser, offer a *View / Run as datasette* file viewer option that will:

- run the CSV file through `csvs-to-sqlite`;
- launch the `datasette` server and display the `datasette` view in a JupyterLab panel.

(? Create a new SQLite db for each CSV file and launch each datasette view on a new port? Or have a JupyterLab (session?) SQLite db that stores all `datasette` viewed CSVs and runs on a single port?) 

As a freebie, the `datasette` API would allow you to run efficient SQL queries against the file eg using using `pandas.read_sql()` queries in a notebook in the same space.

Related:

- [JupyterLab extensions docs](https://jupyterlab.readthedocs.io/en/stable/user/extensions.html)
-  a [cookiecutter for wrting JupyterLab extensions using Javascript](https://github.com/jupyterlab/extension-cookiecutter-js)
- a [cookiecutter for writing JupyterLab extensions using Typescript](https://github.com/jupyterlab/extension-cookiecutter-ts)
- tutorial: [Let’s Make an xkcd JupyterLab Extension](https://jupyterlab.readthedocs.io/en/stable/developer/xkcd_extension_tutorial.html)",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/370/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1182141761,I_kwDOBm6k_c5Gdg1B,1690,"Idea: `datasette.set_actor_cookie(response, actor)`",9599,simonw,open,0,,,,,2,2022-03-26T22:41:52Z,2022-03-26T22:43:00Z,,OWNER,,"I just wrote this code in a plugin and it felt like it could benefit from an abstraction: https://github.com/simonw/datasette-auth0/blob/152e6eb21e96e9b73bd9c205f9749a1297d0ef0b/datasette_auth0/__init__.py#L79-L92

```python
    redirect_response = Response.redirect(""/"")
    expires_at = int(time.time()) + (24 * 60 * 60)
    redirect_response.set_cookie(
        ""ds_actor"",
        datasette.sign(
            {
                ""a"": profile_response.json(),
                ""e"": baseconv.base62.encode(expires_at),
            },
            ""actor"",
        ),
    )
    return redirect_response
```
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1690/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
818430405,MDU6SXNzdWU4MTg0MzA0MDU=,1247,datasette.add_memory_database() method,9599,simonw,closed,0,,,,,2,2021-03-01T03:48:38Z,2021-03-01T04:02:26Z,2021-03-01T04:02:26Z,OWNER,,"I just wrote this code:

https://github.com/simonw/datasette/blob/47eb885cc2c3aafa03645c330c6f597bee9b3b25/tests/test_facets.py#L334-L335

It would be nice if you didn't have to separately instantiate a database object here.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1247/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
691537426,MDU6SXNzdWU2OTE1Mzc0MjY=,959,Internals API idea: results.dicts in addition to results.rows,9599,simonw,open,0,,,,,0,2020-09-03T00:50:17Z,2020-09-03T00:50:17Z,,OWNER,,"I just wrote this code:
```python
    results = await database.execute(SEARCH_SQL, {""query"": query})
    return [dict(r) for r in results.rows]
```
How about having `results.dicts` as a utility property that does that?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/959/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1277328147,I_kwDOCGYnMM5MInsT,446,Use Just to automate running tests and linters locally,9599,simonw,closed,0,,,,,2,2022-06-20T19:51:09Z,2022-06-21T19:28:35Z,2022-06-20T19:54:50Z,OWNER,,I keep committing code that fails additional tests like `mypy` and `flake8` and `black`. Automate those using Just.,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/446/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
2029161033,I_kwDOCGYnMM548opJ,606,str and int as aliases for text and integer,9599,simonw,closed,0,,,,,2,2023-12-06T18:35:49Z,2023-12-06T19:44:04Z,2023-12-06T18:49:32Z,OWNER,,"I keep making this mistake:
```bash
sqlite-utils add-column content.db assets _since int
```
```
Usage: sqlite-utils add-column [OPTIONS] PATH TABLE COL_NAME [[integer|float|b
                               lob|text|INTEGER|FLOAT|BLOB|TEXT]]
Try 'sqlite-utils add-column -h' for help.

Error: Invalid value for '[[integer|float|blob|text|INTEGER|FLOAT|BLOB|TEXT]]':
  'int' is not one of 'integer', 'float', 'blob', 'text', 'INTEGER', 'FLOAT', 'BLOB', 'TEXT'.
```",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/606/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
792625812,MDU6SXNzdWU3OTI2MjU4MTI=,1198,Plugin testing documentation on using pytest-httpx,9599,simonw,closed,0,,,,,1,2021-01-23T18:46:16Z,2021-01-24T20:40:38Z,2021-01-24T20:38:43Z,OWNER,,"I keep on having to figure this out: if you use the https://pypi.org/project/pytest-httpx/ fixture to write tests against mocked external APIs, they will fail because that module will break Datasette's own `datasette.client` testing mechanism.

You can fix this using:

```python
@pytest.fixture
def non_mocked_hosts():
    return [""localhost""]
```

See https://github.com/simonw/datasette-indieauth/blob/1.2/tests/test_indieauth.py

I can add this tip to the https://docs.datasette.io/en/stable/testing_plugins.html page.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1198/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
273895344,MDU6SXNzdWUyNzM4OTUzNDQ=,92,Add --license --license_url --source --source_url --title arguments to datasette publish,9599,simonw,closed,0,,,,,0,2017-11-14T18:27:07Z,2017-11-15T05:04:41Z,2017-11-15T05:04:41Z,OWNER,,"I keep on using the `echo '{""source"": ""...""}' | datasette publish now --metadata=-` pattern, which suggests it makes sense for us to support these as optional arguments.

https://gist.github.com/simonw/9f8bf23b37a42d7628c4dcc4bba10253",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/92/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
602619330,MDU6SXNzdWU2MDI2MTkzMzA=,45,Use raise_for_status() everywhere,9599,simonw,open,0,,,,,1,2020-04-19T04:38:28Z,2020-04-19T04:39:22Z,,MEMBER,,"I keep seeing errors which I think are caused by authentication or rate limit problems but which appear to be unexpected JSON responses - presumably because they are actually an error message.

Recent example: https://github.com/simonw/jsk-fellows-on-twitter/runs/598892575

Using `response.raise_for_status()` everywhere will make these errors less confusing.",206156866,twitter-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/45/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
772408750,MDU6SXNzdWU3NzI0MDg3NTA=,1156,Rename _schemas to _internal,9599,simonw,closed,0,,,6346396,Datasette 0.54,1,2020-12-21T19:27:58Z,2021-01-24T21:20:39Z,2020-12-21T19:51:18Z,OWNER,,"I like `_internal` as the name for the in-memory Datasette schema database better.

Refs #1154 #1150 #1155",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1156/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
646840273,MDU6SXNzdWU2NDY4NDAyNzM=,871,Rename the _timestamp magic parameters to _now,9599,simonw,closed,0,,,5533512,Datasette 0.45,1,2020-06-28T04:49:08Z,2020-06-28T19:49:49Z,2020-06-28T19:49:49Z,OWNER,,I like the shorter name better. Follows on from #842.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/871/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
696908389,MDU6SXNzdWU2OTY5MDgzODk=,961,Verification checks for metadata.json on startup,9599,simonw,open,0,,,,,2,2020-09-09T15:21:53Z,2020-09-09T15:24:31Z,,OWNER,,"I lost a bunch of time yesterday trying to figure out why a Datasette instance wasn't starting up - it turned out it was because I had a `facets:` reference that mentioned a column that did not exist.

Catching these on startup would be good.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/961/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1006016302,I_kwDOBm6k_c479pcu,1477,Consider adding request to the documented default template context,9599,simonw,open,0,,,,,0,2021-09-24T02:34:09Z,2021-09-24T02:34:09Z,,OWNER,,I made a plugin for this today but I think perhaps it should be a default thing instead: https://datasette.io/plugins/datasette-template-request,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1477/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
274315193,MDU6SXNzdWUyNzQzMTUxOTM=,106,Document how pagination works,9599,simonw,closed,0,,,,,1,2017-11-15T21:44:32Z,2019-06-24T06:42:33Z,2019-06-24T06:42:33Z,OWNER,,I made a start at that in this comment: https://news.ycombinator.com/item?id=15691926,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/106/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
445850934,MDU6SXNzdWU0NDU4NTA5MzQ=,473,Plugin hook: filters_from_request,9599,simonw,closed,0,,,,,13,2019-05-19T18:44:33Z,2021-12-17T23:11:30Z,2021-12-17T19:02:17Z,OWNER,,"I meant to add this as part of the facets plugin mechanism but didn't quite get to it. Original idea was to allow plugins to register extra filters, as seen in `datasette/filters.py`:

https://github.com/simonw/datasette/blob/260085838887ee343f4d3b177c422e7aef5ade9d/datasette/filters.py#L83-L98",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/473/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
339095976,MDU6SXNzdWUzMzkwOTU5NzY=,334,extra_options not passed to heroku publisher,719357,kamicut,closed,0,,,,,2,2018-07-06T23:26:12Z,2018-07-24T04:53:21Z,2018-07-10T01:46:04Z,NONE,,"I might be wrong but I was not able to publish to `heroku` with `--extra-options`, I think `extra_options` is not being used in this function [here](https://github.com/simonw/datasette/blob/master/datasette/utils.py#L369). 

Any help appreciated! ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/334/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
453243459,MDU6SXNzdWU0NTMyNDM0NTk=,503,Handle SQLite databases with spaces in their names?,7936571,chrismp,closed,0,9599,simonw,,,1,2019-06-06T21:20:59Z,2019-11-04T23:16:30Z,2019-11-04T23:16:30Z,NONE,,"I named my SQLite database ""Government workers"" and published it to Heroku. When I clicked the ""Government workers"" database online it lead to a 404 page: `Database not found: Government%20workers`.

I believe this is because the database name has a space.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/503/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
431800286,MDU6SXNzdWU0MzE4MDAyODY=,427,"New design for facet abstraction, including querystring and metadata.json",9599,simonw,closed,0,,,,,10,2019-04-11T02:24:15Z,2019-05-29T21:39:12Z,2019-05-03T00:11:29Z,OWNER,,"I need a better design for query strings for facets (and for how facets are enabled in `metadata.json`).

Think of all of the potential kinds of facets:

* `?_facet_array=tags` where tags is a JSON array of values
* `_facet_date=datetimecol` - faceted by date part of a datetime
* `_facet_bins=numeric_column` - can I do some kind of fancy binning here? Might need to take an argument
* `?_facet_bins=numeric_column:5` - could be a way to take an argument. We’ll ignore columns with a : in their name.
* `?_facet_json=jsoncol:jsonpath` - could use a JSON path to extract out something to facet on?
* `?_facet_percentile=numericcolumn` - could this work?
* `?_facet_function=column:sqlfunctionname` - maybe this could be interesting? Would allow for e.g. facet by soundex
* `?_facet_prefix=column:prefix` - facet by terms but only if they start with a specific prefix
* `?_facet_substring=column:3,6` - facet by a substr(column, 3, 6)

Maybe bundling JSON in querystrings is a way to do options?

`?_facet_distance={""latitude_column"":""x"",...}`

Could detect values starting with `{` - and if for some weird reason you have a column starting with that character you can pass this instead: `?_facet_percentile={""column"": ""{value}""}`

This could even be the mechanism that allows us to extend regular facets to support additional options like adding a sum or max to each one.

Problem: it’s not obvious what the name associated with these facets should be. What if one column is faceted multiple times using multiple facet variants?

Maybe just number them? name1=… name2=… etc?

Other option is to use Solr style querystring syntax for notation. Solr does this: `?f.price.facet.range.gap=100&f.age.facet.range.gap=10`

So how about this:

`?_facet_range=age&_facet_range.span=5`

Related: #359",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/427/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
957741820,MDU6SXNzdWU5NTc3NDE4MjA=,305,Python: need a way to execute a count with an extra where clause,9599,simonw,closed,0,,,,,1,2021-08-02T04:52:02Z,2021-08-02T05:08:22Z,2021-08-02T05:08:22Z,OWNER,,I need this for #304. I'll probably add this to the `.execute_count()` method as `where=` and `where_args=`.,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/305/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1421529723,I_kwDOBm6k_c5UutJ7,1850,Write API in Datasette core,9599,simonw,closed,0,,,8658075,Datasette 1.0a0,13,2022-10-24T22:13:24Z,2022-11-29T20:11:20Z,2022-11-29T20:11:20Z,OWNER,,"I need this for Datasette Cloud, and in thinking it through I realized that it's really time Datasette grew a default write API as well.

I'm going to mostly model this off `sqlite-utils`, since I've spent a bunch of time iterating on a pseudo-JSON API for that over the past few years (piping JSON to stdin etc).

I want this for Datasette 1.0. I'm going to be building it in the new [1.0-dev](https://github.com/simonw/datasette/tree/1.0-dev) branch, which is automatically deployed to https://latest-1-0-dev.datasette.io/ running on Cloud Run.

API features to build:

- [x] #1852
  - [x] #1856
  - [x] #1857
  - [x] #1858
  - [x] #1859
- [x] #1871
  - [x] #1888
- [x] #1868
- [x] #1851
- [x] #1863
- [x] #1864
- [x] #1866
- [x] https://github.com/simonw/datasette/issues/1882
- [x] #1862
- [x] #1874
  - [x] https://github.com/simonw/datasette/issues/1887
- [x] #1877

Bumped to later on:

- #1855
- #1878
- #1873
- #1875
- Make sure CORS works
- https://github.com/simonw/datasette/issues/1889
- Alter a table - `sqlite-utils transform` style (more powerful than straight ALTER)
- Execute SQL against a write connection
- Maybe even multiple write SQL statements bundled in a single transaction
- https://github.com/simonw/datasette/issues/1867
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1850/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
636426530,MDU6SXNzdWU2MzY0MjY1MzA=,829,Ability to set ds_actor cookie such that it expires,9599,simonw,closed,0,,,5512395,Datasette 0.44,6,2020-06-10T17:31:40Z,2020-06-10T19:41:35Z,2020-06-10T19:40:05Z,OWNER,,I need this for `datasette-auth-github`: https://github.com/simonw/datasette-auth-github/issues/62#issuecomment-642152076,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/829/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
672421411,MDU6SXNzdWU2NzI0MjE0MTE=,916,"Support reverse pagination (previous page, has-previous-items)",9599,simonw,open,0,,,,,7,2020-08-04T00:32:06Z,2021-04-03T23:43:11Z,,OWNER,,"I need this for `datasette-graphql` for full compatibility with the way Relay likes to paginate - using cursors for paginating backwards as well as for paginating forwards.

> This may be the kick I need to get Datasette pagination to work in reverse too.
_Originally posted by @simonw in https://github.com/simonw/datasette-graphql/issues/2#issuecomment-668305853_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/916/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1306492437,I_kwDOBm6k_c5N334V,1770,`handle_exception` plugin hook for custom error handling,9599,simonw,closed,0,,,8303187,Datasette 0.62,14,2022-07-15T20:52:49Z,2022-08-14T15:25:51Z,2022-08-14T15:25:51Z,OWNER,,"I need this for a couple of plugins, both of which are broken at the moment:
- https://github.com/simonw/datasette-sentry/issues/1
- https://github.com/simonw/datasette-show-errors/issues/2",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1770/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
920884085,MDU6SXNzdWU5MjA4ODQwODU=,1377,Mechanism for plugins to exclude certain paths from CSRF checks,9599,simonw,closed,0,,,,,3,2021-06-15T00:48:20Z,2021-06-23T22:51:33Z,2021-06-23T22:51:33Z,OWNER,,I need this for a plugin I'm building that offers a POST API.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1377/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1495431932,I_kwDOBm6k_c5ZInr8,1951,`datasette.create_token(...)` method for creating signed API tokens,9599,simonw,closed,0,,,8711695, Datasette 1.0a2,6,2022-12-14T01:25:34Z,2022-12-14T02:43:45Z,2022-12-14T02:42:05Z,OWNER,,"I need this for:
- #1947

And I can refactor this to use it too:
- #1855

By making this a documented internal API it can be used by other plugins too. It's also going to be really useful for writing tests.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1951/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
737153927,MDU6SXNzdWU3MzcxNTM5Mjc=,197,Rethink how table.search() method works,9599,simonw,closed,0,,,6079500,3.0,5,2020-11-05T18:04:34Z,2020-11-08T17:07:37Z,2020-11-08T17:07:37Z,OWNER,,"I need to improve this method to help build `sqlite-utils search` in #192 (PR is #195).

The challenge is deciding how it should handle sorting by relevance - especially since that is easy in FTS5 but not at all easy in FTS4.

> Latest test failure:
> ```
> 114  ->	        assert [(""racoons are biting trash pandas"", ""USA"", ""bar"")] == table.search(
> 115  	            ""bite"", order=""rowid""
> 116  	        )
> 117  	
> 118  	
> 119  	def test_optimize_fts(fresh_db):
> (Pdb) table.search(""bite"")
> [(2, 'racoons are biting trash pandas', 'USA', 'bar', -9.641434262948206e-07)]
> ```
> The problem here is that the `table.search()` method now behaves differently for FTS4 v.s. FTS5 tables.
> 
> With FTS4 you get back just the table columns.
>
> With FTS5 you also get back the `rowid` as the first column and the `rank` score as the last column.
> 
> This is weird. It also makes me question whether having `.search()` return a list of tuples is the right API design.

_Originally posted by @simonw in https://github.com/simonw/sqlite-utils/pull/195#issuecomment-722542895_",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/197/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1054246919,I_kwDOBm6k_c4-1ogH,1511,Review plugin hooks for Datasette 1.0,9599,simonw,open,0,,,3268330,Datasette 1.0,1,2021-11-15T23:26:05Z,2021-11-16T01:20:14Z,,OWNER,,I need to perform a detailed review of the plugin interface - especially the plugin hooks like [register_facet_classes()](https://docs.datasette.io/en/stable/plugin_hooks.html#register-facet-classes) which I don't yet have complete confidence in.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1511/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1337541526,I_kwDOBm6k_c5PuUOW,1780,`facet_time_limit_ms` and `sql_time_limit_ms` overlap?,53165,davepeck,open,0,,,,,1,2022-08-12T17:55:37Z,2022-08-15T23:50:08Z,,NONE,,"I needed more than the default 200ms to facet a specific column in a database I was working with, so I ran `datasette` with `--setting facet_time_limit_ms 30000` &mdash; definitely overkill! 

But it still didn't work; it took a moment to realize I also needed to up my `sql_time_limit_ms` to something larger too.

I'm happy to submit a PR that documents this behavior if it's helpful. Or, if there's a code change we'd like to make (like making sure `sql_time_limit_ms` is always set to the larger of itself and `facet_time_limit_ms`), happy to do that too.

Apologies if I missed this somewhere in the docs. And: thanks. I'm really enjoying the simple, effective tooling datasette gives me out of the box for exploring my databases!",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1780/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
699622046,MDU6SXNzdWU2OTk2MjIwNDY=,962,datasette --pdb option for debugging errors,9599,simonw,closed,0,,,5818042,Datasette 0.49,1,2020-09-11T18:33:10Z,2020-09-11T21:34:24Z,2020-09-11T18:38:01Z,OWNER,,I needed to debug an exception from deep inside a Jinja template the other day. I hacked this together and it helped.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/962/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
959278472,MDU6SXNzdWU5NTkyNzg0NzI=,1417,Use codespell in CI to spot spelling errors,9599,simonw,closed,0,,,,,1,2021-08-03T16:14:15Z,2021-08-03T16:36:40Z,2021-08-03T16:36:40Z,OWNER,,"I noticed Rich is using this: https://github.com/willmcgugan/rich/commit/9c12a4537499797c43725fff5276ef0da62423ef#diff-ce84a1b2c9eb4ab3ea22f610cad7111cb9a2f66365c3b24679901376a2a73ab2

Ran it against the Datasette docs and found a bunch of obvious fixes, surprisingly with no false positives.

```
datasette % codespell docs/*.rst
docs/authentication.rst:63: perfom ==> perform
docs/authentication.rst:76: perfom ==> perform
docs/changelog.rst:429: repsonse ==> response
docs/changelog.rst:503: permissons ==> permissions
docs/changelog.rst:717: compatibilty ==> compatibility
docs/changelog.rst:1172: browseable ==> browsable
docs/deploying.rst:191: similiar ==> similar
docs/internals.rst:434: Respons ==> Response, respond
docs/internals.rst:440: Respons ==> Response, respond
docs/internals.rst:717: tha ==> than, that, the
docs/performance.rst:42: databse ==> database
docs/plugin_hooks.rst:667: utilites ==> utilities
docs/publish.rst:168: countainer ==> container
docs/settings.rst:352: inalid ==> invalid
docs/sql_queries.rst:406: preceeded ==> preceded, proceeded
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1417/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1396948693,I_kwDOBm6k_c5TQ77V,1829,Table/database that is private due to inherited permissions does not show padlock,9599,simonw,closed,0,,,,,8,2022-10-04T23:14:16Z,2022-10-24T02:23:46Z,2022-10-24T02:11:37Z,OWNER,,"I noticed that a table page that is private because the database or instance is private, e.g. this one:

<img width=""982"" alt=""image"" src=""https://user-images.githubusercontent.com/9599/193947303-ed643dcd-a07c-4ffb-ab22-539c4c803206.png"">

Is not displaying the padlock icon that indicates the table is not visible to the public.

<img width=""969"" alt=""image"" src=""https://user-images.githubusercontent.com/9599/193947373-ddbdf398-ea67-4e21-b54d-60c48496701d.png"">

Same issue for the database page too, which in this case is private due to `view-instance`.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1829/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1876407598,I_kwDOBm6k_c5v17Uu,2169,execute-sql on a database should imply view-database/view-permission,9599,simonw,closed,0,,,,,0,2023-08-31T22:45:56Z,2023-08-31T22:46:28Z,2023-08-31T22:46:28Z,OWNER,,"I noticed that a token with `execute-sql` permission alone did not work, because it was not allowed to view the instance of the database.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2169/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
705190723,MDU6SXNzdWU3MDUxOTA3MjM=,160,"table.enable_fts(..., replace=True)",9599,simonw,closed,0,,,5896742,2.19,1,2020-09-20T21:36:23Z,2020-09-24T20:35:47Z,2020-09-20T22:05:51Z,OWNER,,"I noticed that https://til.simonwillison.net/ search doesn't use porter stemming. I'd like to add that, but since [the build script](https://github.com/simonw/til/blob/9d3f0fca30e94df3970df52b0447907a077e4673/build_database.py) always operates on an existing database (to avoid re-rendering markdown and re-building image thumbnails) I'd like it to only add porter stemming if it's not there already.

So I'd like to be able to say ""set up FTS to look like this, and fix it if it doesn't"".

I think the neatest way to do that is with a `replace=True` argument to `.enable_fts()`, for consistency with `def .create_view(self, name, sql, replace=True)`.

So the `replace=True` argument would check and see if the configured FTS exists already with the correct options (columns, stemming, triggers) - and if any of those are incorrect it would call `.disable_fts()` and then create a new FTS configuration with the correct options.

",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/160/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1223699280,I_kwDOBm6k_c5I8CtQ,1739,.db downloads should be served with an ETag,9599,simonw,closed,0,,,,,6,2022-05-03T05:11:21Z,2022-05-04T18:21:18Z,2022-05-03T14:59:51Z,OWNER,,"I noticed that my Pyodide Datasette prototype is downloading the same database file every single time rather than browser caching it:

![image](https://user-images.githubusercontent.com/9599/166407074-dee19587-0667-4424-9e88-d3b5b90fd819.png)
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1739/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
838148087,MDU6SXNzdWU4MzgxNDgwODc=,250,Handle byte order marks (BOMs) in CSV files,9599,simonw,closed,0,,,,,3,2021-03-22T22:13:18Z,2021-05-29T05:34:21Z,2021-05-29T05:34:21Z,OWNER,,I often find `sqlite-utils insert ... --csv` creates a first column with a weird character at the start of it - which it turns out is the UTF-8 BOM. Fix that.,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/250/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1702354223,I_kwDOBm6k_c5ld90v,2070,Mechanism for deploying a preview of a branch using Vercel,9599,simonw,closed,0,,,,,2,2023-05-09T16:21:45Z,2023-05-09T16:25:00Z,2023-05-09T16:24:31Z,OWNER,,"I prototyped that here: https://github.com/simonw/one-off-actions/blob/main/.github/workflows/deploy-datasette-branch-preview.yml

It deployed the `json-extras-query` branch here: https://datasette-preview-json-extras-query.vercel.app/",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2070/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1166731361,I_kwDOCGYnMM5Fiuhh,414,I forgot to include the changelog in the 3.25.1 release,9599,simonw,closed,0,,,,,7,2022-03-11T18:32:36Z,2022-03-11T18:40:39Z,2022-03-11T18:40:39Z,OWNER,,"I pushed a release for https://github.com/simonw/sqlite-utils/releases/tag/3.25.1 but forgot to include the release notes in `docs/changelog.rst`

This means https://sqlite-utils.datasette.io/en/stable/changelog.html isn't showing them.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/414/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
921878733,MDU6SXNzdWU5MjE4Nzg3MzM=,272,"Idea: import CSV to memory, run SQL, export in a single command",9599,simonw,closed,0,,,,,22,2021-06-15T23:02:48Z,2021-06-19T23:36:48Z,2021-06-18T15:05:03Z,OWNER,,"I quite often load a CSV file into a SQLite DB, then do stuff with it (like export results back out again as a new CSV) without any intention of keeping the CSV file around afterwards.

What if `sqlite-utils` could do this for me? Something like this:

    sqlite-utils --csv blah.csv --csv baz.csv ""select * from blah join baz ...""
",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/272/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
326778161,MDU6SXNzdWUzMjY3NzgxNjE=,290,Consider increasing the default for num_sql_threads (currently 3),9599,simonw,open,0,,,,,0,2018-05-27T00:52:41Z,2018-05-27T00:52:41Z,,OWNER,,"I ran a very rough micro-benchmark on the new `num_sql_threads` config option (added in #285)

    datasette --config num_sql_threads:1 fivethirtyeight.db

Then

    ab -n 100 -c 10 'http://127.0.0.1:8011/fivethirtyeight-2628db9/twitter-ratio%2Fsenators'

| Number of threads | Requests/second |
|---|---|
| 1 | 4.57 |
| 3 | 9.77 |
| 10 | 13.53 |
| 20 | 15.24 
| 50 |  8.21 | 

This was on my early 2018 OS X laptop. Need to benchmark in other common environments before making a decision on changing the default. That said, the default of 3 was a number I plucked out of thin air.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/290/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1617602868,I_kwDOJHON9s5gaqk0,6,Character encoding problem,9599,simonw,open,0,,,,,2,2023-03-09T16:44:34Z,2023-04-14T15:22:09Z,,MEMBER,,"I ran against a recent note with this in it:

> Or just ""Actions ⚙️ ""

And got back:

<img width=""367"" alt=""image"" src=""https://user-images.githubusercontent.com/9599/224092586-e2475d8f-868c-489d-b495-2bfeb1a7db99.png"">

> `Actions ⚙️`

Pasting that into https://ftfy.vercel.app/?s=Actions+%E2%80%9A%C3%B6%C3%B4%C3%94%E2%88%8F%C3%A8+ gives this:

```python
s = 'Actions â\x80\x9aöôÃ\x94â\x88\x8fè'
s = s.encode('latin-1')
s = s.decode('utf-8')
s = s.encode('macroman')
s = s.decode('utf-8')
print(s)
```
<img width=""816"" alt=""image"" src=""https://user-images.githubusercontent.com/9599/224092864-2b796c40-7c68-40b1-ae38-d0c61721b742.png"">
",611552758,apple-notes-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/6/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
750330029,MDU6SXNzdWU3NTAzMzAwMjk=,1110,datasette publish option for installing extra apt-get packages,9599,simonw,closed,0,,,6055094,Datasette 0.52,2,2020-11-25T03:03:43Z,2020-11-28T23:28:56Z,2020-11-25T03:05:41Z,OWNER,,I ran into a need for this while playing with https://github.com/simonw/datasette-ripgrep - I need to install the `ripgrep` Ubuntu package when I deploy the plugin using Cloud Run.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1110/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1174162781,I_kwDOBm6k_c5F_E1d,1666,Refactor URL routing to enable testing,9599,simonw,closed,0,,,3268330,Datasette 1.0,3,2022-03-19T03:52:29Z,2022-03-19T16:32:03Z,2022-03-19T16:32:03Z,OWNER,,"I ran into some bugs earlier with URL routing - having more robust testing around this (especially since they are defined using regular expressions) would be really useful.

- A utility function that resolves a path against a list of reflexes and returns the match
- Make the routes and regular expressions available from a private Datasette method
- Add tests that exercise them

Related:
- #1660",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1666/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
816523763,MDU6SXNzdWU4MTY1MjM3NjM=,238,.add_foreign_key() corrupts database if column contains a space,9599,simonw,closed,0,,,,,1,2021-02-25T15:07:20Z,2021-02-25T16:54:02Z,2021-02-25T16:54:02Z,OWNER,,"I ran this:

    db[""Reports""].add_foreign_key(""Reported by ID"", ""Reporters"", ""id"")

And got this:

```
~/jupyter-venv/lib/python3.9/site-packages/sqlite_utils/db.py in add_foreign_keys(self, foreign_keys)
    616         # Have to VACUUM outside the transaction to ensure .foreign_keys property
    617         # can see the newly created foreign key.
--> 618         self.vacuum()
    619 
    620     def index_foreign_keys(self):

~/jupyter-venv/lib/python3.9/site-packages/sqlite_utils/db.py in vacuum(self)
    629 
    630     def vacuum(self):
--> 631         self.execute(""VACUUM;"")
    632 
    633 

~/jupyter-venv/lib/python3.9/site-packages/sqlite_utils/db.py in execute(self, sql, parameters)
    234             return self.conn.execute(sql, parameters)
    235         else:
--> 236             return self.conn.execute(sql)
    237 
    238     def executescript(self, sql):

DatabaseError: database disk image is malformed
```",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/238/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
749982022,MDU6SXNzdWU3NDk5ODIwMjI=,1105,Rebrand config as settings,9599,simonw,closed,0,,,6055094,Datasette 0.52,2,2020-11-24T19:35:12Z,2020-11-24T21:40:28Z,2020-11-24T21:40:28Z,OWNER,,"I realized I need a tracking ticket for this.

I want to start splitting things like plugin configuration and default facets / sort order out of `metadata.json` - so I want to start calling those things configuration. But the term configuration is already used for the `--config` family of global settings. So I'm rebranding that type of configuration as settings to free up the name ""configuration"" for more run-time concerns (default sort order) and plugin configuration.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1105/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1065429936,I_kwDOBm6k_c4_gSuw,1532,Use datasette-table Web Component to guide the design of the JSON API for 1.0,9599,simonw,open,0,,,3268330,Datasette 1.0,4,2021-11-28T20:37:18Z,2022-03-16T20:13:34Z,,OWNER,,"I realized that one of the reasons I'm having trouble committing to nailing down the JSON API for 1.0 is that I don't use it much myself - I use the `?_shape=array` one quite often, but I don't have any projects that are using the default, more fully-featured API.

As an experiment I built a Web Component for embedding Datasette tables on pages - https://github.com/simonw/datasette-table - and I think it's actually going to be a really useful tool for helping me dog food the v1.0 API design.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1532/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
610408908,MDU6SXNzdWU2MTA0MDg5MDg=,34,Command for retrieving dependents for a repo,9599,simonw,closed,0,,,,,6,2020-04-30T21:47:51Z,2020-05-03T15:53:01Z,2020-05-03T15:53:01Z,MEMBER,,"I really, really want to start grabbing this data: https://github.com/simonw/datasette/network/dependents",207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/34/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
959710008,MDU6SXNzdWU5NTk3MTAwMDg=,1419,`publish cloudrun` should deploy a more recent SQLite version,536941,fgregg,open,0,,,,,3,2021-08-04T00:45:55Z,2021-08-05T03:23:24Z,,CONTRIBUTOR,,"I recently changed from deploying a datasette using `datasette publish heroku` to `datasette publish cloudrun`. [A query that ran on the heroku site](https://odpr.bunkum.us/odpr-6c2f4fc?sql=with+pivot_members+as+%28%0D%0A++select%0D%0A++++f_num%2C%0D%0A++++max%28union_name%29+as+union_name%2C%0D%0A++++max%28aff_abbr%29+as+abbreviation%2C%0D%0A++++sum%28number%29+filter+%28%0D%0A++++++where%0D%0A++++++++yr_covered+%3D+2007%0D%0A++++%29+as+%222007%22%2C%0D%0A++++sum%28number%29+filter+%28%0D%0A++++++where%0D%0A++++++++yr_covered+%3D+2008%0D%0A++++%29+as+%222008%22%2C%0D%0A++++sum%28number%29+filter+%28%0D%0A++++++where%0D%0A++++++++yr_covered+%3D+2009%0D%0A++++%29+as+%222009%22%2C%0D%0A++++sum%28number%29+filter+%28%0D%0A++++++where%0D%0A++++++++yr_covered+%3D+2010%0D%0A++++%29+as+%222010%22%2C%0D%0A++++sum%28number%29+filter+%28%0D%0A++++++where%0D%0A++++++++yr_covered+%3D+2011%0D%0A++++%29+as+%222011%22%2C%0D%0A++++sum%28number%29+filter+%28%0D%0A++++++where%0D%0A++++++++yr_covered+%3D+2012%0D%0A++++%29+as+%222012%22%2C%0D%0A++++sum%28number%29+filter+%28%0D%0A++++++where%0D%0A++++++++yr_covered+%3D+2013%0D%0A++++%29+as+%222013%22%2C%0D%0A++++sum%28number%29+filter+%28%0D%0A++++++where%0D%0A++++++++yr_covered+%3D+2014%0D%0A++++%29+as+%222014%22%2C%0D%0A++++sum%28number%29+filter+%28%0D%0A++++++where%0D%0A++++++++yr_covered+%3D+2015%0D%0A++++%29+as+%222015%22%2C%0D%0A++++sum%28number%29+filter+%28%0D%0A++++++where%0D%0A++++++++yr_covered+%3D+2016%0D%0A++++%29+as+%222016%22%2C%0D%0A++++sum%28number%29+filter+%28%0D%0A++++++where%0D%0A++++++++yr_covered+%3D+2017%0D%0A++++%29+as+%222017%22%2C%0D%0A++++sum%28number%29+filter+%28%0D%0A++++++where%0D%0A++++++++yr_covered+%3D+2018%0D%0A++++%29+as+%222018%22%2C%0D%0A++++sum%28number%29+filter+%28%0D%0A++++++where%0D%0A++++++++yr_covered+%3D+2019%0D%0A++++%29+as+%222019%22%0D%0A++from%0D%0A++++lm_data%0D%0A++++left+join+ar_membership+using+%28rpt_id%29%0D%0A++where%0D%0A++++members+%21%3D+%27%27%0D%0A++++and+trim%28desig_name%29+%3D+%27NHQ%27%0D%0A++++and+union_name+not+in+%28%0D%0A++++++%27AFL-CIO%27%2C%0D%0A++++++%27CHANGE+TO+WIN%27%2C%0D%0A++++++%27FOOD+ALLIED+SVC+TRADES+DEPT+AFL-CIO%27%0D%0A++++%29%0D%0A++++and+f_num+not+in+%28387%2C+296%2C+123%2C+347%2C+531897%2C+30410%2C+49%29%0D%0A++++and+lower%28ar_membership.category%29+IN+%28%0D%0A++++++%27active+members%27%2C%0D%0A++++++%27active+professional%27%2C%0D%0A++++++%27regular+members%27%2C%0D%0A++++++%27full+time+member%27%2C%0D%0A++++++%27full+time+members%27%2C%0D%0A++++++%27active+education+support+professional%27%2C%0D%0A++++++%27active%27%2C%0D%0A++++++%27active+member%27%2C%0D%0A++++++%27members%27%2C%0D%0A++++++%27see+item+69%27%2C%0D%0A++++++%27regular%27%2C%0D%0A++++++%27dues+paying+members%27%2C%0D%0A++++++%27building+trades+journeyman%27%2C%0D%0A++++++%27full+per+capita+tax+payers%27%2C%0D%0A++++++%27active+postal%27%2C%0D%0A++++++%27active+membership%27%2C%0D%0A++++++%27full-time+member%27%2C%0D%0A++++++%27regular+active+member%27%2C%0D%0A++++++%27journeyman%27%2C%0D%0A++++++%27member%27%2C%0D%0A++++++%27journeymen%27%2C%0D%0A++++++%27one+half+per+capita+tax+payers%27%2C%0D%0A++++++%27schedule+1%27%2C%0D%0A++++++%27active+memebers%27%2C%0D%0A++++++%27active+members+-+us%27%2C%0D%0A++++++%27dues+paying+membership%27%0D%0A++++%29%0D%0A++GROUP+by%0D%0A++++f_num%0D%0A%29%0D%0Aselect%0D%0A++*%0D%0Afrom%0D%0A++pivot_members%0D%0Aorder+by%0D%0A++%222019%22+desc%3B), now throws a syntax error on the [cloudrun site](https://labordata.bunkum.us/odpr-6c2f4fc?sql=with+pivot_members+as+%28%0D%0A++select%0D%0A++++f_num%2C%0D%0A++++max%28union_name%29+as+union_name%2C%0D%0A++++max%28aff_abbr%29+as+abbreviation%2C%0D%0A++++sum%28number%29+filter+%28%0D%0A++++++where%0D%0A++++++++yr_covered+%3D+2007%0D%0A++++%29+as+%222007%22%2C%0D%0A++++sum%28number%29+filter+%28%0D%0A++++++where%0D%0A++++++++yr_covered+%3D+2008%0D%0A++++%29+as+%222008%22%2C%0D%0A++++sum%28number%29+filter+%28%0D%0A++++++where%0D%0A++++++++yr_covered+%3D+2009%0D%0A++++%29+as+%222009%22%2C%0D%0A++++sum%28number%29+filter+%28%0D%0A++++++where%0D%0A++++++++yr_covered+%3D+2010%0D%0A++++%29+as+%222010%22%2C%0D%0A++++sum%28number%29+filter+%28%0D%0A++++++where%0D%0A++++++++yr_covered+%3D+2011%0D%0A++++%29+as+%222011%22%2C%0D%0A++++sum%28number%29+filter+%28%0D%0A++++++where%0D%0A++++++++yr_covered+%3D+2012%0D%0A++++%29+as+%222012%22%2C%0D%0A++++sum%28number%29+filter+%28%0D%0A++++++where%0D%0A++++++++yr_covered+%3D+2013%0D%0A++++%29+as+%222013%22%2C%0D%0A++++sum%28number%29+filter+%28%0D%0A++++++where%0D%0A++++++++yr_covered+%3D+2014%0D%0A++++%29+as+%222014%22%2C%0D%0A++++sum%28number%29+filter+%28%0D%0A++++++where%0D%0A++++++++yr_covered+%3D+2015%0D%0A++++%29+as+%222015%22%2C%0D%0A++++sum%28number%29+filter+%28%0D%0A++++++where%0D%0A++++++++yr_covered+%3D+2016%0D%0A++++%29+as+%222016%22%2C%0D%0A++++sum%28number%29+filter+%28%0D%0A++++++where%0D%0A++++++++yr_covered+%3D+2017%0D%0A++++%29+as+%222017%22%2C%0D%0A++++sum%28number%29+filter+%28%0D%0A++++++where%0D%0A++++++++yr_covered+%3D+2018%0D%0A++++%29+as+%222018%22%2C%0D%0A++++sum%28number%29+filter+%28%0D%0A++++++where%0D%0A++++++++yr_covered+%3D+2019%0D%0A++++%29+as+%222019%22%0D%0A++from%0D%0A++++lm_data%0D%0A++++left+join+ar_membership+using+%28rpt_id%29%0D%0A++where%0D%0A++++members+%21%3D+%27%27%0D%0A++++and+trim%28desig_name%29+%3D+%27NHQ%27%0D%0A++++and+union_name+not+in+%28%0D%0A++++++%27AFL-CIO%27%2C%0D%0A++++++%27CHANGE+TO+WIN%27%2C%0D%0A++++++%27FOOD+ALLIED+SVC+TRADES+DEPT+AFL-CIO%27%0D%0A++++%29%0D%0A++++and+f_num+not+in+%28387%2C+296%2C+123%2C+347%2C+531897%2C+30410%2C+49%29%0D%0A++++and+lower%28ar_membership.category%29+IN+%28%0D%0A++++++%27active+members%27%2C%0D%0A++++++%27active+professional%27%2C%0D%0A++++++%27regular+members%27%2C%0D%0A++++++%27full+time+member%27%2C%0D%0A++++++%27full+time+members%27%2C%0D%0A++++++%27active+education+support+professional%27%2C%0D%0A++++++%27active%27%2C%0D%0A++++++%27active+member%27%2C%0D%0A++++++%27members%27%2C%0D%0A++++++%27see+item+69%27%2C%0D%0A++++++%27regular%27%2C%0D%0A++++++%27dues+paying+members%27%2C%0D%0A++++++%27building+trades+journeyman%27%2C%0D%0A++++++%27full+per+capita+tax+payers%27%2C%0D%0A++++++%27active+postal%27%2C%0D%0A++++++%27active+membership%27%2C%0D%0A++++++%27full-time+member%27%2C%0D%0A++++++%27regular+active+member%27%2C%0D%0A++++++%27journeyman%27%2C%0D%0A++++++%27member%27%2C%0D%0A++++++%27journeymen%27%2C%0D%0A++++++%27one+half+per+capita+tax+payers%27%2C%0D%0A++++++%27schedule+1%27%2C%0D%0A++++++%27active+memebers%27%2C%0D%0A++++++%27active+members+-+us%27%2C%0D%0A++++++%27dues+paying+membership%27%0D%0A++++%29%0D%0A++GROUP+by%0D%0A++++f_num%0D%0A%29%0D%0Aselect%0D%0A++*%0D%0Afrom%0D%0A++pivot_members%0D%0Aorder+by%0D%0A++%222019%22+desc%3B).

I suspect this is because they are running different versions of sqlite3.

- Heroku: sqlite3 3.31.1 ([-/versions](https://odpr.bunkum.us/-/versions))
- Cloudrun: sqlite3 3.27.2 ([-/versions](https://labordata.bunkum.us/-/versions))

If so, it would be great to

1. harmonize the sqlite3 versions across platforms
2. update the docker files so as to update the sqlite3 version for cloudrun",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1419/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
810397025,MDU6SXNzdWU4MTAzOTcwMjU=,1228,500 error caused  by faceting if a column called `n` exists,7107523,Kabouik,closed,0,,,,,5,2021-02-17T17:41:20Z,2022-03-19T06:44:40Z,2022-03-19T01:38:04Z,NONE,,"I recently discovered `datasette` thanks to your great talk at FOSDEM and would like to use it for some projects. However, when trying to use it on databases created from some csv ot tsv files, I am sometimes getting this issue when going to http://127.0.0.1:8001/databasetest/databasetest and I don't exactly understand what it refers to.

So far, I couldn't find anything relevant when reviewing the raw text files that could explain this issue, nor could I find something obvious between the files that generate this issue and those that don't. Does the error ring a bell and, if so, could you please point me to the right direction?

```
$ datasette databasetest.db 
INFO:     Started server process [1408482]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://127.0.0.1:8001 (Press CTRL+C to quit)
INFO:     127.0.0.1:56394 - ""GET / HTTP/1.1"" 200 OK
INFO:     127.0.0.1:56394 - ""GET /-/static/app.css?4e362c HTTP/1.1"" 200 OK
INFO:     127.0.0.1:56396 - ""GET /-/static-plugins/datasette_vega/main.2acbb312.css HTTP/1.1"" 200 OK
INFO:     127.0.0.1:56398 - ""GET /-/static-plugins/datasette_vega/main.08f5d3d8.js HTTP/1.1"" 200 OK
Traceback (most recent call last):
  File ""/home/kabouik/.local/lib/python3.7/site-packages/datasette/app.py"", line 1099, in route_path
    response = await view(request, send)
  File ""/home/kabouik/.local/lib/python3.7/site-packages/datasette/views/base.py"", line 147, in view
    request, **request.scope[""url_route""][""kwargs""]
  File ""/home/kabouik/.local/lib/python3.7/site-packages/datasette/views/base.py"", line 121, in dispatch_request
    return await handler(request, *args, **kwargs)
  File ""/home/kabouik/.local/lib/python3.7/site-packages/datasette/views/base.py"", line 260, in get
    request, database, hash, correct_hash_provided, **kwargs
  File ""/home/kabouik/.local/lib/python3.7/site-packages/datasette/views/base.py"", line 434, in view_get
    request, database, hash, **kwargs
  File ""/home/kabouik/.local/lib/python3.7/site-packages/datasette/views/table.py"", line 782, in data
    suggested_facets.extend(await facet.suggest())
  File ""/home/kabouik/.local/lib/python3.7/site-packages/datasette/facets.py"", line 168, in suggest
    and any(r[""n""] > 1 for r in distinct_values)
  File ""/home/kabouik/.local/lib/python3.7/site-packages/datasette/facets.py"", line 168, in <genexpr>
    and any(r[""n""] > 1 for r in distinct_values)
TypeError: '>' not supported between instances of 'str' and 'int'
INFO:     127.0.0.1:56402 - ""GET /databasetest/databasetest HTTP/1.1"" 500 Internal Server Error
INFO:     127.0.0.1:56402 - ""GET /-/static/app.css?4e362c HTTP/1.1"" 200 OK
INFO:     127.0.0.1:56404 - ""GET / HTTP/1.1"" 200 OK
INFO:     127.0.0.1:56404 - ""GET /-/static/app.css?4e362c HTTP/1.1"" 200 OK
INFO:     127.0.0.1:56406 - ""GET /-/static-plugins/datasette_vega/main.2acbb312.css HTTP/1.1"" 200 OK
INFO:     127.0.0.1:56408 - ""GET /-/static-plugins/datasette_vega/main.08f5d3d8.js HTTP/1.1"" 200 OK
INFO:     127.0.0.1:56408 - ""GET /databasetest HTTP/1.1"" 200 OK
INFO:     127.0.0.1:56408 - ""GET /-/static/app.css?4e362c HTTP/1.1"" 200 OK
INFO:     127.0.0.1:56404 - ""GET /-/static-plugins/datasette_vega/main.2acbb312.css HTTP/1.1"" 200 OK
INFO:     127.0.0.1:56406 - ""GET /-/static/codemirror-5.57.0.min.css HTTP/1.1"" 200 OK
INFO:     127.0.0.1:56410 - ""GET /-/static-plugins/datasette_vega/main.08f5d3d8.js HTTP/1.1"" 200 OK
INFO:     127.0.0.1:56414 - ""GET /-/static/codemirror-5.57.0-sql.min.js HTTP/1.1"" 200 OK
INFO:     127.0.0.1:56412 - ""GET /-/static/codemirror-5.57.0.min.js HTTP/1.1"" 200 OK
INFO:     127.0.0.1:56408 - ""GET /-/static/sql-formatter-2.3.3.min.js HTTP/1.1"" 200 OK
INFO:     127.0.0.1:56408 - ""GET /databasetest?sql=select+*+from+databasetest HTTP/1.1"" 200 OK
INFO:     127.0.0.1:56410 - ""GET /-/static/app.css?4e362c HTTP/1.1"" 200 OK
INFO:     127.0.0.1:56408 - ""GET /-/static-plugins/datasette_vega/main.2acbb312.css HTTP/1.1"" 200 OK
INFO:     127.0.0.1:56412 - ""GET /-/static/codemirror-5.57.0.min.css HTTP/1.1"" 200 OK
INFO:     127.0.0.1:56404 - ""GET /-/static/sql-formatter-2.3.3.min.js HTTP/1.1"" 200 OK
INFO:     127.0.0.1:56406 - ""GET /-/static/codemirror-5.57.0.min.js HTTP/1.1"" 200 OK
INFO:     127.0.0.1:56414 - ""GET /-/static-plugins/datasette_vega/main.08f5d3d8.js HTTP/1.1"" 200 OK
INFO:     127.0.0.1:56408 - ""GET /-/static/codemirror-5.57.0-sql.min.js HTTP/1.1"" 200 OK
INFO:     127.0.0.1:56410 - ""GET /databasetest.json?sql=select+*+from+databasetest&_shape=array&_shape=array HTTP/1.1"" 200 OK
^CINFO:     Shutting down
INFO:     Waiting for application shutdown.
INFO:     Application shutdown complete.
INFO:     Finished server process [1408482]
```

Note that there is no error if I go to http://127.0.0.1:8001/databasetest and then click on `Run SQL`.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1228/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1326349129,I_kwDOCGYnMM5PDntJ,461,Consider including animated SVG console demos,9599,simonw,open,0,,,,,1,2022-08-02T20:10:04Z,2022-08-02T20:12:14Z,,OWNER,,"I recorded this one using https://github.com/nbedos/termtosvg - with `pipx install termtosvg` and then `termtosvg` - execute demo - `exit` to save.

![sqlite-utils-insert-json](https://user-images.githubusercontent.com/9599/182464206-f4976af4-eda8-4020-8257-4ada1867fb44.svg)

```json
[
  {
    ""id"": 1,
    ""name"": ""Catimus""
  },
  {
    ""id"": 2,
    ""name"": ""Feliopia""
  }
]
```",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/461/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
779088071,MDU6SXNzdWU3NzkwODgwNzE=,54,Archive import appears to be broken on recent exports,21148,jacobian,open,0,,,,,5,2021-01-05T14:18:01Z,2023-01-04T11:06:55Z,,CONTRIBUTOR,,"I requested a Twitter export yesterday, and unfortunately they seem to have changed it such that `twitter-to-sqlite import` can't handle it anymore 😢 

So far I've ran into two issues. The first was easy to work around, but the second will take more investigation. If I can find the time I'll keep working on it and update this issue accordingly.

The issues (so far):

### 1. Data seems to have moved to a `data/` subdirectory

Running `twitter-to-sqlite import` on the raw zip file reports a bunch of ""not yet implemented"" errors, and then exits without actually importing anything:

```
❯ twitter-to-sqlite import tarchive.db twitter.zip
...
data/manifest: not yet implemented
data/account-creation-ip: not yet implemented
data/account-suspension: not yet implemented
... (dozens of more lines like this, including critical stuff like data/tweets) ...
```

(`tarchive.db` now exists, but is empty)

Workaround: unpack the zip file, and run `twitter-to-sqlite import tarchive.db path/to/archive/data`

That gets further, but:

### 2. Some schema(s?) have changed

At least, the `blocks` schema seems different now:

```
❯ twitter-to-sqlite import tarchive.db archive/data
direct-messages-group: not yet implemented
branch-links: not yet implemented
periscope-expired-broadcasts: not yet implemented
direct-messages: not yet implemented
mute: not yet implemented
Traceback (most recent call last):
  File ""/Users/jacob/Library/Caches/pypoetry/virtualenvs/jacobian-dogsheep-4AXaN4tu-py3.8/bin/twitter-to-sqlite"", line 8, in <module>
    sys.exit(cli())
  File ""/Users/jacob/Library/Caches/pypoetry/virtualenvs/jacobian-dogsheep-4AXaN4tu-py3.8/lib/python3.8/site-packages/click/core.py"", line 829, in __call__
    return self.main(*args, **kwargs)
  File ""/Users/jacob/Library/Caches/pypoetry/virtualenvs/jacobian-dogsheep-4AXaN4tu-py3.8/lib/python3.8/site-packages/click/core.py"", line 782, in main
    rv = self.invoke(ctx)
  File ""/Users/jacob/Library/Caches/pypoetry/virtualenvs/jacobian-dogsheep-4AXaN4tu-py3.8/lib/python3.8/site-packages/click/core.py"", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File ""/Users/jacob/Library/Caches/pypoetry/virtualenvs/jacobian-dogsheep-4AXaN4tu-py3.8/lib/python3.8/site-packages/click/core.py"", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File ""/Users/jacob/Library/Caches/pypoetry/virtualenvs/jacobian-dogsheep-4AXaN4tu-py3.8/lib/python3.8/site-packages/click/core.py"", line 610, in invoke
    return callback(*args, **kwargs)
  File ""/Users/jacob/Library/Caches/pypoetry/virtualenvs/jacobian-dogsheep-4AXaN4tu-py3.8/lib/python3.8/site-packages/twitter_to_sqlite/cli.py"", line 772, in import_
    archive.import_from_file(db, filepath.name, open(filepath, ""rb"").read())
  File ""/Users/jacob/Library/Caches/pypoetry/virtualenvs/jacobian-dogsheep-4AXaN4tu-py3.8/lib/python3.8/site-packages/twitter_to_sqlite/archive.py"", line 215, in import_from_file
    to_insert = transformer(data)
  File ""/Users/jacob/Library/Caches/pypoetry/virtualenvs/jacobian-dogsheep-4AXaN4tu-py3.8/lib/python3.8/site-packages/twitter_to_sqlite/archive.py"", line 115, in lists_member
    return {""lists-member"": _list_from_common(data)}
  File ""/Users/jacob/Library/Caches/pypoetry/virtualenvs/jacobian-dogsheep-4AXaN4tu-py3.8/lib/python3.8/site-packages/twitter_to_sqlite/archive.py"", line 200, in _list_from_common
    for url in block[""userListInfo""][""urls""]:
KeyError: 'urls'
```

That's as far as I got before I needed to work on something else. I'll report back if I get further!",206156866,twitter-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/54/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
268590777,MDU6SXNzdWUyNjg1OTA3Nzc=,41,Homepage should show summary of databases,9599,simonw,closed,0,,,2857392,Ship first public release,1,2017-10-26T00:18:11Z,2017-10-27T04:05:35Z,2017-10-27T04:05:35Z,OWNER,,"I sch database should have a name, optional description, download link and a summary of the tables

Flights.db
Flights and suchlike blah. 
URL? License?
577373 rows across 14 tables
airports, routes, airlines... 

Title of the homepage is derived from the databases or can be manually overridden e. “Datasets of Flights, NHS, Blah...” - or if only one database just the title of that.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/41/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
555832585,MDU6SXNzdWU1NTU4MzI1ODU=,661,"--port option to expose a port other than 8001 in ""datasette package""",134771,dvhthomas,closed,0,,,,,3,2020-01-27T21:05:56Z,2020-01-30T04:17:52Z,2020-01-29T22:46:45Z,NONE,,"I see  how to alter the port using `datasette serve -p XXX` per the docs. However, I'm packaging up to server the container on AppEngine flexible, which [requires](https://cloud.google.com/appengine/docs/flexible/custom-runtimes/build#listening_to_port_8080)  that the container is serving traffic on port 8080.

https://github.com/simonw/datasette/blob/7950105c278b140e6cb665c68b59df219870f9bc/Dockerfile#L41

Is there a way to inject a non-default port into the Dockerfile, or should I just do something like `sed` to replace 8001 with 8080 after `dataset package` has done it's thing? Thanks for the advice.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/661/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1533673397,I_kwDOBm6k_c5baf-1,1991,fts5 tables are not auto-detected and hidden,83819,keturn,open,0,,,,,0,2023-01-15T06:00:42Z,2023-01-20T04:54:24Z,,NONE,,"I set up a [Datasette instance](https://huggingface.co/spaces/Sygil/INE-dataset-explorer/tree/main) and was following the docs on full-text search.

When I used fts4, datasette automatically hid the FTS tables and added the FTS search box where appropriate, but when I changed to fts5 it no longer does either.

If I [manually set](https://huggingface.co/spaces/keturn/INED-datasette/blob/main/metadata.json#L9) `fts_table` for a view, then search does work as expected.

My table and view creation code looks like this:
```py
connection.execute(""""""CREATE TABLE IF NOT EXISTS
    captions(image_key text PRIMARY KEY, caption text NOT NULL)
"""""")
 
connection.execute(""""""CREATE VIRTUAL TABLE
    captions_fts USING
    fts5(caption, image_key UNINDEXED, content=captions)
"""""")
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1991/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1102568047,I_kwDOBm6k_c5Bt9pv,1596,Documentation page warning of changes coming in 1.0,9599,simonw,open,0,,,,,0,2022-01-13T23:26:04Z,2022-01-13T23:26:04Z,,OWNER,,I should start this relatively soon.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1596/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
817597268,MDU6SXNzdWU4MTc1OTcyNjg=,1246,Suggest for ArrayFacet possibly confused by blank values,9599,simonw,closed,0,,,,,3,2021-02-26T19:11:52Z,2021-03-01T03:46:11Z,2021-03-01T03:46:11Z,OWNER,,I sometimes don't get the suggestion for facet-by-array for columns that contain arrays. I think it may be because they have empty spaces in them - or perhaps it's because the null detection doesn't actually work.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1246/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1067771698,I_kwDOCGYnMM4_pOcy,348,Command for creating an empty database,9599,simonw,closed,0,,,7558727,3.21,6,2021-11-30T23:24:27Z,2022-01-13T07:06:59Z,2022-01-09T20:33:20Z,OWNER,,"I sometimes find the need to create an empty SQLite database file - for example if I want to enable WAL on it before using it with another script. I currently do that like this:

    sqlite3 my.db vacuum
    sqlite-utils enable-wal my.db

It would be nice if `sqlite-utils` had a convenience command for doing this.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/348/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
689847361,MDU6SXNzdWU2ODk4NDczNjE=,5,Add a context column that's not searchable,9599,simonw,closed,0,,,,,1,2020-09-01T06:13:42Z,2020-09-03T18:43:50Z,2020-09-03T18:43:50Z,MEMBER,,"I sometimes like to configure titles that are things like ""Comment on issue X"" or ""Photo in Golden Gate Park"" - these shouldn't be included in the search index but should be stored so they can be displayed to provide context.

Add a column for this - probably called `context` - and make it so it can be populated.",197431109,dogsheep-beta,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-beta/issues/5/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
962391325,MDU6SXNzdWU5NjIzOTEzMjU=,1423,Show count of facet values if ?_facet_size=max,9599,simonw,closed,0,,,,,9,2021-08-06T04:42:20Z,2021-12-15T17:48:40Z,2021-08-16T18:56:43Z,OWNER,,"I sometimes want to get a count of the values in a facet - if it's a facet of US states for example I want to know if all 50 are represented.

Idea: if `?_facet_size=max` is present, add a count to the facet heading. So on:

https://latest.datasette.io/fixtures/compound_three_primary_keys?_facet=content&_facet_size=max&_facet=pk1&_facet=pk2#facet-pk2

It could have something like this:

<img width=""658"" alt=""fixtures__compound_three_primary_keys__1_001_rows_and_NY_629_755_•_Untitled-1_—_datasette"" src=""https://user-images.githubusercontent.com/9599/128456981-1f3d5348-90f1-45cc-9287-0f7bb0015257.png"">

Note that the first column shows >1000 - because in that case we've truncated the facet calculation since the maximum allowed returned rows is 1000.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1423/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
470691999,MDU6SXNzdWU0NzA2OTE5OTk=,43,.add_column() doesn't match indentation of initial creation,9599,simonw,closed,0,,,,,3,2019-07-20T16:33:10Z,2019-07-23T13:09:11Z,2019-07-23T13:09:05Z,OWNER,,"I spotted a table which was created once and then had columns added to it and the formatted SQL looks like this:

```sql
CREATE TABLE [records] (
   [type] TEXT,
   [sourceName] TEXT,
   [sourceVersion] TEXT,
   [unit] TEXT,
   [creationDate] TEXT,
   [startDate] TEXT,
   [endDate] TEXT,
   [value] TEXT,
   [metadata_Health Mate App Version] TEXT,
   [metadata_Withings User Identifier] TEXT,
   [metadata_Modified Date] TEXT,
   [metadata_Withings Link] TEXT,
   [metadata_HKWasUserEntered] TEXT
, [device] TEXT, [metadata_HKMetadataKeyHeartRateMotionContext] TEXT, [metadata_HKDeviceManufacturerName] TEXT, [metadata_HKMetadataKeySyncVersion] TEXT, [metadata_HKMetadataKeySyncIdentifier] TEXT, [metadata_HKSwimmingStrokeStyle] TEXT, [metadata_HKVO2MaxTestType] TEXT, [metadata_HKTimeZone] TEXT, [metadata_Average HR] TEXT, [metadata_Recharge] TEXT, [metadata_Lights] TEXT, [metadata_Asleep] TEXT, [metadata_Rating] TEXT, [metadata_Energy Threshold] TEXT, [metadata_Deep Sleep] TEXT, [metadata_Nap] TEXT, [metadata_Edit Slots] TEXT, [metadata_Tags] TEXT, [metadata_Daytime HR] TEXT)
```

It would be nice if the columns that were added later matched the indentation of the initial columns.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/43/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1988525411,I_kwDOCGYnMM52hn1j,603,Pyhton 3.12 Bug report,1324252,constantinedev,open,0,,,,,1,2023-11-10T22:57:48Z,2023-12-08T05:10:31Z,,NONE,,"I start with new python3 verison 3.12.0
Also have the error where connect DataBase

```
Traceback (most recent call last):
  File ""/home/t/Development/python/FKPJ/ClinicSYS/run.py"", line 1, in <module>
    import re, os, io, json, sqlite_utils, requests, pytz, logging
  File ""/home/t/.local/lib/python3.12/site-packages/sqlite_utils/__init__.py"", line 1, in <module>
    from .db import Database
  File ""/home/t/.local/lib/python3.12/site-packages/sqlite_utils/db.py"", line 277, in <module>
    class Database:
  File ""/home/t/.local/lib/python3.12/site-packages/sqlite_utils/db.py"", line 306, in Database
    filename_or_conn: Optional[Union[str, pathlib.Path, sqlite3.Connection]] = None,
                                                        ^^^^^^^^^^^^^^^^^^
```
This bug come from `sqlite-utils` since's v3.33.
Anyone get the same ?

As well now of the resolved plan just keep the sqlite-utils version in python3.12 with v3.32.1 [tested]
but where are the sqlite3.Connection problem.... 

This won't happen on python version down to 3.11[tested]
Just the python3.12.0, I have test this error are come from the sqlite3 connection
The error say from `sqlite_utils` and with the sqlite3 Connection, what can I do.

Let fix together.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/603/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
787173276,MDU6SXNzdWU3ODcxNzMyNzY=,1193,Research plugin hook for alternative database backends,9599,simonw,open,0,,,,,1,2021-01-15T20:27:50Z,2021-03-12T01:01:54Z,,OWNER,,"I started exploring what Datasette would like running against PostgreSQL in #670 and @dazzag24 did some work on Parquet described in #657.

I had initially thought this was WAY too much additional complexity, but I'm beginning to think that the `Database` class may be small enough that having it abstract away the details of running queries against alternative database backends could be feasible.

A bigger issue is SQL generation, but I realized that most of Datasette's SQL generation code exists just in the `TableView` class that runs the table page. If this was abstracted into some kind of SQL builder that could be then customized per-database it might be reasonable to get it working.

Very unlikely for this to make it into Datasette 1.0, but maybe this would be the defining feature of Datasette 2.0?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1193/reactions"", ""total_count"": 3, ""+1"": 3, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
521329771,MDU6SXNzdWU1MjEzMjk3NzE=,628,Render jinja2 templates in async mode,9599,simonw,closed,0,,,,,2,2019-11-12T05:01:55Z,2019-11-14T23:28:09Z,2019-11-14T23:14:24Z,OWNER,,"I started playing with this in #404 and got good results but it didn't work in Python 3.5. As of #627 I don't support 3.5 any more so this can go ahead.

Rendering templates in async mode will mean that template plugins can include async code... which opens the door to custom template functions that execute SQL queries!",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/628/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
802583450,MDU6SXNzdWU4MDI1ODM0NTA=,226,3.4 release is broken - includes a rogue line,9599,simonw,closed,0,,,,,0,2021-02-06T02:08:01Z,2021-02-06T02:10:26Z,2021-02-06T02:10:26Z,OWNER,,"I started seeing weird errors, caused by this line: https://github.com/simonw/sqlite-utils/blob/f8010ca78fed8c5fca6cde19658ec09fdd468420/sqlite_utils/cli.py#L1-L3

That was added by accident in 1b666f9315d4ea6bb332b2e75e48480c26100199

I'm surprised the tests didn't catch this!",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/226/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1193090967,I_kwDOBm6k_c5HHR-X,1699,Proposal: datasette query,25778,eyeseast,open,0,,,,,6,2022-04-05T12:36:43Z,2022-04-11T01:32:12Z,,CONTRIBUTOR,,"I started sketching out a plugin to add a `datasette query` subcommand to export data from the command line. This is based on discussions in #1356 and #1605. Before I get too far down this rabbit hole, I figure it's worth getting some feedback here (unless this should happen in `Discussions`). Here's what I'm thinking:

At its most basic, it will write the results of a query to STDOUT.

```sh
datasette query -d data.db 'select * from data' > results.json
```

This isn't much improvement over using [sqlite-utils](https://github.com/simonw/sqlite-utils). To make better use of datasette and its ecosystem, run `datasette query` using a canned query defined in a `metadata.yml` file.

For example, using the metadata file from [alltheplaces-datasette](https://github.com/eyeseast/alltheplaces-datasette/blob/main/metadata.yml):

```sh
cd alltheplaces-datasette
datasette query -d alltheplaces.db -m metadata.yml count_by_spider
```

That query would be good to get as CSV, and we can auto-discover metadata and databases in the current directory:

```sh
cd alltheplaces-datasette
datasette query count_by_spider -f csv
```

In this case, `count_by_spider` is a canned query defined on the `alltheplaces` database. If the same query is defined on multiple databases or its otherwise unclear which database `query` should use, pass the `-d` or `--database` option.

If a query takes parameters, I can pass them in at runtime, using the `--param` or `-p` option:

```sh
datasette query -d data.db -p value something 'select * from neighborhoods where some_column = :value'
```

I'm very interested in feedback on this, including whether it should be a plugin or in Datasette core. (I don't have a strong opinion about this, but I'm prototyping it as a plugin to start.)",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1699/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
299760684,MDU6SXNzdWUyOTk3NjA2ODQ=,185,Metadata should be a nested arbitrary KV store,222245,carlmjohnson,open,0,,,,,12,2018-02-23T16:02:07Z,2019-05-13T18:33:33Z,,NONE,,"I started using the metadata feature and was surprised to find that values are not inherited from the root object down to specific databases and tables. This makes metadata much less useful and requires a lot of pointless duplication.

Ideally, metadata should allow arbitrary key-value pairs, and there should be a way of accessing metadata either in an inherited or non-inherited manner. Something like `metadata.page.key` vs. `metadata.this.key` might work as an interface.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/185/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1021550542,I_kwDOBm6k_c4845_O,1482,Support Python 3.10,9599,simonw,closed,0,,,,,2,2021-10-09T00:30:52Z,2021-10-24T22:21:40Z,2021-10-24T22:19:55Z,OWNER,,"I started work on this in #1481 where I found a Python 3.10 bug that needs a workaround in Janus, see:

- https://github.com/aio-libs/janus/issues/358

This is a tracking issue for anything else that shows up.

This is also needed for the Homebrew package to upgrade to 3.10:

- https://github.com/Homebrew/homebrew-core/pull/86932",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1482/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1292368833,I_kwDOBm6k_c5NB_vB,1764,Keep track of config_dir in directory mode (for plugins),25778,eyeseast,closed,0,,,,,0,2022-07-03T16:57:49Z,2022-07-18T01:12:45Z,2022-07-18T01:12:45Z,CONTRIBUTOR,,"I started working on using `config_dir` with my [datasette-query-files plugin](https://github.com/eyeseast/datasette-query-files) and realized Datasette doesn't actually hold onto the `config_dir` argument. It gets used in `__init__` but then forgotten. It would be nice to be able to use it in plugins, though.

Here's the reference issue: https://github.com/eyeseast/datasette-query-files/issues/4
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1764/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
721068929,MDU6SXNzdWU3MjEwNjg5Mjk=,1020,Method for datasette.client() to forward on authentication,9599,simonw,open,0,,,,,6,2020-10-14T01:47:49Z,2020-10-19T22:45:01Z,,OWNER,,"I stumbled into this while working on Dogsheep Beta: the requests it re-dispatched through `TableView` did not carry authentication cookies, and since this was against a private instance they were thus denied.

https://github.com/dogsheep/dogsheep-beta/blob/bed9df2b3ef68189e2e445427721a28f4e9b4887/dogsheep_beta/__init__.py#L223-L231

This made me think that `datasette.client.get()` (which Dogsheep Beta will start using shortly) could benefit from some kind of utility mechanism for passing through the cookies and general authenticated state from the current request.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1020/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
694500679,MDU6SXNzdWU2OTQ1MDA2Nzk=,17,"Rename ""table"" to ""type""",9599,simonw,closed,0,,,,,2,2020-09-06T19:34:41Z,2020-09-09T03:03:22Z,2020-09-09T03:03:22Z,MEMBER,,"I think ""table"" is the wrong name for the concept I'm using it for here.

Two reasons: firstly, `table` is a reserved word in SQLite. More importantly, it turns out there's not a direct mapping from tables to types of search result. In particular, for GitHub I ended up having two different ""tables"" of repositories - one for repos created by me, another for repos that I have starred.",197431109,dogsheep-beta,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-beta/issues/17/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
849543502,MDU6SXNzdWU4NDk1NDM1MDI=,1289,Speed up tests with pytest-xdist,9599,simonw,closed,0,,,,,3,2021-04-03T00:47:39Z,2021-04-03T03:42:28Z,2021-04-03T03:42:28Z,OWNER,,"I think I can get this working for almost every test, then use the pattern in https://github.com/pytest-dev/pytest-xdist/issues/385#issuecomment-444545641 to opt specific tests out of being run in parallel.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1289/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1251700382,I_kwDOBm6k_c5Km26e,1750,Allow `label_column` to specify array of columns,408765,knutwannheden,open,0,,,,,0,2022-05-28T18:45:48Z,2022-05-28T18:45:48Z,,NONE,,"I think it would be great if the Datasette metadata would allow the `label_column` table key to list multiple columns. Something like:
```json
    ""tables"": {
        ""person"": {
            ""label_column"": [""first_name"", ""last_name""]
        },
```
It would even be interesting with a ""label expression"" similar to a Python f-string. E.g. `{row.last_name}, {row.first_name}`.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1750/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
516874735,MDU6SXNzdWU1MTY4NzQ3MzU=,613,Basic join support for table view,9599,simonw,open,0,,,,,1,2019-11-03T19:12:53Z,2019-11-03T19:14:01Z,,OWNER,,"I think it would be possible to support basic foreign key joins on the table page.

The user could specify columns that should result in a join (from a set of suggestions similar to how facets work right now) and they could then be passed as `?_join=city_id`  arguments.

This feature will make a lot of sense when combined with the ability to show / hide / customize columns, see #292",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/613/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
306811513,MDU6SXNzdWUzMDY4MTE1MTM=,186,proposal new option to disable user agents cache,47107,stefanocudini,closed,0,,,,,3,2018-03-20T10:42:20Z,2018-03-21T09:07:22Z,2018-03-21T01:28:31Z,NONE,,"I think it would be very useful for debugging an option of adding headers to http replies

```
Cache-Control: no-cache 
```

especially in the html output",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/186/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
745393298,MDU6SXNzdWU3NDUzOTMyOTg=,52,Discussion: Adding support for fetching only fresh tweets,4169772,fatihky,closed,0,,,,,1,2020-11-18T07:01:48Z,2020-11-18T07:12:45Z,2020-11-18T07:12:45Z,NONE,,I think it'd be very useful if this tool has an option like `--incremental` to fetch only newer tweets. This way operations could complete very fast in sequential runs. I'd want to try to implement this feature if it seems OK for this tool's purpose. ,206156866,twitter-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/52/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1425682079,I_kwDOBm6k_c5U-i6f,1865,Stop syncing main to master,9599,simonw,closed,0,,,,,1,2022-10-27T13:55:38Z,2022-10-27T13:58:27Z,2022-10-27T13:56:13Z,OWNER,,"I think it's been long enough now that I can drop the code that syncs the main branch to master.

I originally added this for people who might be using `datasette publish ... --branch master` - which might only have been me anyway!",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1865/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1205867842,I_kwDODtX3eM5H4BVC,4,Retrieve the top-level story for a comment,1755789,telotortium,open,0,,,,,0,2022-04-15T20:25:39Z,2022-04-15T20:25:39Z,,NONE,,"I think that each comment inserted into the database should include a column `onstory` that contains the ID of the story on which the comment was made. This is exactly equivalent to the link after ""on:"" at the top of an HN comment page ([example](https://news.ycombinator.com/item?id=18358028)). We could do this either by directly retrieving the HTML page and using Beautiful Soup to find that link, or alternatively recurse up the tree in the Firebase API using the `parent` field (probably using `functools.lru_cache` in case a person has commented a bunch of times on the same story).",248903544,hacker-news-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/hacker-news-to-sqlite/issues/4/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
459882902,MDU6SXNzdWU0NTk4ODI5MDI=,526,Stream all results for arbitrary SQL and canned queries,50578294,matej-fr,open,0,,,,,23,2019-06-24T13:09:45Z,2022-09-28T04:01:25Z,,NONE,,"I think that there is a difficulty with canned queries.

When I want to stream all results of a canned query TwoDays I get only first 1.000 records.

Example:
`http://myserver/history_sample/two_days.csv?_stream=on`

returns only first 1.000 records.

If I do the same with the whole database i.e.
`http://myserver/history_sample/database.csv?_stream=on`

I get correctly all records.

Any ideas?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/526/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
572896293,MDU6SXNzdWU1NzI4OTYyOTM=,687,Expand plugins documentation to multiple pages,9599,simonw,closed,0,,,5533512,Datasette 0.45,11,2020-02-28T17:26:21Z,2020-06-22T03:55:20Z,2020-06-22T03:53:54Z,OWNER,,"I think the plugins docs need to extend beyond a single page now. I want to add a whole section on writing tests for plugins, showing how `httpx` can be used as seen in https://github.com/simonw/datasette-atom/issues/3 and suchlike.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/687/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1823393475,I_kwDOBm6k_c5srsbD,2119,"database color shows only on index page, not other pages",9599,simonw,closed,0,,,3268330,Datasette 1.0,3,2023-07-27T00:19:39Z,2023-08-11T05:25:45Z,2023-08-11T05:16:24Z,OWNER,,"I think this has been a bug for a long time.

https://latest.datasette.io/ currently shows:

<img width=""827"" alt=""image"" src=""https://github.com/simonw/datasette/assets/9599/fcd08d0c-5b07-40be-bc9f-3ba9199ad858"">

Those colors are based on a hash of the database name. But when you click through to https://latest.datasette.io/fixtures

<img width=""692"" alt=""image"" src=""https://github.com/simonw/datasette/assets/9599/7c9dfd66-06e8-47fc-9ed7-ce425da22cb4"">

It's red on all sub-pages too.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2119/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
753668177,MDU6SXNzdWU3NTM2NjgxNzc=,1116,GENERATED column support,2789593,nattaylor,closed,0,,,,,9,2020-11-30T17:33:47Z,2020-11-30T21:29:59Z,2020-11-30T21:29:59Z,NONE,,"I think this is a feature request... perhaps I should just try to contribute it myself, but thought I'd check in case support is planned already.

For a table with the following schema, datasette 0.51.1 doesn't pick up the GENERATED columns and the column list only contains `(rowid, body)`  If I edit the SQL and select the generated columns, it will happily show them.

At first glance it appears that [`def table_column_details(conn, table):`](https://github.com/simonw/datasette/blob/4777362bf2692bc72b221ec47c3e6216151d1b89/datasette/utils/__init__.py#L575) would have to be refactored to use a different methodology to get the columns, since `PRAGMA table_info(deeds);` returns just `0|body|TEXT|0||0` so maybe it wouldn't be worth it.

```
CREATE TABLE deeds (
    body TEXT,
    id INT GENERATED ALWAYS AS (json_extract(body, '$.id')) STORED,
    consideration INT GENERATED ALWAYS AS (json_extract(body, '$.consideration')) STORED
);
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1116/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
569268612,MDU6SXNzdWU1NjkyNjg2MTI=,679,Release 0.36,9599,simonw,closed,0,,,,,2,2020-02-22T02:41:01Z,2020-02-22T03:52:13Z,2020-02-22T03:52:13Z,OWNER,,"I think we have enough changes to warrant a release - and I want to take advantage of the changes to the `prepare_connection()` plugin hook in #678

Changes since 0.35 so far: https://github.com/simonw/datasette/compare/0.35...be2265b0e811d0ac2875c2f748125c17b0f9289e

- [x] Update ecosystem page
- [x] Write release notes
- [x] Ship the release",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/679/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
594189527,MDU6SXNzdWU1OTQxODk1Mjc=,717,See if I can get Datasette working on Zeit Now v2,9599,simonw,closed,0,,,,,10,2020-04-05T00:56:48Z,2020-04-06T22:47:22Z,2020-04-06T22:47:21Z,OWNER,,"I thought this was impossible because AWS Lambda doesn't ship the `sqlite3` standard library module... but apparenttly that's not the case on Now v2 any more!

https://now-2-python-versions-ks69olzpi.now.sh/api

```
 _________________________________________________________________________________________________________________________________________________________________ 
/ Hello from Python from a ZEIT Now Serverless Function! Version is 3.6.10 (default, Mar 10 2020, 22:54:43)                                                       \
\ [GCC 4.8.3 20140911 (Red Hat 4.8.3-9)], sqlite3 module = <module 'sqlite3' from '/var/lang/lib/python3.6/sqlite3/__init__.py'>, sqlite3 version = [('3.7.17',)] /
 ----------------------------------------------------------------------------------------------------------------------------------------------------------------- 
     \   ^__^
      \  (oo)\_______
         (__)\       )\/\
           ||----w |
           ||     ||
```
That's from shipping this code as `api/index.py`:
```python
from http.server import BaseHTTPRequestHandler
from cowpy import cow
import sys


try:
    import sqlite3
except ImportError:
    sqlite3 = None


class handler(BaseHTTPRequestHandler):
    def do_GET(self):
        self.send_response(200)
        self.send_header(""Content-type"", ""text/plain"")
        self.end_headers()
        message = cow.Cowacter().milk(
            ""Hello from Python from a ZEIT Now Serverless Function! Version is {}, sqlite3 module = {}, sqlite3 version = {}"".format(
                sys.version, sqlite3, sqlite3.connect("":memory:"").execute(""select sqlite_version()"").fetchall()
            )
        )
        self.wfile.write(message.encode())
        return
```
Now v2 supports ASGI so this might be possible without too much work: https://zeit.co/docs/runtimes#advanced-usage/advanced-python-usage/asynchronous-server-gateway-interface",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/717/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
564833696,MDU6SXNzdWU1NjQ4MzM2OTY=,670,Prototoype for Datasette on PostgreSQL,9599,simonw,open,0,,,,,15,2020-02-13T17:17:55Z,2023-11-17T15:32:21Z,,OWNER,,"I thought this would never happen, but now that I'm deep in the weeds of running SQLite in production for Datasette Cloud I'm starting to reconsider my policy of only supporting SQLite.

Some of the factors making me think PostgreSQL support could be worth the effort:
- Serverless. I'm getting increasingly excited about writable-database use-cases for Datasette. If it could talk to PostgreSQL then users could easily deploy it on Heroku or other serverless providers that can talk to a managed RDS-style PostgreSQL.
- Existing databases. Plenty of organizations have PostgreSQL databases. They can export to SQLite using [db-to-sqlite](https://github.com/simonw/db-to-sqlite) but that's a pretty big barrier to getting started - being able to run `datasette postgresql://connection-string` and start trying it out would be a massively better experience.
- Data size. I keep running into use-cases where I want to run Datasette against many GBs of data. SQLite can do this but PostgreSQL is much more optimized for large data, especially given the existence of tools like Citus.
- Marketing. Convincing people to trust their data to SQLite is potentially a big barrier to adoption. Even if I've convinced myself it's trustworthy I still have to convince everyone else.
- It might not be that hard? If this required a ground-up rewrite it wouldn't be worth the effort, but I have a hunch that it may not be too hard - most of the SQL in Datasette should work on both databases since it's almost all portable SELECT statements. If Datasette did DML this would be a lot harder, but it doesn't.
- Plugins! This feels like a natural surface for a plugin - at which point people could add MySQL support and suchlike in the future.

The above reasons feel strong enough to justify a prototype.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/670/reactions"", ""total_count"": 19, ""+1"": 14, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 5, ""rocket"": 0, ""eyes"": 0}",,
1115435536,I_kwDOBm6k_c5CfDIQ,1614,Try again with SQLite codemirror support,9599,simonw,open,0,,,,,1,2022-01-26T20:05:20Z,2022-12-23T21:27:10Z,,OWNER,,"I tried and failed to implement autocomplete a while ago. Relevant code:

https://github.com/codemirror/legacy-modes/blob/8f36abca5f55024258cd23d9cfb0203d8d244f0d/mode/sql.js#L335

Sounds like upgrading to CodeMirror 6 ASAP would be worthwhile since it has better accessibility and touch screen support: https://codemirror.net/6/",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1614/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
716756082,MDU6SXNzdWU3MTY3NTYwODI=,996,Better handling of multiple matching template wildcard paths,9599,simonw,closed,0,,,5971510,Datasette 0.50,3,2020-10-07T18:25:40Z,2020-10-08T23:55:41Z,2020-10-07T22:51:17Z,OWNER,,"I tried building this:

    templates/pages/{topic}.html
    templates/pages/{topic}/{slug}.html

And it didn't work - hits to /foo/bar which should have been rendered by the `{slug}.html` template were instead rendered by the top level `{topic.html}` template.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/996/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
811367257,MDU6SXNzdWU4MTEzNjcyNTc=,1231,Race condition errors in new refresh_schemas() mechanism,9599,simonw,closed,0,,,,,11,2021-02-18T18:49:54Z,2021-07-16T19:44:59Z,2021-07-16T19:44:59Z,OWNER,,I tried running a Locust load test against Datasette and hit an error message about a failure to create tables because they already existed. I think this means there are race conditions in the new `refresh_schemas()` mechanism added in #1150.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1231/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1087181951,I_kwDOBm6k_c5AzRR_,1576,Traces should include SQL executed by subtasks created with `asyncio.gather`,9599,simonw,closed,0,,,3268330,Datasette 1.0,12,2021-12-22T20:52:02Z,2022-02-05T05:21:35Z,2022-02-05T05:19:53Z,OWNER,,"I tried running some parallel SQL queries using `asyncio.gather()` but the SQL that was executed didn't show up in the trace rendered by https://datasette.io/plugins/datasette-pretty-traces

I realized that was because traces are keyed against the current task ID, which changes when a sub-task is run using `asyncio.gather` or similar.

The faceting and suggest faceting queries are missing from this trace:

![image](https://user-images.githubusercontent.com/9599/147153855-2d611f07-922a-4d18-9e6e-4be89e010dc4.png)

> The reason they aren't showing up in the traces is that traces are stored just for the currently executing `asyncio` task ID: https://github.com/simonw/datasette/blob/ace86566b28280091b3844cf5fbecd20158e9004/datasette/tracer.py#L13-L25
>
> This is so traces for other incoming requests don't end up mixed together. But there's no current mechanism to track async tasks that are effectively ""child tasks"" of the current request, and hence should be tracked the same.
>
> https://stackoverflow.com/a/69349501/6083 suggests that you pass the task ID as an argument to the child tasks that are executed using `asyncio.gather()` to work around this kind of problem.

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1518#issuecomment-999870993_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1576/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
314771615,MDU6SXNzdWUzMTQ3NzE2MTU=,218,"Support custom unit display in order to handle ""$10,000""",9599,simonw,open,0,,,,,0,2018-04-16T18:39:31Z,2018-07-10T17:45:38Z,,OWNER,,"I tried to get Datasette to display `$10,000` using the new units support but we currently only display units as a suffix:

https://github.com/simonw/datasette/blob/10a34f995c70daa37a8a2aa02c3135a4b023a24c/datasette/app.py#L563-L572

It would be neat if there was a mechanism for specifying a custom unit display - maybe something like this:

```
{
    ""custom_units"": {
        ""us_dollar"": {
            ""unit"": ""us_dollar = [] = $"",
            ""format"": ""${:,}""
        }
    }
}
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/218/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
924748955,MDU6SXNzdWU5MjQ3NDg5NTU=,1380,Serve all db files in a folder,193463,stratosgear,open,0,,,,,5,2021-06-18T10:03:32Z,2021-11-13T08:09:11Z,,NONE,,"I tried to get the `serve` command to serve all the .db files in the `/mnt` folder but is seems that the server does not refresh the list of files.

In more detail:

* Starting datasette as a docker container with: 
```
docker run -p 8001:8001 -v `pwd`:/mnt \
    datasetteproject/datasette \
    datasette -p 8001 -h 0.0.0.0 /mnt
```

* Datasette correctly serves all the *.db files found in the /mnt folder 
* When the server is running, if I copy a new file in the $PWD folder, Datasette does not seem to see the new files, forcing me to restart Docker.

Is there an option/setting that I overlooked, or is this something missing?

BTW, the `--reload` setting, although at first glance is what you think you need, does not seem to do anything in regards of seeing all *.db files.

Thanks!",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1380/reactions"", ""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
951817328,MDU6SXNzdWU5NTE4MTczMjg=,12,403 when getting token,285352,treyhunner,open,0,,,,,1,2021-07-23T18:43:26Z,2021-10-12T18:31:57Z,,NONE,,"I tried to use https://your-foursquare-oauth-token.glitch.me/ to get my Swarm auth token and got a 403 after I clicked the Allow button:

![image](https://user-images.githubusercontent.com/285352/126826478-60e53614-263d-40bb-9f1d-c1a676644eb0.png)

I'm not sure if this is the right repo to report this in",205429375,swarm-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/12/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
663228985,MDU6SXNzdWU2NjMyMjg5ODU=,904,"datasette.urls.table() / .instance() / .database() methods for constructing URLs, also exposed to templates",9599,simonw,closed,0,,,6026070,0.51,11,2020-07-21T18:42:52Z,2020-10-23T19:44:05Z,2020-10-20T00:51:51Z,OWNER,,"I tried using this block of template in a plugin and got an error:
```html
{% block nav %}
    <p class=""crumbs"">
        <a href=""{{ base_url }}"">home</a> /
        <a href=""{{ database_url(database) }}"">{{ database }}</a> /
        <a href=""{{ database_url(database) }}/{{ table|quote_plus }}"">{{ table }}</a>
    </p>
    {{ super() }}
{% endblock %}
```
Error: `'database_url' is undefined`

That's because `database_url` is only made available by the BaseView template here:

https://github.com/simonw/datasette/blob/d6e03b04302a0852e7133dc030eab50177c37be7/datasette/views/base.py#L110-L125",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/904/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
397129564,MDU6SXNzdWUzOTcxMjk1NjQ=,397,Update official datasetteproject/datasette Docker container to SQLite 3.26.0,43564,claes,closed,0,,,,,3,2019-01-08T22:51:50Z,2019-01-11T01:25:33Z,2019-01-11T00:56:18Z,NONE,,"I try to start datasette on a database that contains the below view

It fails in a way that makes me think it does not support the window functions SQL syntax.


```
create view general_ledger as
select transactions.account_number, strftime(""%Y-%m-%d"", verifications.verification_date) as verification_date, verifications.verification_number, verifications.verification_text, 
case when transactions.centi_amount >= 0 and verifications.verification_number > 0 then printf(""%.2f"", (transactions.centi_amount/100.0))
end as debit,
case when transactions.centi_amount <= 0 and verifications.verification_number > 0 then printf(""%.2f"", (transactions.centi_amount/100.0)) 
end as credit,
printf(""%.2f"", sum(transactions.centi_amount) over (partition by transactions.account_number 
    order by verifications.verification_number range between unbounded preceding and current row)/100.0)
from verifications inner join transactions on transactions.verification_id = verifications.id 
order by transactions.account_number, verifications.verification_number;
```

```
docker run -p 8001:8001 -v `pwd`:/mnt     datasetteproject/datasette     datasette -p 8001 -h 0.0.0.0 /mnt/ledger.db
Serve! files=('/mnt/ledger.db',) on port 8001
Traceback (most recent call last):
  File ""/usr/local/bin/datasette"", line 11, in <module>
    sys.exit(cli())
  File ""/usr/local/lib/python3.6/site-packages/click/core.py"", line 722, in __call__
    return self.main(*args, **kwargs)
  File ""/usr/local/lib/python3.6/site-packages/click/core.py"", line 697, in main
    rv = self.invoke(ctx)
  File ""/usr/local/lib/python3.6/site-packages/click/core.py"", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File ""/usr/local/lib/python3.6/site-packages/click/core.py"", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File ""/usr/local/lib/python3.6/site-packages/click/core.py"", line 535, in invoke
    return callback(*args, **kwargs)
  File ""/usr/local/lib/python3.6/site-packages/datasette/cli.py"", line 375, in serve
    ds.inspect()
  File ""/usr/local/lib/python3.6/site-packages/datasette/app.py"", line 308, in inspect
    ""views"": inspect_views(conn),
  File ""/usr/local/lib/python3.6/site-packages/datasette/inspect.py"", line 30, in inspect_views
    return [v[0] for v in conn.execute('select name from sqlite_master where type = ""view""')]
sqlite3.DatabaseError: malformed database schema (general_ledger) - near ""over"": syntax error

```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/397/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1010112818,I_kwDOBm6k_c48NRky,1479,"Win32 ""used by another process"" error with datasette publish",76450761,kirajano,open,0,,,,,7,2021-09-28T19:12:00Z,2023-09-07T02:14:16Z,,NONE,,"I unfortunately was not successful to deploy to fly.io. Please see the details above of the three scenarios that I took. I am also new to datasette.

Failed to deploy. Attaching logs:
1. Tried with an app created via `flyctl apps create frosty-fog-8565` and the ran `datasette publish fly covid.db --app frosty-fog-8565` 
``` 
Deploying frosty-fog-8565
==> Validating app configuration
--> Validating app configuration done
Services
TCP 80/443 ⇢ 8080

Error error connecting to docker: An unknown error occured.

Traceback (most recent call last):
  File ""c:\users\grott\anaconda3\lib\runpy.py"", line 193, in _run_module_as_main
    ""__main__"", mod_spec)
  File ""c:\users\grott\anaconda3\lib\runpy.py"", line 85, in _run_code
    exec(code, run_globals)
  File ""C:\Users\grott\Anaconda3\Scripts\datasette.exe\__main__.py"", line 7, in <module>
  File ""c:\users\grott\anaconda3\lib\site-packages\click\core.py"", line 829, in __call__
    return self.main(*args, **kwargs)
  File ""c:\users\grott\anaconda3\lib\site-packages\click\core.py"", line 782, in main
    rv = self.invoke(ctx)
  File ""c:\users\grott\anaconda3\lib\site-packages\click\core.py"", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File ""c:\users\grott\anaconda3\lib\site-packages\click\core.py"", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File ""c:\users\grott\anaconda3\lib\site-packages\click\core.py"", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File ""c:\users\grott\anaconda3\lib\site-packages\click\core.py"", line 610, in invoke
    return callback(*args, **kwargs)
  File ""c:\users\grott\anaconda3\lib\site-packages\datasette_publish_fly\__init__.py"", line 156, in fly
    ""--remote-only"",
  File ""c:\users\grott\anaconda3\lib\contextlib.py"", line 119, in __exit__
    next(self.gen)
  File ""c:\users\grott\anaconda3\lib\site-packages\datasette\utils\__init__.py"", line 451, in temporary_docker_directory
    tmp.cleanup()
  File ""c:\users\grott\anaconda3\lib\tempfile.py"", line 811, in cleanup
    _shutil.rmtree(self.name)
  File ""c:\users\grott\anaconda3\lib\shutil.py"", line 516, in rmtree
    return _rmtree_unsafe(path, onerror)
  File ""c:\users\grott\anaconda3\lib\shutil.py"", line 395, in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
  File ""c:\users\grott\anaconda3\lib\shutil.py"", line 404, in _rmtree_unsafe
    onerror(os.rmdir, path, sys.exc_info())
  File ""c:\users\grott\anaconda3\lib\shutil.py"", line 402, in _rmtree_unsafe
    os.rmdir(path)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\grott\\AppData\\Local\\Temp\\tmpgcm8cz66\\frosty-fog-8565'
```

2. Tried also with an app that gets autogenerate when running `flyctl launch`. This also generates the .toml file. Ran then `datasette publish fly covid.db --app dark-feather-168` **but different error now**
```Deploying dark-feather-168
==> Validating app configuration

Error not possible to validate configuration: server returned Post ""https://api.fly.io/graphql"": unexpected EOF

Traceback (most recent call last):
  File ""c:\users\grott\anaconda3\lib\runpy.py"", line 193, in _run_module_as_main        
    ""__main__"", mod_spec)
    exec(code, run_globals)
  File ""C:\Users\grott\Anaconda3\Scripts\datasette.exe\__main__.py"", line 7, in <module>
  File ""c:\users\grott\anaconda3\lib\site-packages\click\core.py"", line 829, in __call__
    return self.main(*args, **kwargs)
  File ""c:\users\grott\anaconda3\lib\site-packages\click\core.py"", line 782, in main
    rv = self.invoke(ctx)
  File ""c:\users\grott\anaconda3\lib\site-packages\click\core.py"", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File ""c:\users\grott\anaconda3\lib\site-packages\click\core.py"", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File ""c:\users\grott\anaconda3\lib\site-packages\click\core.py"", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File ""c:\users\grott\anaconda3\lib\site-packages\click\core.py"", line 610, in invoke
    return callback(*args, **kwargs)
  File ""c:\users\grott\anaconda3\lib\site-packages\datasette_publish_fly\__init__.py"", line 156, in fly
    ""--remote-only"",
  File ""c:\users\grott\anaconda3\lib\contextlib.py"", line 119, in __exit__
    next(self.gen)
  File ""c:\users\grott\anaconda3\lib\site-packages\datasette\utils\__init__.py"", line 451, in temporary_docker_directory
    tmp.cleanup()
  File ""c:\users\grott\anaconda3\lib\tempfile.py"", line 811, in cleanup
    _shutil.rmtree(self.name)
  File ""c:\users\grott\anaconda3\lib\shutil.py"", line 516, in rmtree
    return _rmtree_unsafe(path, onerror)
  File ""c:\users\grott\anaconda3\lib\shutil.py"", line 395, in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
  File ""c:\users\grott\anaconda3\lib\shutil.py"", line 404, in _rmtree_unsafe
    onerror(os.rmdir, path, sys.exc_info())
  File ""c:\users\grott\anaconda3\lib\shutil.py"", line 402, in _rmtree_unsafe
    os.rmdir(path)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\grott\\AppData\\Local\\Temp\\tmpnoyewcre\\dark-feather-168'
```

These are also the contents of the generated **.toml file** in 2 scenario:

```
# fly.toml file generated for dark-feather-168 on 2021-09-28T20:35:44+02:00

app = ""dark-feather-168""

kill_signal = ""SIGINT""
kill_timeout = 5
processes = []

[env]

[experimental]
  allowed_public_ports = []
  auto_rollback = true

[[services]]
  http_checks = []
  internal_port = 8080
  processes = [""app""]
  protocol = ""tcp""
  script_checks = []

  [services.concurrency]
    hard_limit = 25
    soft_limit = 20
    type = ""connections""

  [[services.ports]]
    handlers = [""http""]
    port = 80

  [[services.ports]]
    handlers = [""tls"", ""http""]
    port = 443

  [[services.tcp_checks]]
    grace_period = ""1s""
    interval = ""15s""
    restart_limit = 6
    timeout = ""2s""
```

3. But also trying `datasette package covid.db` to create a local DOCKERFILE to later try to push it via `flyctl deploy` fails as well.

```[+] Building 147.3s (11/11) FINISHED
 => [internal] load build definition from Dockerfile                                                             0.2s 
 => => transferring dockerfile: 396B                                                                             0.0s 
 => [internal] load .dockerignore                                                                                0.1s 
 => => transferring context: 2B                                                                                  0.0s 
 => [internal] load metadata for docker.io/library/python:3.8                                                    4.7s 
 => [auth] library/python:pull token for registry-1.docker.io                                                    0.0s 
 => [internal] load build context                                                                                0.1s 
 => => transferring context: 82.37kB                                                                             0.0s 
 => [1/5] FROM docker.io/library/python:3.8@sha256:530de807b46a11734e2587a784573c12c5034f2f14025f838589e6c0e3  108.3s 
 => => resolve docker.io/library/python:3.8@sha256:530de807b46a11734e2587a784573c12c5034f2f14025f838589e6c0e3b5  0.0s 
 => => sha256:56182bcdf4d4283aa1f46944b4ef7ac881e28b4d5526720a4e9ba03a4730846a 2.22kB / 2.22kB                   0.0s 
 => => sha256:955615a668ce169f8a1443fc6b6e6215f43fe0babfb4790712a2d3171f34d366 54.93MB / 54.93MB                21.6s 
 => => sha256:911ea9f2bd51e53a455297e0631e18a72a86d7e2c8e1807176e80f991bde5d64 10.87MB / 10.87MB                15.5s 
 => => sha256:530de807b46a11734e2587a784573c12c5034f2f14025f838589e6c0e3b5c5b6 1.86kB / 1.86kB                   0.0s 
 => => sha256:ff08f08727e50193dcf499afc30594c47e70cc96f6fcfd1a01240524624264d0 8.65kB / 8.65kB                   0.0s 
 => => sha256:2756ef5f69a5190f4308619e0f446d95f5515eef4a814dbad0bcebbbbc7b25a8 5.15MB / 5.15MB                   6.4s 
 => => sha256:27b0a22ee906271a6ce9ddd1754fdd7d3b59078e0b57b6cc054c7ed7ac301587 54.57MB / 54.57MB                37.7s 
 => => sha256:8584d51a9262f9a3a436dea09ba40fa50f85802018f9bd299eee1bf538481077 196.45MB / 196.45MB              82.3s 
 => => sha256:524774b7d3638702fe9ae0ea3fcfb81b027dfd75cc2fc14f0119e764b9543d58 6.29MB / 6.29MB                  26.6s 
 => => extracting sha256:955615a668ce169f8a1443fc6b6e6215f43fe0babfb4790712a2d3171f34d366                        5.4s 
 => => sha256:9460f6b75036e38367e2f27bb15e85777c5d6cd52ad168741c9566186415aa26 16.81MB / 16.81MB                40.5s 
 => => extracting sha256:2756ef5f69a5190f4308619e0f446d95f5515eef4a814dbad0bcebbbbc7b25a8                        0.6s 
 => => extracting sha256:911ea9f2bd51e53a455297e0631e18a72a86d7e2c8e1807176e80f991bde5d64                        0.6s 
 => => sha256:9bc548096c181514aa1253966a330134d939496027f92f57ab376cd236eb280b 232B / 232B                      40.1s 
 => => extracting sha256:27b0a22ee906271a6ce9ddd1754fdd7d3b59078e0b57b6cc054c7ed7ac301587                        5.8s 
 => => sha256:1d87379b86b89fd3b8bb1621128f00c8f962756e6aaaed264ec38db733273543 2.35MB / 2.35MB                  41.8s 
 => => extracting sha256:8584d51a9262f9a3a436dea09ba40fa50f85802018f9bd299eee1bf538481077                       18.8s 
 => => extracting sha256:524774b7d3638702fe9ae0ea3fcfb81b027dfd75cc2fc14f0119e764b9543d58                        1.2s 
 => => extracting sha256:9460f6b75036e38367e2f27bb15e85777c5d6cd52ad168741c9566186415aa26                        2.9s 
 => => extracting sha256:9bc548096c181514aa1253966a330134d939496027f92f57ab376cd236eb280b                        0.0s 
 => => extracting sha256:1d87379b86b89fd3b8bb1621128f00c8f962756e6aaaed264ec38db733273543                        0.8s 
 => [2/5] COPY . /app                                                                                            2.3s 
 => [3/5] WORKDIR /app                                                                                           0.2s 
 => [4/5] RUN pip install -U datasette                                                                          26.9s 
 => [5/5] RUN datasette inspect covid.db --inspect-file inspect-data.json                                        3.1s
 => exporting to image                                                                                           1.2s 
 => => exporting layers                                                                                          1.2s 
 => => writing image sha256:b5db0c205cd3454c21fbb00ecf6043f261540bcf91c2dfc36d418f1a23a75d7a                     0.0s

Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them
Traceback (most recent call last):
    ""__main__"", mod_spec)
  File ""c:\users\grott\anaconda3\lib\runpy.py"", line 85, in _run_code
    exec(code, run_globals)
  File ""C:\Users\grott\Anaconda3\Scripts\datasette.exe\__main__.py"", line 7, in <module>
  File ""c:\users\grott\anaconda3\lib\site-packages\click\core.py"", line 829, in __call__
    return self.main(*args, **kwargs)
  File ""c:\users\grott\anaconda3\lib\site-packages\click\core.py"", line 782, in main
    rv = self.invoke(ctx)
  File ""c:\users\grott\anaconda3\lib\site-packages\click\core.py"", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File ""c:\users\grott\anaconda3\lib\site-packages\click\core.py"", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File ""c:\users\grott\anaconda3\lib\site-packages\click\core.py"", line 610, in invoke
    return callback(*args, **kwargs)
  File ""c:\users\grott\anaconda3\lib\site-packages\datasette\cli.py"", line 283, in package
    call(args)
  File ""c:\users\grott\anaconda3\lib\contextlib.py"", line 119, in __exit__
    next(self.gen)
  File ""c:\users\grott\anaconda3\lib\site-packages\datasette\utils\__init__.py"", line 451, in temporary_docker_directory
    tmp.cleanup()
  File ""c:\users\grott\anaconda3\lib\tempfile.py"", line 811, in cleanup
    _shutil.rmtree(self.name)
  File ""c:\users\grott\anaconda3\lib\shutil.py"", line 516, in rmtree
    return _rmtree_unsafe(path, onerror)
  File ""c:\users\grott\anaconda3\lib\shutil.py"", line 395, in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
  File ""c:\users\grott\anaconda3\lib\shutil.py"", line 404, in _rmtree_unsafe
    onerror(os.rmdir, path, sys.exc_info())
  File ""c:\users\grott\anaconda3\lib\shutil.py"", line 402, in _rmtree_unsafe
    os.rmdir(path)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\grott\\AppData\\Local\\Temp\\tmpkb27qid3\\datasette'```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1479/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1930008379,I_kwDOBm6k_c5zCZc7,2197,click-default-group-wheel dependency conflict,1176293,ar-jan,closed,0,,,,,3,2023-10-06T11:49:20Z,2023-10-12T21:53:17Z,2023-10-12T21:53:17Z,NONE,,"I upgraded my dependencies, then ran into this problem running `datasette inspect`:

> env/lib/python3.9/site-packages/datasette/cli.py"", line 6, in <module>
>     from click_default_group import DefaultGroup
> ModuleNotFoundError: No module named 'click_default_group'

Turns out the released version of datasette still depends on `click-default-group-wheel`, so `click-default-group` doesn't get installed/recognized:

```
$ virtualenv venv
$ source venv/bin/activate
$ pip install datasette
$ pip list | grep click-default-group
click-default-group       1.2.4
click-default-group-wheel 1.2.3
$ python -c ""from click_default_group import DefaultGroup""
Traceback (most recent call last):
  File ""<string>"", line 1, in <module>
ModuleNotFoundError: No module named 'click_default_group'
$ pip install --force-reinstall click-default-group
...
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed.
This behaviour is the source of the following dependency conflicts.
datasette 0.64.4 requires click-default-group-wheel>=1.2.2, which is not installed.
Successfully installed click-8.1.7 click-default-group-1.2.4
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2197/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
814595021,MDU6SXNzdWU4MTQ1OTUwMjE=,1241,Share button for copying current URL,7107523,Kabouik,open,0,,,,,6,2021-02-23T15:55:40Z,2023-08-24T20:09:52Z,,NONE,,"I use datasette in an `iframe` inside another HTML file that contains other ways to represent my data (mostly leaflets maps built with R on summarized data), and the datasette `iframe` is a tab in that page. 

This particular use prevents users to access the full URLs of their datasette views and queries, which is a shame because the way datasette handles URLs to make every view or query easy to share is awesome. I know how to get the URL from the context menu of my browser, but I don't think many visitors would do it or even notice that datasette uses permalinks for pretty much every action they do. Would it be possible to add a ""Share link"" button to the interface, either in datasette itself or in a plugin?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1241/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1811824307,I_kwDOBm6k_c5r_j6z,2105,When reverse proxying datasette with nginx an URL element gets erronously added,2235371,aki-k,open,0,,,,,3,2023-07-19T12:16:53Z,2023-07-21T21:17:09Z,,NONE,,"I use this nginx config:
```
    location /datasette-llm {
      return 302 /datasette-llm/;
    }

    location /datasette-llm/ {
      proxy_set_header Upgrade           $http_upgrade;
      proxy_set_header Connection        ""Upgrade"";
      proxy_http_version 1.1;
      proxy_set_header X-Real-IP         $remote_addr;
      proxy_set_header X-Forwarded-For   $proxy_add_x_forwarded_for;
      proxy_set_header X-Forwarded-Proto https;
      proxy_set_header X-Forwarded-Host  $http_host;
      proxy_set_header Host              $host;
      proxy_max_temp_file_size           0;
      proxy_pass                         http://127.0.0.1:8001/datasette-llm/;
      proxy_redirect                     http:// https://;
      proxy_buffering off;
      proxy_request_buffering off;
      proxy_set_header Origin            '';
      client_max_body_size 0;
      auth_basic                         ""datasette-llm"";
      auth_basic_user_file               /etc/nginx/custom-userdb;
    }
```
Then I start datasette with this command:
```
datasette serve --setting base_url /datasette-llm/ $(llm logs path)
```
Everything else works right, except the links in ""This data as json, CSV"".
They get an extra URL element ""datasette-llm"" like this:

https://192.168.1.3:5432/datasette-llm/datasette-llm/logs.json?sql=select+*+from+_llm_migrations

https://192.168.1.3:5432/datasette-llm/datasette-llm/logs.csv?sql=select+*+from+_llm_migrations&_size=max

When I remove that extra ""datasette-llm"" from the URL, those links work too.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2105/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1396977994,I_kwDOBm6k_c5TRDFK,1830,Add documentation for writing tests with signed actor cookies,9599,simonw,open,0,,,,,0,2022-10-04T23:51:26Z,2022-10-04T23:51:26Z,,OWNER,,"I use this pattirn in a lot of plugin tests, e.g. https://github.com/simonw/datasette-edit-templates/blob/087f6a6cabc20020f2b0524f11aa3a7836320848/tests/test_edit_templates.py#L55-L58
```python
    actor = ds.sign({""a"": {""id"": ""root""}}, ""actor"")
    response1 = await ds.client.get(
        ""/-/edit-templates/_footer.html"", cookies={""ds_actor"": actor}
    )
```
I should add this to the documentation on this page: https://docs.datasette.io/en/latest/testing_plugins.html",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1830/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1181364043,I_kwDOBm6k_c5Gai9L,1687,Make show_json.html or a similar mechanism stable for plugins,9599,simonw,open,0,,,,,0,2022-03-25T23:42:45Z,2022-03-25T23:42:45Z,,OWNER,,"I used `show_json.html` in the new `datasette-packages` plugin, which means it will break if that template changes:
- https://github.com/simonw/datasette-packages/issues/3

It would be useful if it (or something like it) was documented and stable for plugins to use.

Also relevant:
- #878",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1687/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1879209560,I_kwDOCGYnMM5wAnZY,589,Mechanism for de-registering registered SQL functions,9599,simonw,open,0,,,,,3,2023-09-03T19:32:39Z,2023-09-03T19:36:34Z,,OWNER,,I used a custom SQL function in a migration script and then realized that it should be de-registered before the end of the script to avoid leaking into the calling code.,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/589/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
558600274,MDU6SXNzdWU1NTg2MDAyNzQ=,81,"Remove .detect_column_types() from table, make it a documented API",9599,simonw,closed,0,,,,,4,2020-02-01T21:25:54Z,2020-02-01T21:55:35Z,2020-02-01T21:55:35Z,OWNER,,"I used it in `geojson-to-sqlite` here: https://github.com/simonw/geojson-to-sqlite/blob/f10e44264712dd59ae7dfa2e6fd5a904b682fb33/geojson_to_sqlite/utils.py#L45-L50

It would make more sense for this method to live on the Database rather than the Table - or even to exist as a separate utility method entirely.

Then it should be documented.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/81/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
770436876,MDU6SXNzdWU3NzA0MzY4NzY=,1150,Maintain an in-memory SQLite table of connected databases and their tables,9599,simonw,closed,0,,,3268330,Datasette 1.0,32,2020-12-17T23:02:13Z,2020-12-27T14:51:39Z,2020-12-18T22:34:12Z,OWNER,,"I want Datasette to have its own internal metadata about connected tables, to power features like a paginated searchable homepage in #461. I want this to be a SQLite table.

This could also be part of the directory scanning mechanism prototyped in #672 - where Datasette can be set to continually scan a directory for new database files that it can serve.

Also relevant to the Datasette Library concept in #417.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1150/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1323346408,I_kwDOBm6k_c5O4Kno,1775,i18n support,428820,johnfelipe,open,0,,,,,9,2022-07-31T02:51:04Z,2023-02-10T18:04:40Z,,NONE,,"I want contribute for translate UI to es, de, de and it if you share strings",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1775/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
613002220,MDU6SXNzdWU2MTMwMDIyMjA=,19,apple-photos command should work even if upload has not run,9599,simonw,closed,0,,,,,1,2020-05-06T02:02:25Z,2020-05-19T20:59:59Z,2020-05-19T20:59:59Z,MEMBER,,"I want people to be able to query their Apple Photos metadata without having to first run `upload` to upload all of their files to their own S3 bucket.

To do this I can have `apple-photos` calculate SHA256 hashes of each photo if the `uploads` table does not yet exist (or does not contain that photo).",256834907,dogsheep-photos,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-photos/issues/19/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
952179830,MDU6SXNzdWU5NTIxNzk4MzA=,2,Command for fetching Hacker News threads from the search API,9599,simonw,open,0,,,,,4,2021-07-25T02:00:45Z,2021-07-25T03:12:57Z,,MEMBER,,"I want to be able to fetch every item for a domain, e.g. https://news.ycombinator.com/from?site=simonwillison.net",248903544,hacker-news-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/hacker-news-to-sqlite/issues/2/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
557825032,MDU6SXNzdWU1NTc4MjUwMzI=,77,Ability to insert data that is transformed by a SQL function,9599,simonw,closed,0,,,,,2,2020-01-30T23:45:55Z,2022-02-05T00:04:25Z,2020-01-31T00:24:32Z,OWNER,,"I want to be able to run the equivalent of this SQL insert:
```python
# Convert to ""Well Known Text"" format
wkt = shape(geojson['geometry']).wkt
# Insert and commit the record
conn.execute(""INSERT INTO places (id, name, geom) VALUES(null, ?, GeomFromText(?, 4326))"", (
   ""Wales"", wkt
))
conn.commit()
```
From the Datasette SpatiaLite docs: https://datasette.readthedocs.io/en/stable/spatialite.html

To do this, I need a way of telling `sqlite-utils` that a specific column should be wrapped in `GeomFromText(?, 4326)`.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/77/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
569253072,MDU6SXNzdWU1NjkyNTMwNzI=,678,prepare_connection() plugin hook should accept optional datasette argument,9599,simonw,closed,0,,,,,3,2020-02-22T00:50:26Z,2020-02-22T03:53:19Z,2020-02-22T02:28:51Z,OWNER,,"I want to build a plugin that allows users to configure certain database columns to be ""masked"" - so the `password` column on a users table is never revealed, for example.

To do this, I need to use the `conn.set_authorizer()` SQLite mechanism. So the plugin needs to build off the `prepare_connection(conn)` hook. But that hook doesn't currently get passed `datasette` so it doesn't have a way of looking up its plugin configuration!",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/678/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
760621356,MDU6SXNzdWU3NjA2MjEzNTY=,1136,Establish pattern for release branches to support bug fixes,9599,simonw,closed,0,,,,,8,2020-12-09T19:48:18Z,2020-12-09T20:17:02Z,2020-12-09T20:14:41Z,OWNER,,"I want to fix the bug in #1134 and ship it as Datasette 0.52.5 - but the `main` branch now has a feature in it (4c25b035b2370983c8dd5e0c8762e9154e379774 added `arraynotcontains`, #1132).

I'm not ready for a feature release, so instead I want to release 0.52.5 with just that bug fix.

This is the first time I will have shipped a release from a branch. I need to establish that pattern and add it to the documentation in https://docs.datasette.io/en/stable/contributing.html",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1136/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
506268945,MDU6SXNzdWU1MDYyNjg5NDU=,20,--since support for various commands for refresh-by-cron,9599,simonw,closed,0,,,,,3,2019-10-13T03:40:46Z,2019-10-21T03:32:04Z,2019-10-16T19:26:11Z,MEMBER,,"I want to run a cron that updates my Twitter database every X minutes.

It should be able to retrieve the following without needing to paginate through everything:

- [x] Tweets I have tweeted
- [x] My home timeline (see #19)
- [x] Tweets I have favourited

It would be nice if this could be standardized across all commands as a `--since` option.",206156866,twitter-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/20/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1071071397,I_kwDODFdgUs4_10Cl,69,View that combines issues and issue comments,9599,simonw,open,0,,,,,1,2021-12-04T00:34:33Z,2021-12-04T00:34:52Z,,MEMBER,,I want to see a reverse chronologically ordered interface onto both issues and comments - essentially a unified log of comments and issues opened across one or multiple projects.,207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/69/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
758899581,MDU6SXNzdWU3NTg4OTk1ODE=,1132,New filter: array does not contain,9599,simonw,closed,0,,,,,1,2020-12-07T22:28:20Z,2020-12-07T22:50:36Z,2020-12-07T22:41:09Z,OWNER,,"I want to see all of my GitHub repos that are tagged `datasette-plugin` but are NOT tagged `datasette-io`:

https://github-to-sqlite.dogsheep.net/github/repos?_facet=owner&owner=9599&_facet_array=topics&topics__arraycontains=datasette-plugin&topics__arraycontains=datasette-io#facet-topics",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1132/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
778530523,MDU6SXNzdWU3Nzg1MzA1MjM=,1172,/-/static should be excluded from auth and permission checks,9599,simonw,open,0,,,,,0,2021-01-05T02:53:41Z,2021-01-05T02:53:41Z,,OWNER,,"I want to set far future / immutable cache headers on everything served from `/-/static` and `/-/static-plugins`

This has security implications since it will be possible to see what plugins are installed by checking for known static URLs. I'm fine with that - performance is more important here.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1172/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
621280529,MDU6SXNzdWU2MjEyODA1Mjk=,23,create-subset command for creating a publishable subset of a photos database,9599,simonw,closed,0,,,,,1,2020-05-19T20:58:20Z,2020-05-19T22:32:48Z,2020-05-19T22:32:37Z,MEMBER,,"I want to share a subset of my photos, without sharing everything. Idea:

    $ photos-to-sqlite create-subset photos.db public.db ""select sha256 from ... where ...""

So the command takes a SQL query that returns sha256 hashes, then creates a new file called `public.db` containing just the data corresponding to those photos.",256834907,dogsheep-photos,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-photos/issues/23/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
275092453,MDU6SXNzdWUyNzUwOTI0NTM=,122,"Redesign JSON output, ditch jsono, offer variants controlled by parameter instead",9599,simonw,closed,0,,,,,5,2017-11-18T16:52:28Z,2018-04-08T14:54:09Z,2018-04-08T14:54:09Z,OWNER,,"I want to support three variants for the rows output:

* a list of lists, with a columns key saying what they are
* a list of dictionaries
* a single dictionary where the keys are the primary keys of the rows and the values are the row dictionaries themselves

I also want to make the various bits of metadata opt-in - so you don't get the SQL statement unless you ask for it.

These output options should be controlled by query string arguments.

I will set the .jsono URL to redirect to .json with the corresponding options. ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/122/reactions"", ""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 1, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
769520939,MDU6SXNzdWU3Njk1MjA5Mzk=,1149,Make it easier to theme Datasette with CSS,9599,simonw,open,0,,,3268330,Datasette 1.0,3,2020-12-17T05:01:26Z,2021-03-22T21:43:16Z,,OWNER,,"I want to theme https://datasette.io/ so that when you visit https://datasette.io/content (the Datasette UI part of it) the navigation from the parent site is used.

I tried dropping in a `base.html` template like this:

```html
{% extends ""page_base.html"" %}

{% block base_extra_head %}

<meta name=""viewport"" content=""width=device-width, initial-scale=1, shrink-to-fit=no"">
{% for url in extra_css_urls %}
    <link rel=""stylesheet"" href=""{{ url.url }}""{% if url.sri %} integrity=""{{ url.sri }}"" crossorigin=""anonymous""{% endif %}>
{% endfor %}
{% for url in extra_js_urls %}
    <script src=""{{ url.url }}""{% if url.sri %} integrity=""{{ url.sri }}"" crossorigin=""anonymous""{% endif %}></script>
{% endfor %}
{% block extra_head %}{% endblock %}
{% endblock %}

{% block extra_body_end %}
{% include ""_close_open_menus.html"" %}

{% for body_script in body_scripts %}
    <script>{{ body_script }}</script>
{% endfor %}
{% endblock %}
```
But this resulted in pages looking like this:

<img width=""1067"" alt=""content__categories__3_rows"" src=""https://user-images.githubusercontent.com/9599/102446045-c168e280-3fe1-11eb-94d6-e7350798eb96.png"">

Note that the cog menu is broken and the filter UI is unstyled. To get these working correctly I would need to copy over a whole lot of Datasette's default CSS - and that means that when Datasette changes in the future those pages could break in subtle ways.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1149/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
268462768,MDU6SXNzdWUyNjg0NjI3Njg=,38,Experiment with patterns for concurrent long running queries,9599,simonw,closed,0,,,,,5,2017-10-25T16:23:42Z,2018-05-28T20:47:31Z,2018-05-28T20:47:31Z,OWNER,,I want to understand how the system could perform under load with many concurrent long-running queries. Can we serve these without blocking the event loop?,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/38/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
459509126,MDU6SXNzdWU0NTk1MDkxMjY=,516,Enforce import sort order with isort,9599,simonw,open,0,,,,,8,2019-06-22T20:35:50Z,2023-08-23T02:15:36Z,,OWNER,,"I want to use isort to order imports. A few steps here:

- [x] Add a .isort.cfg file (see below)
- [x] Use `isort -rc` to reformat existing code
- [ ] Commit this change
- [x] Add a unit test that ensures future changes remain isort compatible",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/516/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1298531653,I_kwDOCGYnMM5NZgVF,451,Make sqlite_utils.utils.chunks a documented function,9599,simonw,closed,0,,,,,2,2022-07-08T06:01:04Z,2022-07-15T22:09:34Z,2022-07-15T21:59:33Z,OWNER,,I want to use it in another project: https://github.com/simonw/sqlite-utils/blob/8a9fe6498faf783a1fdeb1793e661ad194a05267/sqlite_utils/utils.py#L471-L474,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/451/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1699174055,I_kwDOCGYnMM5lR1an,539,"`--raw-lines` option, like `--raw` for multiple lines",9599,simonw,closed,0,,,,,4,2023-05-07T18:07:46Z,2023-05-07T18:43:24Z,2023-05-07T18:26:18Z,OWNER,,I wanted to output newline-separated output of the first column of every row in the results - like `--row` but for more than one line.,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/539/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
508100844,MDU6SXNzdWU1MDgxMDA4NDQ=,598,Character encoding bug with CSV export,46313,JoeGermuska,closed,0,,,,,1,2019-10-16T21:09:30Z,2021-06-17T18:13:20Z,2019-10-18T22:52:21Z,NONE,,"I was just poking around, and at [this URL](https://sql-murder-mystery.datasette.io/sql-murder-mystery/crime_scene_report.csv?_stream=on&type=arson&_size=max), I encountered this error:

```
'latin-1' codec can't encode character '\u2019' in position 27: ordinal not in range(256)
```

",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/598/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
645975649,MDU6SXNzdWU2NDU5NzU2NDk=,867,register_routes() should support non-async view functions too,9599,simonw,closed,0,,,5533512,Datasette 0.45,1,2020-06-26T03:11:25Z,2020-06-27T18:30:41Z,2020-06-27T18:30:40Z,OWNER,,"I was looking at this: https://github.com/simonw/datasette-block-robots/blob/main/datasette_block_robots/__init__.py
```python
from datasette import hookimpl
from datasette.utils.asgi import Response


async def robots_txt():
    return Response.text(""User-agent: *\nDisallow: /"")


@hookimpl
def register_routes():
    return [
        (r""^/robots\.txt$"", robots_txt),
    ]
```
And I realized that if `register_routes()` could support non-async view functions it could be reduced to this:
```python
@hookimpl
def register_routes():
    return [
        (r""^/robots\.txt$"", lambda: Response.text(""User-agent: *\nDisallow: /"")),
    ]
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/867/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1382457780,I_kwDOCGYnMM5SZqG0,490,Ability to insert multi-line files,6180701,jeqo,closed,0,,,,,4,2022-09-22T13:29:22Z,2022-09-26T18:24:44Z,2022-09-23T16:37:58Z,NONE,,"I was looking into how to parse application log files that contain multiline text (e.g. Java stack traces) into sqlite. 
I can see that at the moment `--lines` helps, but falls short when processing multi-line texts.

I wonder if this functionality would be useful for sqlite-utils. A similar approach to Elastic logstash/filebeat can be adopted: https://www.elastic.co/guide/en/beats/filebeat/current/multiline-examples.html 

Potential changes:

- add a `--multiline` option
- additional properties for
  - multiline-pattern (regex expression)
  - multiline-negate: true/false
  - multiline-what: previous or next

Or if this is achievable in a different way, please share. Thanks!",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/490/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
459621683,MDU6SXNzdWU0NTk2MjE2ODM=,521,Easier way of creating custom row templates,9599,simonw,closed,0,,,,,6,2019-06-23T21:49:27Z,2019-07-03T03:23:56Z,2019-07-03T03:23:56Z,OWNER,,"I was messing around with a custom `_rows_and_columns.html` template and ended up with this:
```html
{% for row in display_rows %}
  <div> 
    <hr>
    {% for cell in row %}
      {% if cell.column == ""First_Name"" %}
        <h2 class=""scientist"">{{ cell.value }}
      {% elif cell.column == ""Last_Name"" %}
        {{ cell.value }}</h1>
      {% elif cell.column == ""Short_Description"" %}
        <p><strong>{{ cell.column }}: </strong>{{ cell.value }}</p>
        <p>
      {% else %}
        <strong>{{ cell.column }}: </strong>{{ cell.value }}&nbsp;&nbsp;&nbsp;
      {% endif %}
    {% endfor %}
  </div>
{% endfor %}
```
This is nasty. I'd like to be able to do something like this instead:
```
{% for row in display_rows %}
  <h2 class=""scientist"">{{ row[""First_Name""] }} {{ row[""Last_Name""] }}</h2>
  ...
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/521/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
592829135,MDU6SXNzdWU1OTI4MjkxMzU=,713,Support YAML in metadata - metadata.yaml,9599,simonw,closed,0,,,,,6,2020-04-02T18:10:05Z,2020-04-02T19:36:17Z,2020-04-02T19:30:55Z,OWNER,,"I was originally going to do this with a plugin - see #357 - but the more I work with `metadata.json` the more I want it to just accept YAML as an optional alternative to JSON.

The best example why is still this one: https://github.com/simonw/russian-ira-facebook-ads-datasette/blob/master/russian-ads-metadata.yaml

YAML is just SO much better than JSON for multi-line strings - in particular HTML and SQL, both of which are common in `metadata.json` files.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/713/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1083581011,I_kwDOBm6k_c5AliJT,1564,_prepare_connection not called on write connections,9599,simonw,closed,0,,,7571612,Datasette 0.60,1,2021-12-17T20:06:47Z,2022-01-20T21:29:43Z,2021-12-18T01:58:44Z,OWNER,,"I was trying to initalize SpatiaLite in a write connection:
```pycon
>>> from datasette.app import Datasette
>>> ds = Datasette(memory=True, files=[], sqlite_extensions=[""spatialite""])
>>> db = ds.add_memory_database('geo')
>>> await db.execute_write(""select InitSpatialMetadata(1)"")
UUID('3f143baa-4e3d-5842-a36f-4fa2f683b72f')
no such function: InitSpatialMetadata
```
It looks like the code that loads additional modules only works on read-only connections, not on write connections:

https://github.com/simonw/datasette/blob/92a5280d2e75c39424a75ad6226fc74400ae984f/datasette/database.py#L146-L153

Compared to:

https://github.com/simonw/datasette/blob/92a5280d2e75c39424a75ad6226fc74400ae984f/datasette/database.py#L124-L132",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1564/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
400511206,MDU6SXNzdWU0MDA1MTEyMDY=,403,How does persistence work?,1794527,ccorcos,closed,0,,,,,2,2019-01-17T23:41:57Z,2019-01-19T05:47:55Z,2019-01-18T06:51:14Z,NONE,,I was under the impression that now.sh is for stateless microservices. So where are these SQLite databases stored and when do they get created and destroyed?,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/403/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1839344979,I_kwDOCGYnMM5toi1T,582,Handling CSV/file input that contains NUL bytes,1448859,betatim,open,0,,,,,0,2023-08-07T12:24:14Z,2023-08-07T12:24:14Z,,NONE,,"I was using sqlite-utils to create a DB from a CSV and it turns out the CSV contains a NUL byte.

When the processing reaches the line that contains the NUL an exception is raised.

I'm wondering if there is something that can be done in `sqlite-utils` to say ""skip lines with encoding errors"" or some such. I think it isn't super straightforward though as the exception comes from inside the `csv` module that does all the parsing.

Concretely the file is the `KernelVersions.csv` from https://www.kaggle.com/datasets/kaggle/meta-kaggle

This is the command and output:
```
$ sqlite-utils insert --csv kaggle.db kaggle KernelVersions.csv
  [------------------------------------]    0%
  [#####################---------------]   60%  00:04:24Traceback (most recent call last):
  File ""/home/foobar/miniconda/envs/meta-kaggle/bin/sqlite-utils"", line 10, in <module>
    sys.exit(cli())
  File ""/home/foobar/miniconda/envs/meta-kaggle/lib/python3.10/site-packages/click/core.py"", line 1128, in __call__
    return self.main(*args, **kwargs)
  File ""/home/foobar/miniconda/envs/meta-kaggle/lib/python3.10/site-packages/click/core.py"", line 1053, in main
    rv = self.invoke(ctx)
  File ""/home/foobar/miniconda/envs/meta-kaggle/lib/python3.10/site-packages/click/core.py"", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File ""/home/foobar/miniconda/envs/meta-kaggle/lib/python3.10/site-packages/click/core.py"", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File ""/home/foobar/miniconda/envs/meta-kaggle/lib/python3.10/site-packages/click/core.py"", line 754, in invoke
    return __callback(*args, **kwargs)
  File ""/home/foobar/miniconda/envs/meta-kaggle/lib/python3.10/site-packages/sqlite_utils/cli.py"", line 1223, in insert
    insert_upsert_implementation(
  File ""/home/foobar/miniconda/envs/meta-kaggle/lib/python3.10/site-packages/sqlite_utils/cli.py"", line 1085, in insert_upsert_implementation
    db[table].insert_all(
  File ""/home/foobar/miniconda/envs/meta-kaggle/lib/python3.10/site-packages/sqlite_utils/db.py"", line 3198, in insert_all
    chunk = list(chunk)
  File ""/home/foobar/miniconda/envs/meta-kaggle/lib/python3.10/site-packages/sqlite_utils/db.py"", line 3742, in fix_square_braces
    for record in records:
  File ""/home/foobar/miniconda/envs/meta-kaggle/lib/python3.10/site-packages/sqlite_utils/cli.py"", line 1071, in <genexpr>
    docs = (decode_base64_values(doc) for doc in docs)
  File ""/home/foobar/miniconda/envs/meta-kaggle/lib/python3.10/site-packages/sqlite_utils/cli.py"", line 1068, in <genexpr>
    docs = (verify_is_dict(doc) for doc in docs)
  File ""/home/foobar/miniconda/envs/meta-kaggle/lib/python3.10/site-packages/sqlite_utils/cli.py"", line 1003, in <genexpr>
    docs = (dict(zip(headers, row)) for row in reader)
_csv.Error: line contains NUL
```",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/582/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
639072811,MDU6SXNzdWU2MzkwNzI4MTE=,849,Rename master branch to main,9599,simonw,closed,0,,,3268330,Datasette 1.0,10,2020-06-15T19:05:54Z,2022-10-27T13:57:08Z,2020-09-15T20:37:14Z,OWNER,,"I was waiting for consensus to form around this (and kind-of hoping for `trunk` since I like the tree metaphor) and it looks like `main` is it.

I've seen convincing arguments against `trunk` too - it indicates that the branch has some special significance like in Subversion (where all branches come from trunk) when it doesn't. So `main` is better anyway.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/849/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
487601121,MDU6SXNzdWU0ODc2MDExMjE=,4,Online tool for getting a Foursquare OAuth token,9599,simonw,closed,0,,,,,1,2019-08-30T17:48:14Z,2019-08-31T18:07:26Z,2019-08-31T18:07:26Z,MEMBER,,"I will link to this from the documentation. See also this conversation on Twitter: https://twitter.com/simonw/status/1166822603023011840

I've decided to go with ""copy and paste in a token"" rather than hooking up a local web server that can have tokens passed to it.",205429375,swarm-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/4/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
625930207,MDU6SXNzdWU2MjU5MzAyMDc=,770,register_output_renderer can_render mechanism,9599,simonw,closed,0,,,5471110,Datasette 0.43,4,2020-05-27T18:29:14Z,2020-05-28T05:57:16Z,2020-05-28T05:57:16Z,OWNER,,"I would like is the ability for renderers to opt-in / opt-out of being displayed as options on the page.

https://www.niche-museums.com/browse/museums for example shows a atom link because the datasette-atom plugin is installed... but clicking it will give you a 400 error because the correct columns are not present.

<img width=""1113"" alt=""browse__museums__102_rows"" src=""https://user-images.githubusercontent.com/9599/83058543-62278e80-a00d-11ea-9e04-298e02745118.png"">

Here's the code that passes a list of renderers to the template:

https://github.com/simonw/datasette/blob/2d099ad9c657d2cab59de91cdb8bfed2da236ef6/datasette/views/base.py#L411-L423

A renderer is currently defined as a two-key dictionary:
```python
@hookimpl
def register_output_renderer(datasette):
    return {
        'extension': 'test',
        'callback': render_test
    }
```
I can add a third key, `""should_suggest""` which is a function that returns `True` or `False` for a given query. If that key is missing it is assumed to return `True`.

One catch: what arguments should be passed to the `should_suggest(...)` function?

UPDATE: now calling it `can_render` instead.

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/581#issuecomment-634856748_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/770/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1148638868,I_kwDOBm6k_c5EdtaU,1639,Make datasette-redirect-forbidden unneccessary,9599,simonw,open,0,,,,,0,2022-02-23T22:18:46Z,2022-02-23T22:18:46Z,,OWNER,,"I wrote `datasette-redirect-forbidden` today because I needed 403 errors to redirect to `/-/login` and it was the quickest way to solve that problem.

This should be a feature of Datasette core.

- https://github.com/simonw/datasette-redirect-forbidden/issues/2",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1639/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
679700269,MDU6SXNzdWU2Nzk3MDAyNjk=,938,Pass columns to extra CSS/JS/etc plugin hooks,9599,simonw,closed,0,,,,,3,2020-08-16T06:37:47Z,2020-09-30T20:36:12Z,2020-08-16T18:09:59Z,OWNER,,"I'd like `datasette-cluster-map` to only add links to JavaScript on pages that have tables with latitude and longitude columns.

Passing the names of the columns to the plugin hook can support this and will be backwards compatible thanks to pluggy.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/938/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
737855731,MDU6SXNzdWU3Mzc4NTU3MzE=,199,"@db.register_function(..., replace=False) to avoid double-registering custom functions",9599,simonw,closed,0,,,,,1,2020-11-06T15:39:21Z,2020-11-06T18:30:44Z,2020-11-06T18:30:44Z,OWNER,,"I'd like a mechanism to optionally avoid registering a custom function if it has already been registered.

SQLite doesn't seem to offer a way to introspect registered custom functions so I'll need to track what has already been registered in `sqlite-utils` instead.

> Should I register the custom `rank_bm25` SQLite function for every connection, or should I register it against the connection just the first time the user attempts an FTS4 search? I think I'd rather register it only if it is needed.

_Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/198#issuecomment-723145383_",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/199/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
315738696,MDU6SXNzdWUzMTU3Mzg2OTY=,226,Unit tests for installable plugins,9599,simonw,closed,0,,,,,2,2018-04-19T06:05:32Z,2020-11-24T19:52:51Z,2020-11-24T19:52:46Z,OWNER,,"I'd like more thorough unit test coverage of the plugins mechanism - in particular for installable plugins.

I think I can do this while still having the code live in the same repo, by creating a subdirectory in tests/example_plugin with its own setup.py and then running `python setup.py install` as part of the test runner.

I imagine I will need to bump the version number every time I change the plugin in case someone runs the test again in the same virtual environment.

If that doesn't work I can instead ship a datasette-plugins-tests two to PyPI and add that as a tests_require dependency.

Refs #14",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/226/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
635147716,MDU6SXNzdWU2MzUxNDc3MTY=,825,Way to enable a default=False permission for anonymous users,9599,simonw,closed,0,,,5512395,Datasette 0.44,6,2020-06-09T06:26:27Z,2020-06-09T17:19:19Z,2020-06-09T17:01:10Z,OWNER,,"I'd like plugins to be able to ship with a default that says ""anonymous users cannot do this"", but allow site administrators to over-ride that such that anonymous users can use the feature after all.

This is tricky because right now the anonymous user doesn't have an actor dictionary at all, so there's no key to match to an allow block.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/825/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1801394744,I_kwDOCGYnMM5rXxo4,567,Plugin system,15178711,asg017,closed,0,,,,,9,2023-07-12T17:02:14Z,2023-07-22T22:59:37Z,2023-07-22T22:59:36Z,CONTRIBUTOR,,"I'd like there to be a plugin system for sqlite-utils, similar to the datasette/llm plugins. I'd like to make plugins that would do things like:

- Register SQLite extensions for more SQL functions + virtual tables
- Register new subcommands
- Different input file formats for `sqlite-utils memory`
- Different output file formats (in addition to `--csv` `--tsv` `--nl` etc.

A few real-world use-cases of plugins I'd like to see in sqlite-utils:

- Register many of my sqlite extensions in sqlite-utils (`sqlite-http`, `sqlite-lines`, `sqlite-regex`, etc.)
- New subcommands to work with `sqlite-vss` vector tables
- Input/ouput Parquet/Avro/Arrow IPC files with `sqlite-arrow`",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/567/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1059549523,I_kwDOBm6k_c4_J3FT,1526,"Add to vercel.json, rather than overwriting it.",192568,mroswell,closed,0,,,,,2,2021-11-22T00:47:12Z,2021-11-22T04:49:45Z,2021-11-22T04:13:47Z,CONTRIBUTOR,,"I'd like to be able to add to vercel.json. But Datasette overwrites whatever I put in that file. I originally reported this here:
https://github.com/simonw/datasette-publish-vercel/issues/51

In that case, I wanted to do a rewrite... and now I need to do 301 redirects (because we had to rename our site).

Can this be addressed?
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1526/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
632843030,MDU6SXNzdWU2MzI4NDMwMzA=,807,Ability to ship alpha and beta releases,9599,simonw,closed,0,,,5533512,Datasette 0.45,18,2020-06-07T00:12:55Z,2020-06-18T21:41:16Z,2020-06-18T21:41:16Z,OWNER,,I'd like to be able to ship alphas and betas to PyPI so in-development plugins can depend on them and help test unreleased plugin hooks.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/807/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
760605882,MDU6SXNzdWU3NjA2MDU4ODI=,1135,Feature: --create option to create database file if it does not yet exist,9599,simonw,closed,0,,,,,0,2020-12-09T19:23:58Z,2021-01-24T21:19:39Z,2020-12-09T19:45:52Z,OWNER,,"I'd like to be able to tell people to run the following in the Datasette documentation to get started:

    brew install datasette
    datasette install datasette-upload-csvs
    datasette data.db --create --root --open

This would give them a local Datasette instance with the ability to drag-and-drop CSV files directly into it.

Just one catch: I don't want to have to talk them through creating an empty SQLite database file. So I want to add a new `--create` option which means ""If any of the database files passed on the command-line do not yet exist, create them"".",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1135/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
706167456,MDU6SXNzdWU3MDYxNjc0NTY=,168,Automate (as much as possible) updates published to Homebrew,9599,simonw,closed,0,,,,,2,2020-09-22T08:08:37Z,2020-11-09T07:43:30Z,2020-11-09T07:43:30Z,OWNER,,I'd like to get new `sqlite-utils` (and Datasette) releases submitted to Homebrew as painlessly as possible.,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/168/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
324835838,MDU6SXNzdWUzMjQ4MzU4Mzg=,276,Handle spatialite geometry columns better,45057,russss,closed,0,,,,,21,2018-05-21T08:46:55Z,2022-03-21T22:22:20Z,2022-03-21T22:22:20Z,CONTRIBUTOR,,"I'd like to see spatialite geometry columns rendered more sensibly - at the moment they come through as well-known-binary unless you use custom SQL, and WKB isn't of much use to anyone on the web.

In HTML: they should be shown either as simple lat/long (if it's just a point, for example), or as a sensible placeholder if they're more complex geometries.

In JSON: they should be GeoJSON geometries, (which means they can be automatically fed into a leaflet map with no further messing around).

In CSV: they should be WKT.

I briefly wondered if this should go into a plugin, but I suspect it needs hooking in at a deeper level than the plugin architecture will support any time soon.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/276/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1200649889,I_kwDOBm6k_c5HkHah,1710,Guide for plugin authors to upgrade their plugins for 1.0,9599,simonw,closed,0,,,,,1,2022-04-11T22:58:25Z,2022-04-11T23:04:01Z,2022-04-11T23:03:25Z,OWNER,,I'll also encourage testing against both Datasette 0.x and Datasette 1.0 using a GitHub Actions matrix.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1710/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
860722711,MDU6SXNzdWU4NjA3MjI3MTE=,1301,Publishing to cloudrun with immutable mode?,5413548,louispotok,open,0,,,,,1,2021-04-18T17:51:46Z,2022-10-07T02:38:04Z,,CONTRIBUTOR,,"I'm a bit confused about immutable mode and publishing to cloudrun. (I want to publish with immutable mode so that I can support database downloads.)

Running `datasette publish cloudrun --extra-options=""-i example.db""` leads to an error:
> Error: Invalid value for '-i' / '--immutable': Path 'example.db' does not exist. 

However, running `datasette publish cloudrun example.db` not only works but seems to publish in immutable mode anyway! I'm seeing this both with `/-/databases.json` and the fact that downloads are working.

When I just `datasette serve` locally, this succeeds both ways and works as expected.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1301/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1450312343,I_kwDOBm6k_c5WcgKX,1892,Merge 1.0-dev branch back to main,9599,simonw,closed,0,,,8658075,Datasette 1.0a0,3,2022-11-15T20:04:25Z,2022-11-29T19:40:23Z,2022-11-29T19:40:23Z,OWNER,,"I'm committed enough to the 1.0 work now that I'm ready for the `main` branch to reflect that instead.

If I need to make any dot-releases against 0.63 I can do those from a branch.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1892/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
2007893839,I_kwDOCGYnMM53rgdP,605,Insert fails with `Error: Python int too large to convert to SQLite INTEGER`; can we use `NUMERIC` here?,12229877,Zac-HD,closed,0,,,,,1,2023-11-23T10:19:46Z,2023-12-08T05:07:54Z,2023-12-08T05:07:54Z,NONE,,"I'm currently working on a new feature for Hypothesis, where we can dump a tidy jsonlines table of all the test cases we tried - including arguments, outcomes, timings, coverage, etc.  Exploring this seems like a perfect cases for `sqlite-utils` and `datasette`, but I pretty quickly ran into an integer overflow problem and don't want to recommend that experience to my users.

I originally went to report this as a bug... and then found https://github.com/simonw/sqlite-utils/issues/309#issuecomment-895581038 almost exactly matched my repro 😅 

https://github.com/simonw/sqlite-utils/issues/110#issuecomment-626391063 suggests that using `NUMERIC` would avoid this overflow error, although ""If the TEXT value is a well-formed integer literal that is too large to fit in a 64-bit signed integer, it is converted to REAL."" suggests that this would come at the cost of rounding to the nearest float value.  Maybe I should just convert large integers to float before writing out my json?

After a bit more hacking, ""manually cast large integers to float"" seems like a decent solution for my particular case, but having written it up I thought I might as well post this issue anyway - I hope it's useful feedback, and won't mind at all if you close as wontfix if it's not.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/605/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
771608692,MDU6SXNzdWU3NzE2MDg2OTI=,14,UNIQUE constraint failed: workouts.id,1234956,n8henrie,open,0,,,,,5,2020-12-20T15:11:20Z,2023-07-10T14:46:52Z,,NONE,,"I'm getting an error on my initial attempt to import data:

```console
$ healthkit-to-sqlite 20201119\ healthkit\ export.zip healthkit.db
Importing from HealthKit  [###################################-]   98%  00:00:01
Traceback (most recent call last):
  File ""venv/bin/healthkit-to-sqlite"", line 8, in <module>
    sys.exit(cli())
  File ""venv/lib/python3.9/site-packages/click/core.py"", line 829, in __call__
    return self.main(*args, **kwargs)
  File ""venv/lib/python3.9/site-packages/click/core.py"", line 782, in main
    rv = self.invoke(ctx)
  File ""venv/lib/python3.9/site-packages/click/core.py"", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File ""venv/lib/python3.9/site-packages/click/core.py"", line 610, in invoke
    return callback(*args, **kwargs)
  File ""venv/lib/python3.9/site-packages/healthkit_to_sqlite/cli.py"", line 57, in cli
    convert_xml_to_sqlite(fp, db, progress_callback=bar.update, zipfile=zf)
  File ""venv/lib/python3.9/site-packages/healthkit_to_sqlite/utils.py"", line 34, in convert_xml_to_sqlite
    workout_to_db(el, db, zipfile)
  File ""venv/lib/python3.9/site-packages/healthkit_to_sqlite/utils.py"", line 57, in workout_to_db
    pk = db[""workouts""].insert(record, alter=True, hash_id=""id"").last_pk
  File ""venv/lib/python3.9/site-packages/sqlite_utils/db.py"", line 1660, in insert
    return self.insert_all(
  File ""venv/lib/python3.9/site-packages/sqlite_utils/db.py"", line 1778, in insert_all
    self.insert_chunk(
  File ""venv/lib/python3.9/site-packages/sqlite_utils/db.py"", line 1588, in insert_chunk
    result = self.db.execute(query, params)
  File ""venv/lib/python3.9/site-packages/sqlite_utils/db.py"", line 213, in execute
    return self.conn.execute(sql, parameters)
sqlite3.IntegrityError: UNIQUE constraint failed: workouts.id
```",197882382,healthkit-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/14/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,