id,node_id,number,title,user,state,locked,assignee,milestone,comments,created_at,updated_at,closed_at,author_association,pull_request,body,repo,type,active_lock_reason,performed_via_github_app,reactions,draft,state_reason 268110769,MDU6SXNzdWUyNjgxMTA3Njk=,33,Use locust for benchmarking and load tests,9599,open,0,,,0,2017-10-24T17:00:09Z,2017-12-10T03:12:16Z,,OWNER,,"https://github.com/locustio/locust Needed for #32 ",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/33/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 275159710,MDU6SXNzdWUyNzUxNTk3MTA=,128,"Every visualization should have an ""embed"" button",9599,open,0,,,0,2017-11-19T13:38:13Z,2019-05-13T18:33:51Z,,OWNER,,"At least for the first round of visualizations, any time you construct one using the UI the result should include an ""embed this"" button that returns source code to copy and paste These examples should use unpkg.com (or similarl) urls with SRI hashes, eg https://www.srihash.org - and should load data from the datasette JSON API.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/128/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 312395790,MDU6SXNzdWUzMTIzOTU3OTA=,197,Ability to sort by more than one column,9599,open,0,,,0,2018-04-09T05:13:30Z,2018-07-10T17:45:37Z,,OWNER,,"Split off from #189. I'd like to support ""sort by X descending, then by Y ascending if there are dupes for X"" as well. Suggested syntax for that: ?_sort_desc=X&_sort=Y we currently only allow one argument to be sent. We should allow as many arguments as there are columns, for example: ?_sort=department&_sort_desc=precinct&_sort=age&_sort_desc=size",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/197/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 312396095,MDU6SXNzdWUzMTIzOTYwOTU=,198,Ability to sort with nulls last,9599,open,0,,,0,2018-04-09T05:15:40Z,2018-07-10T17:45:37Z,,OWNER,,"Split off from #189 Here's how to do that in SQL: https://fivethirtyeight.datasettes.com/fivethirtyeight-2628db9?sql=select+rowid%2C+*+from+%5Bnfl-wide-receivers%2Fadvanced-historical%5D%0D%0Aorder+by+case+when+career_ranypa+is+null+then+1+else+0+end%2C+career_ranypa%2C+rowid order by case when career_ranypa is null then 1 else 0 end, career_ranypa",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/198/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 314771615,MDU6SXNzdWUzMTQ3NzE2MTU=,218,"Support custom unit display in order to handle ""$10,000""",9599,open,0,,,0,2018-04-16T18:39:31Z,2018-07-10T17:45:38Z,,OWNER,,"I tried to get Datasette to display `$10,000` using the new units support but we currently only display units as a suffix: https://github.com/simonw/datasette/blob/10a34f995c70daa37a8a2aa02c3135a4b023a24c/datasette/app.py#L563-L572 It would be neat if there was a mechanism for specifying a custom unit display - maybe something like this: ``` { ""custom_units"": { ""us_dollar"": { ""unit"": ""us_dollar = [] = $"", ""format"": ""${:,}"" } } } ```",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/218/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 318490133,MDU6SXNzdWUzMTg0OTAxMzM=,241,Default datasette logging format should be JSON,9599,open,0,,,0,2018-04-27T17:32:48Z,2018-07-10T17:45:40Z,,OWNER,,"Structured logs are better. Datasette should default to outputting it's HTTP access log lines as newline delimited JSON instead of the Sanic default format it uses at the moment. For improved greppability these logs should have keys ordered in a consistent way. Python's JSON module can do this with ordered dictionaries.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/241/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 320132682,MDU6SXNzdWUzMjAxMzI2ODI=,250,Setup some issue templates,9599,open,0,,,0,2018-05-04T01:49:07Z,2018-05-04T01:49:07Z,,OWNER,,"https://twitter.com/left_pad/status/99216385740464537 I like the idea of using these to help people understand some of the ways I want to use issues.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/250/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 326778161,MDU6SXNzdWUzMjY3NzgxNjE=,290,Consider increasing the default for num_sql_threads (currently 3),9599,open,0,,,0,2018-05-27T00:52:41Z,2018-05-27T00:52:41Z,,OWNER,,"I ran a very rough micro-benchmark on the new `num_sql_threads` config option (added in #285) datasette --config num_sql_threads:1 fivethirtyeight.db Then ab -n 100 -c 10 'http://127.0.0.1:8011/fivethirtyeight-2628db9/twitter-ratio%2Fsenators' | Number of threads | Requests/second | |---|---| | 1 | 4.57 | | 3 | 9.77 | | 10 | 13.53 | | 20 | 15.24 | 50 | 8.21 | This was on my early 2018 OS X laptop. Need to benchmark in other common environments before making a decision on changing the default. That said, the default of 3 was a number I plucked out of thin air.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/290/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 344654623,MDU6SXNzdWUzNDQ2NTQ2MjM=,347,"Rename ""datasette package"" to ""datasette publish docker""",9599,open,0,,,0,2018-07-26T00:42:46Z,2018-07-26T00:42:46Z,,OWNER,,,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/347/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 346026869,MDU6SXNzdWUzNDYwMjY4Njk=,354,Handle many-to-many relationships,9599,open,0,,,0,2018-07-31T04:03:13Z,2020-11-24T19:51:18Z,,OWNER,,This is a master tracking ticket for various many-2-many features.,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/354/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 426722204,MDU6SXNzdWU0MjY3MjIyMDQ=,423,?_search_col=X not reflected correctly in the UI,9599,open,0,,,0,2019-03-28T21:48:19Z,2020-11-03T19:01:59Z,,OWNER,,"e.g. https://latest.datasette.io/fixtures/searchable?_search_text1=barry ![2019-03-28 at 2 47 PM](https://user-images.githubusercontent.com/9599/55195035-84ebb800-5168-11e9-910b-fc9868bcd93e.png) ",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/423/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 449445715,MDU6SXNzdWU0NDk0NDU3MTU=,491,Figure out how to use Firebase with cloudrun to enable vanity URLs and CDN caching,9599,open,0,,,0,2019-05-28T19:48:06Z,2019-05-28T19:48:35Z,,OWNER,,"It looks like Firebase can solve a couple of problems with the existing `datasette publish cloudrun` hosting mechanism: * The URLs it produces aren't pretty enough. Firebase offers more control over vanity URLs. * CDN caching (as seen in `datasette publish now`) is great for improving performance and saving money on Cloud Run execution time. https://firebase.google.com/docs/hosting/cloud-run looks like it can help with both of these. Lots of interesting questions: * Should this be a new `datasette publish firebase` command or should it instead be implemented as additional custom options to `datasette publish cloudrun`? * How much harder does it become to do account setup? * How much will this option cost users?",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/491/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 459469278,MDU6SXNzdWU0NTk0NjkyNzg=,515,Try shrinking official image with docker-slim,9599,open,0,,,0,2019-06-22T12:25:37Z,2019-06-22T12:25:37Z,,OWNER,,"This looks really promising: https://github.com/docker-slim/docker-slim If it can shave substantial size from our official container reliably we could add it to the automated build process.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/515/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 460095928,MDU6SXNzdWU0NjAwOTU5Mjg=,528,Establish a pattern for Datasette plugins built on top of Pandas,9599,open,0,,,0,2019-06-24T21:05:52Z,2019-06-24T21:05:52Z,,OWNER,,"The Pandas ecosystem is huge, varied and full of tools that are really good at doing interesting analysis on top of tabular data. Pandas should not be a dependency of Datasette core, but I think there is a lot of potential in having plugins which use Pandas to apply interesting analysis to data sucked out of Datasette's SQLite tables. One example ([thanks, Tony](https://twitter.com/psychemedia/status/1143259809715752962)): https://github.com/ResidentMario/missingno could form the basis of a fantastic plugin for getting a high-level overview of how complete each column in a table is. Some thought is needed here about what shape these kind of plugins might take, and what plugin hooks they would use.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/528/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 503053243,MDU6SXNzdWU1MDMwNTMyNDM=,582,Datasette should not completely crash if one SQLite database is malformed,9599,open,0,,,0,2019-10-06T05:11:43Z,2019-10-06T05:11:43Z,,OWNER,,"If you run Datasette against a number of database files and one of them is malformed, you get this 500 error on the index page: It would be better if Datasette still worked and listed the databases that were NOT malformed, then showed an inline error message just for the one that could not be accessed.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/582/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 505673645,MDU6SXNzdWU1MDU2NzM2NDU=,16,Do a better job with archived direct message threads,9599,open,0,,,0,2019-10-11T06:55:21Z,2019-10-11T06:55:27Z,,MEMBER,,https://github.com/dogsheep/twitter-to-sqlite/blob/fb2698086d766e0333a55bb73435e7283feeb438/twitter_to_sqlite/archive.py#L98-L99,206156866,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/16/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 530468212,MDU6SXNzdWU1MzA0NjgyMTI=,643,Set up some basic benchmarks as part of the unit tests,9599,open,0,,,0,2019-11-29T19:24:19Z,2019-11-29T19:24:19Z,,OWNER,,"https://pypi.org/project/pytest-benchmark/ looks great for this. Here's how to run it as a github action: https://github.com/rhysd/github-action-benchmark/blob/master/examples/pytest/README.md",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/643/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 581795570,MDU6SXNzdWU1ODE3OTU1NzA=,93,Support more string values for types in .add_column(),9599,open,0,,,0,2020-03-15T19:32:49Z,2020-09-24T20:36:46Z,,OWNER,,"https://sqlite-utils.readthedocs.io/en/2.4.2/python-api.html#adding-columns says: > SQLite types you can specify are ""TEXT"", ""INTEGER"", ""FLOAT"" or ""BLOB"". As discovered in #92 this isn't the right list of values. I should expand this to match https://www.sqlite.org/datatype3.html",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/93/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 593006814,MDU6SXNzdWU1OTMwMDY4MTQ=,715,Refactor duplicate cell display logic,9599,open,0,,,0,2020-04-03T00:58:11Z,2020-04-03T00:58:11Z,,OWNER,,"The logic for rendering cells in table view and in database (or canned query) view is currently very similar: https://github.com/simonw/datasette/blob/7656fd64d8b6a32ebc34d89c1b8711cc5ea240f7/datasette/views/base.py#L514-L539 Compared with: https://github.com/simonw/datasette/blob/7656fd64d8b6a32ebc34d89c1b8711cc5ea240f7/datasette/views/table.py#L104-L195 I'll be changing this a bit in #698 but I should still try to clean this up more further in the future.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/715/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 594237015,MDU6SXNzdWU1OTQyMzcwMTU=,718,Plugin idea: datasette-redirects,9599,open,0,,,0,2020-04-05T03:41:38Z,2023-08-30T22:17:31Z,,OWNER,,"I just had to write a one-off custom plugin to redirect niche-musems.com to www.niche-museums.com (https://github.com/simonw/museums/issues/21) - it would be great if this kind of thing could be handled by a configurable plugin. https://github.com/simonw/museums/blob/6b1faf00c463b2228860d4d62d104b11935e01b1/plugins/redirect_www.py",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/718/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,reopened 599776345,MDU6SXNzdWU1OTk3NzYzNDU=,24,Feature idea: github-to-sqlite everything ...,9599,open,0,,,0,2020-04-14T18:34:00Z,2020-04-14T18:34:00Z,,MEMBER,,"At the moment if you want to pull all your repos, issues, issues comments etc you have to do it with a sequence of separate commands. Consider adding a `everything` or `all` command which fetches everything that the tool knows how to fetch, and is designed to be run on a cron in a way that fetches just new stuff each time.",207052882,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/24/reactions"", ""total_count"": 7, ""+1"": 7, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 613491342,MDU6SXNzdWU2MTM0OTEzNDI=,762,Experiment with PRAGMA hard_heap_limit ,9599,open,0,,,0,2020-05-06T17:33:23Z,2020-05-07T03:08:44Z,,OWNER,,"This was added in SQLite 2020-01-22 (3.31.0): https://www.sqlite.org/changes.html#version_3_31_0 > Add the [sqlite3_hard_heap_limit64()](https://www.sqlite.org/c3ref/hard_heap_limit64.html) interface and the corresponding [PRAGMA hard_heap_limit](https://www.sqlite.org/pragma.html#pragma_hard_heap_limit) command. This sounds like it could be a nice extra safety measure.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/762/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 621486115,MDU6SXNzdWU2MjE0ODYxMTU=,27,photos_with_apple_metadata view should include labels,9599,open,0,,,0,2020-05-20T06:06:17Z,2020-05-20T06:06:17Z,,MEMBER,,"https://dogsheep-photos.dogsheep.net/public/photos_with_apple_metadata?place_city=New+Orleans&_facet=place_city&_facet_array=albums&_facet_array=persons Here's one way to add that: ```sql select rowid, photo, ( select json_group_array( json_object( 'label', normalized_string, 'href', '/photos/labelled?_hide_sql=1&label=' || normalized_string ) ) from labels where labels.uuid = photos_with_apple_metadata.uuid ) as labels, date, ```",256834907,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-photos/issues/27/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 626582657,MDU6SXNzdWU2MjY1ODI2NTc=,779,Make human_description_en explicitly available to output renderers,9599,open,0,,,0,2020-05-28T14:59:54Z,2020-05-28T14:59:54Z,,OWNER,,"`datasette-atom` uses this: https://github.com/simonw/datasette-atom/blob/df98a6c43a443224b6cd232f84703ec297ef046b/datasette_atom/__init__.py#L36-L37 ```python if data.get(""human_description_en""): title += "": "" + data[""human_description_en""] ``` It's a nice way to generate a useful title for a filtered table.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/779/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 638238548,MDU6SXNzdWU2MzgyMzg1NDg=,845,Code coverage should ignore files in .coveragerc,9599,open,0,,,0,2020-06-13T21:45:42Z,2020-06-13T21:46:03Z,,OWNER,,"I'm not sure why this is, but the code coverage I have running in a GitHub Action doesn't take my `.coveragerc` file into account. It should: https://github.com/simonw/datasette/blob/cf7a2bdb404734910ec07abc7571351a2d934828/.github/workflows/test-coverage.yml#L31-L35 Here's the bit that's ignored: https://github.com/simonw/datasette/blob/cf7a2bdb404734910ec07abc7571351a2d934828/.coveragerc#L1-L2 As a result my coverage score is 84%, when it should be 92%: ``` 2020-06-13T21:41:18.4404252Z ----------- coverage: platform linux, python 3.8.3-final-0 ----------- 2020-06-13T21:41:18.4404570Z Name Stmts Miss Cover 2020-06-13T21:41:18.4404971Z -------------------------------------------------------- 2020-06-13T21:41:18.4405227Z datasette/__init__.py 3 0 100% 2020-06-13T21:41:18.4405441Z datasette/__main__.py 3 3 0% 2020-06-13T21:41:18.4405668Z datasette/_version.py 279 279 0% 2020-06-13T21:41:18.4405921Z datasette/actor_auth_cookie.py 20 0 100% 2020-06-13T21:41:18.4406135Z datasette/app.py 499 27 95% 2020-06-13T21:41:18.4406343Z datasette/cli.py 162 45 72% 2020-06-13T21:41:18.4406553Z datasette/database.py 236 17 93% 2020-06-13T21:41:18.4406761Z datasette/default_permissions.py 40 0 100% 2020-06-13T21:41:18.4406975Z datasette/facets.py 210 24 89% 2020-06-13T21:41:18.4407186Z datasette/filters.py 122 7 94% 2020-06-13T21:41:18.4407394Z datasette/hookspecs.py 34 0 100% 2020-06-13T21:41:18.4407600Z datasette/inspect.py 36 23 36% 2020-06-13T21:41:18.4407807Z datasette/plugins.py 34 6 82% 2020-06-13T21:41:18.4408014Z datasette/publish/__init__.py 0 0 100% 2020-06-13T21:41:18.4408240Z datasette/publish/cloudrun.py 57 2 96% 2020-06-13T21:41:18.4408786Z datasette/publish/common.py 19 1 95% 2020-06-13T21:41:18.4409029Z datasette/publish/heroku.py 97 13 87% 2020-06-13T21:41:18.4409243Z datasette/renderer.py 63 4 94% 2020-06-13T21:41:18.4409450Z datasette/sql_functions.py 5 0 100% 2020-06-13T21:41:18.4410480Z datasette/tracer.py 87 16 82% 2020-06-13T21:41:18.4410972Z datasette/utils/__init__.py 504 31 94% 2020-06-13T21:41:18.4411755Z datasette/utils/asgi.py 264 24 91% 2020-06-13T21:41:18.4412173Z datasette/utils/shutil_backport.py 44 44 0% 2020-06-13T21:41:18.4412822Z datasette/version.py 4 0 100% 2020-06-13T21:41:18.4413562Z datasette/views/__init__.py 0 0 100% 2020-06-13T21:41:18.4414276Z datasette/views/base.py 288 19 93% 2020-06-13T21:41:18.4414579Z datasette/views/database.py 120 2 98% 2020-06-13T21:41:18.4414860Z datasette/views/index.py 57 2 96% 2020-06-13T21:41:18.4415379Z datasette/views/special.py 72 16 78% 2020-06-13T21:41:18.4418994Z datasette/views/table.py 418 18 96% 2020-06-13T21:41:18.4428811Z -------------------------------------------------------- 2020-06-13T21:41:18.4430394Z TOTAL 3777 623 84% ```",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/845/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 648659536,MDU6SXNzdWU2NDg2NTk1MzY=,881,Figure out why restore_working_directory is needed in some places,9599,open,0,,,0,2020-07-01T04:19:25Z,2020-07-01T04:19:25Z,,OWNER,,"This is a frustrating workaround. I have a `restore_working_directory` fixture that I wrote to solve errors that look like this: ``` /Users/simon/Dropbox/Development/datasette/tests/test_publish_cloudrun.py:148: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/local/opt/python/Frameworks/Python.framework/Versions/3.7/lib/python3.7/contextlib.py:112: in __enter__ return next(self.gen) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = @contextlib.contextmanager def isolated_filesystem(self): """"""A context manager that creates a temporary folder and changes the current working directory to it for isolated filesystem tests. """""" > cwd = os.getcwd() E FileNotFoundError: [Errno 2] No such file or directory ``` Here's an example of it in use: removing the `restore_working_directory` argument from this function causes the failure. https://github.com/simonw/datasette/blob/549b1c2063db48c4622ee5c7b478a1e3cbc1ac07/tests/test_plugins.py#L689-L690 I'd like to not have to do this.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/881/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 668064026,MDU6SXNzdWU2NjgwNjQwMjY=,911,"Rethink the --name option to ""datasette publish""",9599,open,0,,3268330,0,2020-07-29T18:49:49Z,2020-07-29T18:49:49Z,,OWNER,,`--name` works inconsistently across the different publish providers - on Cloud Run you should use `--service` instead for example. Need to review it across all of them and either remove it or clarify what it does.,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/911/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 673602857,MDU6SXNzdWU2NzM2MDI4NTc=,9,Define a view that displays photos correctly,9599,open,0,,,0,2020-08-05T14:53:39Z,2020-08-05T14:53:39Z,,MEMBER,,"The `photos` table stores data like this: id | createdAt | source | prefix | suffix | width | height | visibility | created ▲ | user -- | -- | -- | -- | -- | -- | -- | -- | -- | -- 5e12c9708506bc000840262a | January 06, 2020 - 05:45:20 UTC | Swarm for iOS 1 | https://fastly.4sqi.net/img/general/ | /15889193_AXxGk4I1nbzUZuyYqObgbXdJNyEHiwj6AUDq0tPZWtw.jpg | 1920 | 1440 | public | 2020-01-06T05:45:20 | 15889193 The photo URL can be derived from those pieces - define a SQL view which does that (using `datasette-json-html` to display the pictures)",205429375,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/9/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 675594325,MDU6SXNzdWU2NzU1OTQzMjU=,917,"Idea: ""datasette publish"" option for ""only if the data has changed",9599,open,0,,,0,2020-08-08T21:58:27Z,2020-08-08T21:58:27Z,,OWNER,,"This is a pattern I often find myself needing. I usually implement this in GitHub Actions like this: https://github.com/simonw/covid-19-datasette/blob/efa01c39abc832b8641fc2a92840cc3acae2fb08/.github/workflows/scheduled.yml#L52-L63 ```yaml - name: Set variables to decide if we should deploy id: decide_variables run: |- echo ""##[set-output name=latest;]$(datasette inspect covid.db | jq '.covid.hash' -r)"" echo ""##[set-output name=deployed;]$(curl -s https://covid-19.datasettes.com/-/databases.json | jq '.[0].hash' -r)"" - name: Set up Cloud Run if: github.event_name == 'workflow_dispatch' || steps.decide_variables.outputs.latest != steps.decide_variables.outputs.deployed uses: GoogleCloudPlatform/github-actions/setup-gcloud@master ``` This is pretty fiddly. It might be good for `datasette publish` to grow a helper option that does effectively this - hashes the databases (and the `metadata.json`) and compares them to the deployed version.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/917/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 688352145,MDU6SXNzdWU2ODgzNTIxNDU=,141,insert-files support for compressed values,9599,open,0,,,0,2020-08-28T20:59:46Z,2020-09-24T20:36:08Z,,OWNER,,"The `sqlar` format supports this, it would be useful if `insert-files` could support this too. https://www.sqlite.org/sqlar.html",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/141/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 689848827,MDU6SXNzdWU2ODk4NDg4Mjc=,6,ISO timestamps,9599,open,0,,,0,2020-09-01T06:16:42Z,2020-09-01T06:16:42Z,,MEMBER,,"The `time_added`, `time_updated` and `time_read` columns currently store data like this: September 19, 2019 - 00:30:30 UTC Should use ISO instead, e.g. `2020-07-26T01:05:24+00:00`",213286752,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/6/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 689850810,MDU6SXNzdWU2ODk4NTA4MTA=,6,Set up a demo instance,9599,open,0,,,0,2020-09-01T06:20:24Z,2020-09-01T06:20:24Z,,MEMBER,,"Once I've got the Datasette plugin to a state where it's worth building a demo: #3 I can use data from my public https://github-to-sqlite.dogsheep.net/ demo plus the Pocket data subset I use for the demo in https://github.com/dogsheep/pocket-to-sqlite/issues/5 - I could pull in the https://dogsheep-photos.dogsheep.net/ photos data too.",197431109,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-beta/issues/6/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 691537426,MDU6SXNzdWU2OTE1Mzc0MjY=,959,Internals API idea: results.dicts in addition to results.rows,9599,open,0,,,0,2020-09-03T00:50:17Z,2020-09-03T00:50:17Z,,OWNER,,"I just wrote this code: ```python results = await database.execute(SEARCH_SQL, {""query"": query}) return [dict(r) for r in results.rows] ``` How about having `results.dicts` as a utility property that does that?",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/959/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 692202408,MDU6SXNzdWU2OTIyMDI0MDg=,12,Idea: maps and GeoJSON support,9599,open,0,,,0,2020-09-03T18:47:10Z,2020-09-04T01:45:03Z,,MEMBER,,"It would be cool if the `display_sql` could return a column populated with GeoJSON which would the automatically be displayed on a map in the results (or maybe default JS would look for a `class=""geojson""` element output by the `display` template) - ala https://github.com/simonw/datasette-leaflet-geojson Then I could render workout routes on a map, or Swarm checkin points.",197431109,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-beta/issues/12/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 703216044,MDU6SXNzdWU3MDMyMTYwNDQ=,49,Feature: gists and starred gists,9599,open,0,,,0,2020-09-17T02:30:52Z,2020-09-17T02:30:52Z,,MEMBER,,https://developer.github.com/v3/gists/#list-starred-gists,207052882,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/49/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 703218448,MDU6SXNzdWU3MDMyMTg0NDg=,51,Documentation for twitter-to-sqlite fetch,9599,open,0,,,0,2020-09-17T02:38:10Z,2020-09-17T02:38:10Z,,MEMBER,,"It's mentioned in passing in the README but it deserves its own section: ``` $ twitter-to-sqlite fetch \ ""https://api.twitter.com/1.1/account/verify_credentials.json"" \ | grep '""id""' | head -n 1 ```",206156866,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/51/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 709789634,MDU6SXNzdWU3MDk3ODk2MzQ=,27,Sort order is not persisted by facet filter links,9599,open,0,,,0,2020-09-27T18:22:07Z,2020-09-27T18:22:07Z,,MEMBER,,A link to `/-/beta?category=1×tamp__date=2018-08-01&q=swedish` should be to `/-/beta?category=1×tamp__date=2018-08-01&q=swedish&sort=newest`,197431109,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-beta/issues/27/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 730210880,MDU6SXNzdWU3MzAyMTA4ODA=,1055,query.html and table.html should share the same table implementation,9599,open,0,,3268330,0,2020-10-27T07:58:21Z,2020-10-27T07:58:29Z,,OWNER,,In #998 I made a change that affected the table page but didn't affect the query page because I incorrectly assumed they shared rendering logic.,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1055/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 741231849,MDU6SXNzdWU3NDEyMzE4NDk=,1087,Idea: ?_extra=urls for getting back URLs to useful things,9599,open,0,,,0,2020-11-12T02:55:41Z,2021-12-15T18:06:16Z,,OWNER,,"Working on https://github.com/simonw/datasette-search-all/issues/10 made me realize that sometimes it can be difficult to calculate the URL for a database, table or row within Datasette. It would be useful to have an optional extra JSON extension (using `?_extra=` from #262) that can help with this.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1087/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 750089847,MDU6SXNzdWU3NTAwODk4NDc=,1109,Deprecate --config in Datasette 1.0 (in favour of --setting),9599,open,0,,3268330,0,2020-11-24T21:43:57Z,2020-12-17T22:07:49Z,,OWNER,,I added a deprecation warning to this in #992.,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1109/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 776128269,MDU6SXNzdWU3NzYxMjgyNjk=,1162,"First working version of ""datasette insert data.db file.csv""",9599,open,0,,,0,2020-12-29T23:20:11Z,2021-06-17T18:12:32Z,,OWNER,,Refs #1160,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1162/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 778530523,MDU6SXNzdWU3Nzg1MzA1MjM=,1172,/-/static should be excluded from auth and permission checks,9599,open,0,,,0,2021-01-05T02:53:41Z,2021-01-05T02:53:41Z,,OWNER,,"I want to set far future / immutable cache headers on everything served from `/-/static` and `/-/static-plugins` This has security implications since it will be possible to see what plugins are installed by checking for known static URLs. I'm fine with that - performance is more important here.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1172/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 816601354,MDExOlB1bGxSZXF1ZXN0NTgwMjM1NDI3,241,Extract expand - work in progress,9599,open,0,,,0,2021-02-25T16:36:38Z,2021-02-25T16:36:38Z,,OWNER,simonw/sqlite-utils/pulls/241,Refs #239. Still needs documentation and CLI implementation.,140912432,pull,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/241/reactions"", ""total_count"": 3, ""+1"": 3, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1, 836923194,MDU6SXNzdWU4MzY5MjMxOTQ=,32,JSON API for search results,9599,open,0,,,0,2021-03-20T22:21:36Z,2021-03-20T22:21:36Z,,MEMBER,,Refs https://github.com/simonw/datasette/issues/878,197431109,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-beta/issues/32/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 839367451,MDU6SXNzdWU4MzkzNjc0NTE=,1275,Idea: long-running query mode,9599,open,0,,,0,2021-03-24T05:23:20Z,2021-03-24T05:23:20Z,,OWNER,,"It would be cool if you could run Datasette in a long-running query mode, for use with trusted users - something like this: datasette --unlimited my.db This would disable the query limit, but would also enable a feature where if a query takes longer than e.g. 1s to return Datasette returns an HTML page to the browser with a progress indicator and polls the server until the query is complete.... but also provides the user with a ""cancel"" button. This relates to the `.interrupt()` research in #1270.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1275/reactions"", ""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 856895291,MDU6SXNzdWU4NTY4OTUyOTE=,1299,Design better empty states,9599,open,0,,,0,2021-04-13T12:06:12Z,2021-04-13T12:06:12Z,,OWNER,,Inspiration here: https://emptystat.es/,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1299/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 871304967,MDU6SXNzdWU4NzEzMDQ5Njc=,1315,"settings.json should be picked up by ""datasette publish cloudrun""",9599,open,0,,,0,2021-04-29T18:16:41Z,2021-04-29T18:16:41Z,,OWNER,,,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1315/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 897212458,MDU6SXNzdWU4OTcyMTI0NTg=,63,Ability to fetch commits from branches other than the default,9599,open,0,,,0,2021-05-20T17:58:08Z,2021-05-20T17:58:08Z,,MEMBER,,This tool is currently almost entirely ignorant of the concept of branches. One example: you can't retrieve commits from any branch other than the default (usually main).,207052882,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/63/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 915488244,MDU6SXNzdWU5MTU0ODgyNDQ=,1372,"Add section to ""writing plugins"" about security, e.g. avoiding XSS",9599,open,0,,,0,2021-06-08T20:49:33Z,2021-06-08T20:49:46Z,,OWNER,,https://docs.datasette.io/en/stable/writing_plugins.html should have tips on writing secure plugins.,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1372/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 924203783,MDU6SXNzdWU5MjQyMDM3ODM=,1379,Idea: ?_end=1 option for streaming CSV responses,9599,open,0,,,0,2021-06-17T18:11:21Z,2021-06-17T18:11:30Z,,OWNER,,"As discussed in this thread: https://twitter.com/simonw/status/1405554676993433605 - one of the disadvantages of Datasette's streaming CSV feature is that it's hard to tell if you got the whole file or if the connection ended early - or if an error occurred. Idea: offer an optional `?_end=1` parameter which, if enabled, adds a single row to the end of the CSV file that looks like this: `END,,,,,,,,,` For however many columns the CSV file usually has.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1379/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 925491857,MDU6SXNzdWU5MjU0OTE4NTc=,1383,Improve test coverage for `inspect.py`,9599,open,0,,,0,2021-06-20T00:22:43Z,2021-06-20T00:22:49Z,,OWNER,,https://codecov.io/gh/simonw/datasette/src/main/datasette/inspect.py shows only 36% coverage for that module at the moment.,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1383/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 953218043,MDU6SXNzdWU5NTMyMTgwNDM=,1403,Labels explaining what hidden tables are for,9599,open,0,,,0,2021-07-26T19:29:22Z,2022-03-21T22:20:37Z,,OWNER,,"A reasonable question: ""What are those hidden tables for?"" This could be answered by adding a small piece of explanatory text to each table - based on if it's related to FTS or to SpatiaLite or configured to be hidden for some other reason.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1403/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 957315684,MDU6SXNzdWU5NTczMTU2ODQ=,1410,Rename settings to `default_allow_facet` and `default_allow_download` and `default_allow_csv_stream`,9599,open,0,,3268330,0,2021-07-31T20:27:12Z,2021-07-31T20:27:49Z,,OWNER,,"> If I was prone to over-thinking (which I am) I'd note that `allow_facet` and `allow_download` and `allow_csv_stream` are all settings that do NOT have an equivalent in the newer permissions system, which is itself a little weird and inconsistent. > > So maybe there's a future task where I introduce those as both permissions and metadata `""allow_x""` blocks, then rename the settings themselves to be called `default_allow_facet` and `default_allow_download` and `default_allow_csv_stream`. > > If I was going to do that I should get it in before Datasette 1.0. _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1409#issuecomment-890400425_",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1410/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 957347860,MDU6SXNzdWU5NTczNDc4NjA=,1412,Mention WAL mode in the documentation (plus backup tips),9599,open,0,,,0,2021-08-01T00:27:11Z,2021-08-01T00:27:11Z,,OWNER,,"This is useful for people who are deploying Datasette with any write functionality, especially if they might be using something like EFS. I can add a section about this to the bottom of https://docs.datasette.io/en/stable/deploying.html and then link to that section from both https://docs.datasette.io/en/stable/sql_queries.html#writable-canned-queries and https://docs.datasette.io/en/stable/internals.html#await-db-execute-write-sql-params-none-block-false Also useful: mention that just copying a SQLite database file while it is being written to may not get a consistent file, so tell people to use one of the SQLite backup mechanisms instead.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1412/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 970626625,MDU6SXNzdWU5NzA2MjY2MjU=,1435,Turn off suggest facets on tables with large numbers of columns,9599,open,0,,,0,2021-08-13T18:30:48Z,2021-08-13T18:30:48Z,,OWNER,,If a table has 200 columns it will take multiple seconds to try and suggest facets. I should either quit after the first 20 or not suggest facets at all.,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1435/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 1006016302,I_kwDOBm6k_c479pcu,1477,Consider adding request to the documented default template context,9599,open,0,,,0,2021-09-24T02:34:09Z,2021-09-24T02:34:09Z,,OWNER,,I made a plugin for this today but I think perhaps it should be a default thing instead: https://datasette.io/plugins/datasette-template-request,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1477/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 1051277222,I_kwDOBm6k_c4-qTem,1504,Link to ?_size=max at bottom of table page,9599,open,0,,,0,2021-11-11T19:06:33Z,2021-11-11T19:06:33Z,,OWNER,,"This can have text such as ""Show 1,000 rows per page"", based on the max size limit setting. Would make it easier for people to see more data at once without having to know how to hack the URL, similar to the `...` for facet sizes I added in #1337.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1504/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 1056746091,I_kwDOBm6k_c4-_Kpr,1515,Handle foreign keys that point to a non-existent table,9599,open,0,,,0,2021-11-17T23:40:13Z,2021-11-18T01:31:56Z,,OWNER,,"Spotted in https://github.com/simonw/datasette-graphql/issues/79 Demo: https://datasette-graphql-demo.datasette.io/fixtures/bad_foreign_key The foreign key links to a 404 page. ![B87009C7-CFCA-4DF9-8FBA-FA3E6CA28EC2](https://user-images.githubusercontent.com/9599/142334788-4d1a4acd-bc87-4426-b333-d46b221afcec.jpeg) ",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1515/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 1059209412,I_kwDOBm6k_c4_IkDE,1523,Come up with a more elegant solution for base_url than ds.urls.path(),9599,open,0,,,0,2021-11-20T19:05:22Z,2021-11-20T19:05:22Z,,OWNER,,"While fixing #1519 I added a lot of ugly code that looks like this: https://github.com/simonw/datasette/blob/08947fa76433d18988aa1ee1d929bd8320c75fe2/datasette/facets.py#L228-L230 See these two commits in particular: fe687fd0207c4c56c4778d3e92e3505fc4b18172 and 08947fa76433d18988aa1ee1d929bd8320c75fe2 It would be great to come up with a less verbose and error-prone way of handling this problem.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1523/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 1067775061,I_kwDOBm6k_c4_pPRV,1539,Research PRAGMA query_only,9599,open,0,,,0,2021-11-30T23:30:24Z,2021-11-30T23:30:24Z,,OWNER,,"https://www.sqlite.org/pragma.html#pragma_query_only > The query_only pragma prevents data changes on database files when enabled. When this pragma is enabled, any attempt to CREATE, DELETE, DROP, INSERT, or UPDATE will result in an [SQLITE_READONLY](https://www.sqlite.org/rescode.html#readonly) error. However, the database is not truly read-only. You can still run a [checkpoint](https://www.sqlite.org/wal.html#ckpt) or a [COMMIT](https://www.sqlite.org/lang_transaction.html) and the return value of the [sqlite3_db_readonly()](https://www.sqlite.org/c3ref/db_readonly.html) routine is not affected. Would it be worth adding this as an extra protection against accidental writes to a DB file over a read-only connection?",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1539/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 1090798237,I_kwDOCGYnMM5BBEKd,359,Use RETURNING if available to populate last_pk,9599,open,0,,,0,2021-12-29T23:43:23Z,2021-12-29T23:43:23Z,,OWNER,,"Inspired by this: https://news.ycombinator.com/item?id=29729283 > Because SQLite is effectively serializing all the writes for us, we have zero locking in our code. We used to have to lock when inserting new items (to get the LastInsertRowId), but the newer version of SQLite supports the RETURNING keyword, so we don't even have to lock on inserts now.",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/359/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 1102568047,I_kwDOBm6k_c5Bt9pv,1596,Documentation page warning of changes coming in 1.0,9599,open,0,,,0,2022-01-13T23:26:04Z,2022-01-13T23:26:04Z,,OWNER,,I should start this relatively soon.,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1596/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 1102966378,I_kwDOBm6k_c5Bve5q,1599,Add architecture documentation,9599,open,0,,,0,2022-01-14T04:55:38Z,2022-01-14T04:56:03Z,,OWNER,,"Inspired by https://matklad.github.io/2021/02/06/ARCHITECTURE.md.html Good example: https://github.com/rust-analyzer/rust-analyzer/blob/d7c99931d05e3723d878bea5dc26766791fa4e69/docs/dev/architecture.md",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1599/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 1122450452,I_kwDOBm6k_c5C5zwU,1625,Try running tests against macOS and Windows in addition to Ubuntu,9599,open,0,,,0,2022-02-02T22:25:57Z,2022-02-02T22:25:57Z,,OWNER,,"I already do this for `sqlite-utils`: https://github.com/simonw/sqlite-utils/blob/3.22.1/.github/workflows/test.yml Related: - #1617 - #1545",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1625/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 1122557010,I_kwDOBm6k_c5C6NxS,1627,Get the tests passing against Windows,9599,open,0,,,0,2022-02-03T01:23:06Z,2022-02-03T01:23:32Z,,OWNER,,"> OK, the tests do NOT pass against Windows! https://github.com/simonw/datasette/runs/5044105941 > > _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1626#issuecomment-1028515161_",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1627/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 1125576543,I_kwDOBm6k_c5DFu9f,1630,Review datasette.utils and decide which functions should be documented for 1.0,9599,open,0,,3268330,0,2022-02-07T06:39:52Z,2022-02-07T06:39:52Z,,OWNER,,"Follows: - #1176",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1630/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 1142107925,I_kwDOBm6k_c5EEy8V,1638,`filters_from_request` plugin hook docs should mention that returning an async function is allowed,9599,open,0,,,0,2022-02-18T00:08:26Z,2022-02-18T00:08:26Z,,OWNER,,"https://docs.datasette.io/en/stable/plugin_hooks.html#filters-from-request-request-database-table-datasette doesn't mention that you can return an `async` function - but you can, and in fact Datasette itself uses that here: https://github.com/simonw/datasette/blob/aa7f0037a46eb76ae6fe9bf2a1f616c58738ecdf/datasette/filters.py#L43-L47",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1638/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 1148638868,I_kwDOBm6k_c5EdtaU,1639,Make datasette-redirect-forbidden unneccessary,9599,open,0,,,0,2022-02-23T22:18:46Z,2022-02-23T22:18:46Z,,OWNER,,"I wrote `datasette-redirect-forbidden` today because I needed 403 errors to redirect to `/-/login` and it was the quickest way to solve that problem. This should be a feature of Datasette core. - https://github.com/simonw/datasette-redirect-forbidden/issues/2",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1639/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 1175894898,I_kwDOBm6k_c5GFrty,1680,Consider simplifying permissions for 1.0,9599,open,0,,3268330,0,2022-03-21T20:17:29Z,2022-03-21T20:17:29Z,,OWNER,,"Permission checks right now can express one of three opinions: - `False` means ""so not grant this permisson"" - `True` means ""grant this permission"" - `None` means ""I have no opinion"" But... there's also a concept of a ""default"" for a given permission check, which might be `False` or `True`. I worry this is too complicated. Could this be simplified before 1.0? In particular the default concept. See also: - #1676 ",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1680/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 1181364043,I_kwDOBm6k_c5Gai9L,1687,Make show_json.html or a similar mechanism stable for plugins,9599,open,0,,,0,2022-03-25T23:42:45Z,2022-03-25T23:42:45Z,,OWNER,,"I used `show_json.html` in the new `datasette-packages` plugin, which means it will break if that template changes: - https://github.com/simonw/datasette-packages/issues/3 It would be useful if it (or something like it) was documented and stable for plugins to use. Also relevant: - #878",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1687/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 1196327155,I_kwDOBm6k_c5HToDz,1702,Be more consistent with column quoting,9599,open,0,,,0,2022-04-07T16:59:20Z,2022-04-07T16:59:20Z,,OWNER,,"This tutorial made me notice that Datasette is pretty inconsistent with how column quoting works: https://datasette.io/tutorials/learn-sql It has examples of each of `""table_name""` and `[table_name]` and `table_name`, and it uses single quoted values too. Datasette should generate SQL as consistently as possible to support learners. That tutorial should also provide a tiny bit of extra information about what's going on here.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1702/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 1197925865,I_kwDOBm6k_c5HZuXp,1704,File PRs against incompatible plugins pinning to datasette<1.0,9599,open,0,,3268330,0,2022-04-08T23:15:30Z,2022-04-08T23:15:30Z,,OWNER,,"As part of the preparation for the 1.0 release, test all existing known plugins against the alpha. For any that break, submit a PR suggesting they pin to a version <1.0 - and include a link to the documentation on how to upgrade the plugin for 1.0.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1704/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 1216436131,I_kwDOBm6k_c5IgVej,1721,"Implement plugin hooks: `register_table_extras`, `register_row_extras`, `register_query_extras`",9599,open,0,,8755003,0,2022-04-26T20:21:49Z,2022-12-13T05:29:07Z,,OWNER,,"Designed in: - #1720 Part of: - #262 - #1709",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1721/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 1216479167,I_kwDOBm6k_c5Igf-_,1722,`db.primary_keys()` and `db.table_columns()` don't show up in traces,9599,open,0,,,0,2022-04-26T21:08:36Z,2022-04-26T21:08:36Z,,OWNER,,"Noticed this while working on: - #1715 This code here isn't showing up in traces: https://github.com/simonw/datasette/blob/579f59dcec43a91dd7d404e00b87a00afd8515f2/datasette/views/table.py#L218-L220 Because those functions don't use the regular trace-instrumented `db.execute()` code path - they work directly against a connection instead: https://github.com/simonw/datasette/blob/579f59dcec43a91dd7d404e00b87a00afd8515f2/datasette/utils/__init__.py#L610-L626 ",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1722/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 1216622905,I_kwDOBm6k_c5IhDE5,1725,Performance question - what is happening in this gap?,9599,open,0,,,0,2022-04-27T00:21:11Z,2022-04-27T00:21:11Z,,OWNER,,"Trace from https://latest-with-plugins.datasette.io/github/commits?_facet=repo&_trace=1&_facet=committer ![CleanShot 2022-04-26 at 17 20 06@2x](https://user-images.githubusercontent.com/9599/165413811-db2cd599-2acc-46ce-b9c2-f9bc45b879e9.png) What's going on in that gap? Can I improve the tracing output to show some non-SQL queries to figure that out?",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1725/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 1217014076,I_kwDOBm6k_c5Iiik8,1726,Security page in the documentation,9599,open,0,,,0,2022-04-27T08:43:30Z,2022-04-27T08:43:30Z,,OWNER,,"A page talking about how to run Datasette securely, and security concerns to take into account.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1726/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 1219398983,I_kwDOBm6k_c5Iro1H,1730,SQL tracing should much more closely track the SQL query execution,9599,open,0,,,0,2022-04-28T22:41:04Z,2022-04-28T22:41:10Z,,OWNER,,"In #1727 I realized that the SQL tracing was measuring a whole bunch of stuff outside of the SQL query itself. I started experimenting with this fix for that but it didn't work - I got back an empty JSON array of traces for some reason: ```diff diff --git a/datasette/database.py b/datasette/database.py index ba594a8..d7f9172 100644 --- a/datasette/database.py +++ b/datasette/database.py @@ -7,7 +7,7 @@ import sys import threading import uuid -from .tracer import trace +from .tracer import trace, trace_child_tasks from .utils import ( detect_fts, detect_primary_keys, @@ -207,30 +207,31 @@ class Database: time_limit_ms = custom_time_limit with sqlite_timelimit(conn, time_limit_ms): - try: - cursor = conn.cursor() - cursor.execute(sql, params if params is not None else {}) - max_returned_rows = self.ds.max_returned_rows - if max_returned_rows == page_size: - max_returned_rows += 1 - if max_returned_rows and truncate: - rows = cursor.fetchmany(max_returned_rows + 1) - truncated = len(rows) > max_returned_rows - rows = rows[:max_returned_rows] - else: - rows = cursor.fetchall() - truncated = False - except (sqlite3.OperationalError, sqlite3.DatabaseError) as e: - if e.args == (""interrupted"",): - raise QueryInterrupted(e, sql, params) - if log_sql_errors: - sys.stderr.write( - ""ERROR: conn={}, sql = {}, params = {}: {}\n"".format( - conn, repr(sql), params, e + with trace(""sql"", database=self.name, sql=sql.strip(), params=params): + try: + cursor = conn.cursor() + cursor.execute(sql, params if params is not None else {}) + max_returned_rows = self.ds.max_returned_rows + if max_returned_rows == page_size: + max_returned_rows += 1 + if max_returned_rows and truncate: + rows = cursor.fetchmany(max_returned_rows + 1) + truncated = len(rows) > max_returned_rows + rows = rows[:max_returned_rows] + else: + rows = cursor.fetchall() + truncated = False + except (sqlite3.OperationalError, sqlite3.DatabaseError) as e: + if e.args == (""interrupted"",): + raise QueryInterrupted(e, sql, params) + if log_sql_errors: + sys.stderr.write( + ""ERROR: conn={}, sql = {}, params = {}: {}\n"".format( + conn, repr(sql), params, e + ) ) - ) - sys.stderr.flush() - raise + sys.stderr.flush() + raise if truncate: return Results(rows, truncated, cursor.description) @@ -238,9 +239,8 @@ class Database: else: return Results(rows, False, cursor.description) - with trace(""sql"", database=self.name, sql=sql.strip(), params=params): - results = await self.execute_fn(sql_operation_in_thread) - return results + with trace_child_tasks(): + return await self.execute_fn(sql_operation_in_thread) @property def size(self): ``` _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1727#issuecomment-1111602802_",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1730/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 1237871948,I_kwDOBm6k_c5JyG1M,1743,`datasette.utils.to_css_class()` should be a documented internal,9599,open,0,,,0,2022-05-16T23:57:26Z,2022-05-16T23:57:26Z,,OWNER,,"Because I'm using it in this plugin: - https://github.com/simonw/datasette-upload-dbs/issues/1",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1743/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 1251739062,I_kwDOBm6k_c5KnAW2,1752,Research if I can drop Janus,9599,open,0,,,0,2022-05-28T22:46:52Z,2022-05-28T22:46:52Z,,OWNER,,"> It seems to me Janus dependency is not necessary, `async with app.database_write_mutex(): out = await app.transaction(func)` may be enough. Comment here: https://lobste.rs/s/fki4tj/architecture_notes_datasette#c_a2ihon",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1752/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 1266329095,I_kwDOBm6k_c5LeqYH,1756,Mechanism for creating databases in WAL mode,9599,open,0,,,0,2022-06-09T15:39:28Z,2022-06-09T15:39:28Z,,OWNER,,"The `--create` option currently creates databases if they are missing, but does not enable WAL mode for them. It turns out WAL mode is useful for databases that are accepting writes! I think a `--create-wal` option that both creates them AND sets WAL mode on any that are created would be a good idea.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1756/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 1340900019,I_kwDOBm6k_c5P7IKz,1785,Can't use cog menu to facet by first column in a view,9599,open,0,,,0,2022-08-16T21:27:23Z,2022-08-16T21:27:23Z,,OWNER,,"https://latest.datasette.io/fixtures/paginated_view Compare with: ",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1785/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 1345561209,I_kwDOBm6k_c5QM6J5,1790,A better HTML title for canned query pages,9599,open,0,,,0,2022-08-21T18:27:46Z,2022-08-21T18:27:46Z,,OWNER,,"https://scotrail.datasette.io/scotrail/assemble_sentence?terms=This+train+is+formed+of%2Cbomb+which Current title is: `scotrail: with phrases as ( select key, value from json_each('["' || replace(:terms, ',', '","') || '"]')),matches as (select phrases.key, phrases.value, ( select File from announcements where announcements.Transcription like '%' || trim(phrases.value) || '%' order by length(announcements.Transcription) limit 1 ) as Filefrom phrases),results as ( select key, announcements.Transcription, announcements.mp3 from announcements join matches on announcements.File = matches.File order by key)select 'Combined sentence:' as mp3, group_concat(Transcription, ' ') as Transcription, -1 as keyfrom results unionselect mp3, Transcription, keyfrom resultsorder by key` I think a better title would be: `scotrail: assemble_sentence, terms = This train is formed of,bomb which`",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1790/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 1353088849,I_kwDOBm6k_c5Qpn9R,1795,Consider automatically cleaning up curly quotes in searches,9599,open,0,,,0,2022-08-27T16:35:25Z,2022-08-27T16:35:25Z,,OWNER,,"If your phone helpfully adds curly quotes for you then phrase searches against FTS won't work: “Rebecca Sugar” In regular (not `?_searchmode=raw` search mode Datasette could clean these up for you to help avoid that mistake.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1795/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 1359604075,I_kwDOCGYnMM5RCelr,481,"Idea: `sqlite-utils create-table tablename --sql ""select ...""`",9599,open,0,,,0,2022-09-02T01:41:24Z,2022-09-02T01:42:08Z,,OWNER,,"Could offer syntactic sugar for: ```sql create table foo as select * from bar ``` ``` sqlite-utils create-table data.db foo --sql ""select * from bar"" ``` https://sqlite-utils.datasette.io/en/stable/cli-reference.html#create-table",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/481/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 1378636455,I_kwDOBm6k_c5SLFKn,1815,"`datasette publish provider .` to publish whole directory, similar to configuration directory mode",9599,open,0,,,0,2022-09-19T23:28:59Z,2022-09-19T23:29:11Z,,OWNER,,"> I haven't done this with any of my other `datasette publish` tools, but I do think it's a good idea. Being able to publish the entire directory - with templates and plugins and metadata - does seem very useful to me. _Originally posted by @simonw in https://github.com/simonw/datasette-publish-fly/issues/23#issuecomment-1251673489_",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1815/reactions"", ""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 1396977994,I_kwDOBm6k_c5TRDFK,1830,Add documentation for writing tests with signed actor cookies,9599,open,0,,,0,2022-10-04T23:51:26Z,2022-10-04T23:51:26Z,,OWNER,,"I use this pattirn in a lot of plugin tests, e.g. https://github.com/simonw/datasette-edit-templates/blob/087f6a6cabc20020f2b0524f11aa3a7836320848/tests/test_edit_templates.py#L55-L58 ```python actor = ds.sign({""a"": {""id"": ""root""}}, ""actor"") response1 = await ds.client.get( ""/-/edit-templates/_footer.html"", cookies={""ds_actor"": actor} ) ``` I should add this to the documentation on this page: https://docs.datasette.io/en/latest/testing_plugins.html",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1830/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 1399933513,I_kwDOBm6k_c5TcUpJ,1833,Ability to submit long queries by POST,9599,open,0,,,0,2022-10-06T16:03:26Z,2022-10-06T16:18:00Z,,OWNER,,"Datasette doesn't limit URL lengths but some common web proxies do - the one in front of Google Cloud Run for example limits to 8KB total for incoming request headers: https://cloud.google.com/load-balancing/docs/quotas#https-lb-header-limits This means longer SQL queries can break! Need an optional mechanism for submitting queries by POST instead.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1833/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 1406860394,I_kwDOBm6k_c5T2vxq,1841,Drop format_bytes for Jinja filesizeformat filter,9599,open,0,,,0,2022-10-12T22:06:34Z,2022-10-12T22:06:34Z,,OWNER,,"Turns out this isn't necessary: https://github.com/simonw/datasette/blob/5aa359b86907d11b3ee601510775a85a90224da8/datasette/utils/__init__.py#L849-L858 I can use this instead: https://jinja.palletsprojects.com/en/3.1.x/templates/#jinja-filters.filesizeformat",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1841/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 1424980545,I_kwDOBm6k_c5U73pB,1861,"request.headers.get(""Content-Type"") fails",9599,open,0,,,0,2022-10-27T03:39:12Z,2022-10-27T03:39:12Z,,OWNER,,"Turns out this is case-sensitive, needs to be: request.headers.get(""content-type"") != ""application/json"" That's not great usability. It should be case insensitive.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1861/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 1447465004,I_kwDOBm6k_c5WRpAs,1889,Ability to create new tokens via the API,9599,open,0,,8755003,0,2022-11-14T06:21:36Z,2022-12-13T05:29:08Z,,OWNER,,"Refs: - #1850 Initially I decided that the API shouldn't be able to create new tokens at all - I don't like the idea of an API token holder creating themselves additional tokens. Then I realized that two of the API features are specifically more useful if you can generate fresh tokens via the API: - Tokes that expire after a time limit are MUCH more useful if they can be automatically generated - Likewise, tokens that are restricted to a subset of permissions (see #1855) make more sense to be generated like this, especially in conjunction with expiry times",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1889/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 1454532488,I_kwDOBm6k_c5WsmeI,1902,Document {% block crumbs %} for plugin authors,9599,open,0,,3268330,0,2022-11-18T06:16:30Z,2022-11-18T06:16:39Z,,OWNER,,"> For `datasette-copyable` I want to show breadcrumbs that take database/instance permissions into account, so I'm removing `{% block nav %}` entirely and replacing it with this: > > ```html+jinja > {% block crumbs %} > {{ crumbs.nav(request=request, database=database, table=table) }} > {% endblock %} > ``` _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1901#issuecomment-1319588163_ I should document this.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1902/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 1456013930,I_kwDOBm6k_c5WyQJq,1906,Extract publish Heroku support to a plugin,9599,open,0,,3268330,0,2022-11-19T00:02:51Z,2022-11-19T00:03:10Z,,OWNER,,"> This is a strong argument for extracting the Heroku support out to a plugin - it would allow this to be fixed with a plugin release without needing to push a full release of Datasette itself. _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1905#issuecomment-1320678715_ ",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1906/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 1468495358,I_kwDOBm6k_c5Xh3X-,1910,Check incoming column types on various write APIs,9599,open,0,,8755003,0,2022-11-29T18:09:10Z,2022-12-13T05:29:09Z,,OWNER,,"> I do think this needs type checking - I just tried and you really can send a string to an integer column and have it work, which feels bad. _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1863#issuecomment-1331089156_ ",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1910/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 1479920517,I_kwDOBm6k_c5YNcuF,1934,Return number of ignored/replaced items from /-/insert,9599,open,0,,3268330,0,2022-12-06T19:01:58Z,2022-12-06T19:02:03Z,,OWNER,,"Idea from here: - https://github.com/simonw/sqlite-utils/issues/516",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1934/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 1516815571,I_kwDOBm6k_c5aaMTT,1975,_col=id can cause id column to export twice in CSV export,9599,open,0,,,0,2023-01-03T00:25:15Z,2023-01-03T00:25:21Z,,OWNER,,"https://datasette.simonwillison.net/simonwillisonblog/blog_entry.csv?_col=id&_col=title&_col=body&_labels=on&_size=1 ```csv id,id,title,body 1,1,WaSP Phase II,""

The Web Standards project has launched Phase II.

"" ``` That should not have two `id` columns.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1975/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 1554032168,I_kwDOBm6k_c5coKYo,2002,Document how actors are displayed,9599,open,0,,,0,2023-01-24T00:08:49Z,2023-01-24T00:08:49Z,,OWNER,,"https://github.com/simonw/datasette/blob/e4ebef082de90db4e1b8527abc0d582b7ae0bc9d/datasette/utils/__init__.py#L1052-L1056 This logic should be reflected in the documentation on https://docs.datasette.io/en/stable/authentication.html#actors",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2002/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 1564774831,I_kwDOBm6k_c5dRJGv,2012,Missing space in database summary,9599,open,0,,,0,2023-01-31T18:01:13Z,2023-01-31T18:01:13Z,,OWNER,,"Spotted this on an instance index page: ",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2012/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 1616440856,I_kwDOJHON9s5gWO4Y,5,Configure full text search,9599,open,0,,,0,2023-03-09T05:20:46Z,2023-03-09T05:20:46Z,,MEMBER,,"FTS would be useful. Maybe even extract the plain text from the notes to make that index easier to create, rather than creating it against the HTML. Can use the `plaintext` property for that.",611552758,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/5/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 1617938730,I_kwDOJHON9s5gb8kq,9,"Default to just storing plaintext, store HTML if `--html` is passed",9599,open,0,,,0,2023-03-09T20:19:06Z,2023-03-09T20:19:06Z,,MEMBER,,"The full `body` version of the notes can get HUGE, due to embedded images. It turns out for my own purposes I'm usually happy with just the `plaintext` version. I'm tempted to say you don't get HTML unless you pass a `--html` option.",611552758,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/9/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 1636616315,I_kwDOBm6k_c5hjMh7,2042,Gather feedback on new ?_extra= design,9599,open,0,,,0,2023-03-22T23:07:43Z,2023-03-22T23:08:19Z,,OWNER,,"Now that I've landed: - #1999 See also: - #262 I want to get some feedback from people on the design of the new `?_extra=` feature, before freezing it into Datasette 1.0. The big change is that the default JSON representation is now MUCH slimmer - it only gives you keys for `""next""` and `""rows""`, where rows is a list of JSON objects (not a list of arrays as was previously the default) - for example https://latest.datasette.io/fixtures/sortable.json If you want extra stuff you can ask for it with the new `?_extra=` parameter - e.g. https://latest.datasette.io/fixtures/sortable.json?_extra=columns&_extra=suggested_facets You can use `?_extra=extras` to see a list of available extras: https://latest.datasette.io/fixtures/sortable.json?_extra=extras ",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2042/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",, 1641013220,I_kwDOBm6k_c5hz9_k,2045,First column on a view page has no facet option in cog menu,9599,open,0,,3268330,0,2023-03-26T18:02:47Z,2023-03-26T18:02:48Z,,OWNER,,"e.g. first column on this page - cog menu has no option to facet. https://datasette.io/content/tools ",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2045/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,